Application Frontend Tests

.github/workflows/ci.yml

@@ -90,3 +90,16 @@ jobs:
         if: ${{ !cancelled() && steps.environment.outcome == 'success' }}
         shell: bash
         run: $PYTHON scripts/run_tests.py benchmarks
+
+      - name: Run the screen scenarios (Linux)
+        id: screens
+        if: ${{ !cancelled() && steps.environment.outcome == 'success' && runner.os == 'Linux' }}
+        shell: bash
+        run: $PYTHON scripts/run_tests.py screens --screen-workers 2
+
+      - name: Keep what the failed screen scenarios left
+        if: ${{ failure() && steps.screens.outcome == 'failure' }}
+        uses: actions/upload-artifact@v7
+        with:
+          name: screens-${{ matrix.os }}-py${{ matrix.python }}
+          path: build/screens/

.pre-commit-config.yaml

@@ -145,3 +145,13 @@ repos:
         stages:
           - pre-push
         pass_filenames: false
+
+      - id: screens
+        name: screens
+        entry: make screens
+        language: system
+        types:
+          - python
+        stages:
+          - pre-push
+        pass_filenames: false

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help setup install system-deps build release run calibration tracker-playback clean pre-commit test test-docs benchmarks lint format
+.PHONY: help setup install system-deps build release run calibration tracker-playback clean pre-commit test test-docs benchmarks screens lint format
 
 ifeq ($(OS),Windows_NT)
 PYTHON := python
@@ -25,6 +25,7 @@ help:
 	@echo $(Q)  make test        - Run the test suite with coverage$(Q)
 	@echo $(Q)  make test-docs   - Run the doctests$(Q)
 	@echo $(Q)  make benchmarks  - Run the measured-duration suite$(Q)
+	@echo $(Q)  make screens     - Run the screen scenarios: the application driven on a virtual display (needs Xvfb)$(Q)
 	@echo $(Q)  make calibration - Measure reconstruction on the reference sounds; writes the renders and a report$(Q)
 	@echo $(Q)  make tracker-playback BITPHASE=folder FAMITRACKER=FamiTracker.exe [PROJECT=song.stp] - Play exported projects (the corpus by default) in Bitphase, FamiTracker or both; reports every tick that differs from the app$(Q)
 	@echo $(Q)  make clean       - Remove build artifacts and cache files$(Q)
@@ -74,6 +75,9 @@ test-docs:
 benchmarks:
 	$(PYTHON) scripts/run_tests.py benchmarks
 
+screens:
+	$(PYTHON) scripts/run_tests.py screens
+
 lint:
 	$(PYTHON) scripts/lint.py $(ARGS)
 

docs/development/bugs-and-todos.md

@@ -80,6 +80,10 @@ dimension the import starts carrying.
   `referee/test_axioms.py` does for the referees.
 * API documentation
 * Code documentation
+* Screen scenarios on Windows and macOS. They run on Linux alone: a scenario's pointer and keys reach the
+  application through X11, and whether DearPyGui opens its window on GitHub's Windows and macOS runners is
+  unverified. A run there starts from a spike that opens the application on each runner and presses one
+  control through the callback a click runs.
 * What a build makes of the configuration and the session state an older one left behind. Neither has a
   version, so neither travels an upgrade chain or has an archived corpus. A `state.yaml` naming a panel
   that has since gone, or a configuration missing a setting added since, is read by whatever each loader
@@ -146,3 +150,84 @@ currently out of line. An entry leaves when the code meets the contract again.
   moved.
 
 ## Bugs
+
+* Leaving the application on a machine that offers no output device fails. The shutdown saves the
+  session and asks the audio device manager for its current device, and with no device selected it
+  raises `ValueError: No audio device selected`. The screen scenario in `tests/screens/application/leaving/test_exit_shortcut.py`
+  reproduces it as a known failure.
+* A project stating a data version no upgrade step reaches is refused by its shape, not by its version.
+  `ProjectContainer.load` validates the document before `_validate_document` compares the version, so
+  the user reads a list of validation errors instead of the version mismatch a reconstruction reports.
+  `tests/integration/compatibility/test_project.py` and the screen scenario in
+  `tests/screens/application/old_files/test_broken_projects.py` reproduce it as known failures.
+* Leaving keeps the last dialog folder a run found deleted. A dialog opens in the nearest folder still
+  standing, while `state.yaml` goes on naming the deleted one. `tests/screens/application/restart/test_deleted_files.py`
+  reproduces it as a known failure.
+* Leaving keeps a starred file a run found deleted: `config.yaml` goes on naming it among the favorites.
+  `tests/screens/application/restart/test_deleted_files.py` reproduces it as a known failure.
+* A folder asked for while a stopped read winds down is dropped without a word. Stop closes the scan
+  window at once while the walk runs on to its next entry, and `FolderScan.start` turns away the folder
+  asked for in that time, though it promises that a folder asked for once the window closes is read.
+  `tests/screens/main/scan/test_reading_a_folder.py` reproduces it as a known failure.
+* A box clicked in the Converter list leaves Source settings where it stood: ticking a channel on a
+  recording inside an open folder changes that row, while the card goes on naming the row picked
+  before, or New recordings. `tests/screens/main/row_settings/test_boxes.py` reproduces it as a known failure.
+* General settings stops short of the Converter's right edge while Advanced settings is put away,
+  where it should fill the row and end where the Converter below it does.
+  `tests/screens/main/cards/test_card_layout.py` reproduces it as a known failure.
+* The Destination line names no folder a run writes into once the gathered recordings convert with
+  different channels: it names the folder of every channel the rows use together, while each
+  recording goes into the folder of its own channels. `tests/screens/main/run/test_destination.py` reproduces it as
+  a known failure.
+* A library folder pointed away from and back lists its library unloaded: the library loaded before
+  reads as one that exists, where it should come back loaded.
+  `tests/screens/main/library/test_folders_and_generators.py` reproduces it as a known failure.
+* A reconstruction whose file the browser removed reads as a sample of the project: the NES frequency
+  field locks with the hint that the project sets its rate, though the document belongs to no
+  project. `tests/screens/prompts/vanished/test_reconstruction_removed.py` reproduces it as a known failure.
+* A voice double-clicked with the second press still held opens edited. The double-click brings the
+  Reconstructions tab forward while the button is down, and the envelope graph that comes under the
+  pointer draws a bar for a press that began on the Voices card. `tests/screens/prompts/open_voice/test_open_voice.py`
+  reproduces it as a known failure.
+* A Keyboard settings row reads as listening once Cancel answers the reassign question, while no key
+  reaches it: the keys pressed next are taken by nothing, and Escape closes the dialog.
+  `tests/screens/prompts/modals/test_run_ending_behind_a_dialog.py` reproduces it as a known failure.
+* Two closes before the first is answered ask twice: each close puts its question in line, so Cancel
+  on the first brings the second, and so does a close made twice while an edit is on its way.
+  `tests/screens/prompts/closing/test_over_a_question.py` and
+  `tests/screens/prompts/closing/test_during_an_edit.py` reproduce it as known failures.
+* A reconstruction whose recording is missing draws a flat original line beside the reconstruction,
+  where the waveform shows the approximation on its own. `tests/screens/reconstructions/player/test_source_switch.py`
+  reproduces it as a known failure.
+* With an instrument open on the Reconstructions tab, the note keys take Ctrl+Z, Ctrl+S and every
+  combination ending in a note key: the instruments panel answers a key whatever modifiers are held, so
+  Undo, Save and the rest never reach their shortcuts. `tests/screens/reconstructions/instruments/test_note_keys.py`
+  reproduces it as a known failure.
+* Playing the song from a tracker row leaves the Playback menu reading Play, with Stop greyed out, while
+  the song plays: that path never refreshes the menu. `tests/screens/sequencer/tracker/test_notes_typed.py` reproduces
+  it as a known failure.
+* New instrument with no project open writes into a project nobody opened: the Voices card's button has
+  no open-project guard, so the instrument is added and listed, while Voice ▸ New instrument stands
+  greyed out. `tests/screens/sequencer/voices/test_voices_card.py` reproduces it as a known failure.
+* The history lines of a renamed or a moved voice name it one way alone: a rename names the voice and
+  not its position, and a move names the positions and not the voice.
+  `tests/screens/sequencer/history/test_voice_gestures.py` reproduces it as a known failure.
+* A project opened as the application starts is saved as though it had no file: `load_project_safely`
+  leaves the session's current project unset, so Save asks for a path, or writes to whatever path an
+  earlier session left. `tests/screens/sequencer/song/test_retuning_and_saving.py` reproduces it as a known failure.
+* A voice row's hover can log an error: the Voices list rebuilds every row on each update, and a hover
+  callback queued for a row before the rebuild reads an item that no longer exists
+  (`_on_row_hovered`, "Item not found"). The Sequencer's screen scenarios forgive it by name; no
+  scenario reproduces it on demand, since it rests on the order the queued callbacks run in.
+* Export instrument... in a project whose samples were converted at two tunings does nothing the user
+  can see: `voice_instrument` raises the tuning error inside the menu's callback, so no message, no save
+  dialog and no file follow, while a Bitphase project or an NSF program of the same project stops with a
+  message. `tests/screens/exports/progress/test_refusals.py` reproduces it as a known failure.
+* Closing the window while Display settings holds a window size kept on the countdown but never confirmed
+  writes that size: leaving records the live window size, while the dialog keeps the session at the values
+  it opened with until OK, and the window manager's close passes the open dialog by.
+  `tests/screens/interface/display/test_kept_size_at_close.py` reproduces it as a known failure.
+* Closing the window while a folder is being read crashes the process once the read ends: nothing stops the
+  walk on exit, so it runs past the shutdown, and its report closes the reading window through DearPyGui
+  after the context is gone (SIGSEGV). `tests/screens/application/closing/test_during_work.py` reproduces it as a
+  known failure.

docs/development/guidelines.md

@@ -90,6 +90,7 @@ These rules govern the Python in this repository. They complement
 
 1. A test file mirrors the ownership of the code it exercises.
 1. When functionality moves between packages, move its direct unit tests in the same change.
+1. **What only a drawn frame shows is proven by a screen scenario.** A behavior is proven at the lowest tier that sees its outcome, and a scenario in `tests/screens/` covers what a running, rendered application adds: geometry, frame timing, real input and the exit. `make screens` runs the scenarios, and [screen scenarios](application/screens.md) says how they are written.
 1. **A test whose assertion is a measured duration lives in `tests/benchmarks/`.** The gated suite runs across several workers and under coverage, which multiplies the cost of the code being measured. Benchmarks run in a pass of their own, serial and uncovered, where the reading is the code's own cost. `make test` runs the covered suite, and `make benchmarks` runs the measured pass.
 1. Parametrize tests that share a body, using a test-case dataclass.
 1. Test case classes and cases themselves should be defined inside the testing class, unless these objects are shared between test classes. A suite inherits from `BaseTestSuite` and names its case class `TestCase`, which inherits from `BaseRegularTestCase`, or from `BaseAutolabelTestCase` where the case derives its own label. The parametrized argument carries the case as `test_case`.

docs/index.md

@@ -87,6 +87,7 @@ whole repository. The pages about the graphical application and about releases e
 - [Reconstruction browser](development/application/browser.md) — how a reconstructions directory becomes the tree both browser tabs show, and what narrows it.
 - [Stems in the application](development/application/stems.md) — the Stems card, and what an edit or a removal does to the per-frame record.
 - [Editing the open reconstruction](development/application/reconstruction-edits.md) — how the Reconstructions tab takes the reader's edits one step at a time, and what waits for them.
+- [Screen scenarios](development/application/screens.md) — tests that run the whole application on a display and press its controls the way a user does.
 - [Configuration](development/application/config-organization.md) — how the YAML configuration package is laid out.
 
 ### Releases

pyproject.toml

@@ -89,7 +89,10 @@ dev = [
     "pytest==9.1.1",
     "pytest-cov==7.1.0",
     "pytest-xdist==3.8.0",
+    "python-xlib==0.33",
+    "pyvirtualdisplay==3.0",
     "types-PyYAML==6.0.12.20260518",
+    "types-python-xlib==0.33.0.20260724",
 ]
 
 [build-system]

scripts/bootstrap/layout.py

@@ -5,6 +5,7 @@
 PROJECT_FILE: Final[str] = "pyproject.toml"
 SOURCE_DIRECTORY: Final[str] = "src"
 BENCHMARKS_DIRECTORY: Final[str] = "tests/benchmarks"
+SCREENS_DIRECTORY: Final[str] = "tests/screens"
 DISTRIBUTION: Final[str] = "bin"
 BUNDLES: Final[str] = "bundles"
 BUILD_ENVIRONMENT: Final[str] = ".venv-build"

scripts/bootstrap/platforms/linux.py

@@ -24,6 +24,8 @@
     "libxrandr2",
     "libxrender1",
     "libxxf86vm1",
+    "libgl1-mesa-dri",
+    "xvfb",
 )
 BUNDLING: Final[Bundling] = Bundling(
     icon=PNG_ICON,

scripts/run_tests.py

@@ -3,27 +3,34 @@
 import sys
 from typing import Dict, Final, Sequence, Tuple
 
-from bootstrap.layout import BENCHMARKS_DIRECTORY, SOURCE_DIRECTORY, repository_root
+from bootstrap.layout import BENCHMARKS_DIRECTORY, SCREENS_DIRECTORY, SOURCE_DIRECTORY, repository_root
 from bootstrap.passes import Pass, run_pass
 from bootstrap.processes import run
 
 SUITE: Final[str] = "suite"
 DOCTESTS: Final[str] = "doctests"
 BENCHMARKS: Final[str] = "benchmarks"
+SCREENS: Final[str] = "screens"
 DEFAULT_WORKERS: Final[str] = "6"
+SCREEN_WORKERS: Final[str] = "4"
 PYTEST: Final[Tuple[str, ...]] = ("uv", "run", "python", "-m", "pytest")
 
 
-def planned_passes(workers: str) -> Dict[str, Pass]:
-    """The passes a test run is made of, by name: the covered suite, the doctests, the benchmarks.
+def planned_passes(workers: str, screen_workers: str) -> Dict[str, Pass]:
+    """The passes a test run is made of, by name: the covered suite, the doctests, the benchmarks
+    and the screen scenarios.
 
     Each pass is a target and a hook of its own, so a failure names the pass it belongs to. The
     covered suite runs across ``workers`` pytest workers. The benchmarks run serial, uncovered
     and with their output shown, so a measured duration is the code's own cost and its reading
-    reaches the terminal.
+    reaches the terminal. The screen scenarios drive the running application on a display of
+    each worker's own, each scenario in a process of its own, so they run uncovered across
+    ``screen_workers`` workers, each drawing its frames on the processor.
 
     Args:
         workers: The worker count for the covered suite, or ``auto`` for one per processor.
+        screen_workers: The worker count for the screen scenarios, each worker drawing one
+            application at a time.
 
     Returns:
         Dict[str, Pass]: Every pass, under its name.
@@ -32,7 +39,14 @@ def planned_passes(workers: str) -> Dict[str, Pass]:
         Pass(
             SUITE,
             "Running pytest with coverage...",
-            (*PYTEST, "-n", workers, "--cov", f"--ignore={BENCHMARKS_DIRECTORY}"),
+            (
+                *PYTEST,
+                "-n",
+                workers,
+                "--cov",
+                f"--ignore={BENCHMARKS_DIRECTORY}",
+                f"--ignore={SCREENS_DIRECTORY}",
+            ),
         ),
         Pass(
             DOCTESTS,
@@ -44,23 +58,37 @@ def planned_passes(workers: str) -> Dict[str, Pass]:
             "Running benchmarks...",
             (*PYTEST, BENCHMARKS_DIRECTORY, "--no-cov", "-s"),
         ),
+        Pass(
+            SCREENS,
+            "Running screen scenarios...",
+            (*PYTEST, SCREENS_DIRECTORY, "-n", screen_workers, "--no-cov"),
+        ),
     )
     return {current.name: current for current in passes}
 
 
 def main(argv: Sequence[str]) -> int:
     """Runs one pass of the tests and exits with the status pytest gave it."""
     parser = argparse.ArgumentParser(description="Run one pass of the SampleToNES tests.")
-    parser.add_argument("name", choices=tuple(planned_passes(DEFAULT_WORKERS)), help="the pass to run")
+    parser.add_argument(
+        "name",
+        choices=tuple(planned_passes(DEFAULT_WORKERS, SCREEN_WORKERS)),
+        help="the pass to run",
+    )
     parser.add_argument(
         "--workers",
         default=DEFAULT_WORKERS,
         help="pytest workers for the covered suite: a count, or auto for one per processor",
     )
+    parser.add_argument(
+        "--screen-workers",
+        default=SCREEN_WORKERS,
+        help="pytest workers for the screen scenarios, each drawing one application at a time",
+    )
     arguments = parser.parse_args(list(argv))
 
     return run_pass(
-        planned_passes(arguments.workers)[arguments.name],
+        planned_passes(arguments.workers, arguments.screen_workers)[arguments.name],
         root=repository_root(),
         runner=run,
         environment=os.environ,

src/sampletones_application/ui/elements/tree/tree.py

@@ -500,10 +500,19 @@ def _create_hover_callback(
         self,
         status_bar_callback: Optional[MessageCallback],
     ) -> Callback:
+        """The hover callback of a row, which names the row in the status bar and its detail tooltip.
+
+        The hover is reported a frame after it happened, by which time a rebuilt tree may have taken
+        the row away, so the callback answers for the rows still standing.
+        """
+
         def hover_callback(
             _sender: Sender,
             app_data: int,
         ) -> None:
+            if not dpg.does_item_exist(app_data):
+                return
+
             user_data = dpg.get_item_user_data(app_data)
             if status_bar_callback is not None:
                 self._status_bar.set(status_bar_callback, user_data=user_data)

src/sampletones_application/ui/panels/sequencer/voices/panel.py

@@ -145,8 +145,13 @@ def _on_row_hovered(self, _sender: Sender, app_data: int) -> None:
         """Says what the hovered row holds, which the id cell and the name cell both address.
 
         Both cells carry the row's position and its voice, so one handler covers the whole row and
-        a reader reads the same line wherever the pointer rests on it.
+        a reader reads the same line wherever the pointer rests on it. The hover is reported a frame
+        after it happened, by which time a rebuilt list may have taken the cell away, so the handler
+        answers for the cells still standing.
         """
+        if not dpg.does_item_exist(app_data):
+            return
+
         user_data = dpg.get_item_user_data(app_data)
         if not isinstance(user_data, tuple):
             return

src/sampletones_application/utils/callbacks/queue.py

@@ -74,6 +74,12 @@ def notify_frame(cls) -> None:
         with cls._lock:
             cls._frame_counter += 1
 
+    @classmethod
+    def current_frame(cls) -> int:
+        """The frame the counter stands at, which a delay given to :meth:`add` counts from."""
+        with cls._lock:
+            return cls._frame_counter
+
     @classmethod
     def process(cls, budget_seconds: float) -> None:
         """Run the callbacks due at the current frame, up to a time budget.