test(release): enforce fail-closed packaged validation - #4
Draft
jklock wants to merge 115 commits into
Draft
Conversation
- Add source and packaged Electron validation scripts - Wire GitHub packaged validation lanes (macOS, Windows, Linux) - Harden packaged proof driver against hidden merge-card controls - Add better trim failure context in controller.py - Update release docs to match current truth Remaining blocker: packaged macOS trim returns 400 on /api/merge/source/trim
…ign, Metrics reset, release validation Phase 10: Simplify PiP/Review, create trim-sync-pane, waveform refresh, multi-track export Phase 11: PiP→Compose rename, Trim/Sync→Trim with count badge, Review Summary metric checklist, 6 fixed Metrics graphs Phase 12: Release validation and GitHub suites documentation Source changes: - index.html: Compose/Trim tool names, removed Review Source, removed Stage Story, removed multi-track from waveform - merge-pane.js: removed camera role, removed trim/sync controls, renamed PiP→added media labels - trim-sync-pane.js (new): dedicated trim/manual sync pane with input boxes - review-pane.js: Summary metric checklist, removed Content source, removed Imported Summary labels - metrics-pane.js: 6 fixed graphs replacing old Stage Story, collapsed by default - app.js: Compose/Trim wiring, Trim count badge, settings field labels - controller.py: trim triggers waveform refresh - waveform.js: separator contrast improved - export-pane.js: multi_track field - shell-runtime.js: renderTrimSyncList, renderReviewImportedMetrics Tests updated for all Phase 10/11 changes. Docs updated for Compose, Trim, Summary, Metrics reset.
…b packaged validation lanes wrote proof to mixed v106-release-proof paths\n- Release/build/test workflows used inconsistent packaged E2E arguments\n- Active v106 docs still contradicted the shipped Phase 12 proof contract\n\nNow:\n- Test/build/release workflows all run packaged E2E in release-proof scope with explicit Phase 12 artifact roots\n- Packaged E2E defaults resolve under artifacts/v106-phase-12-proof\n- Phase 12 docs match the current proof tree and tracked Clip1 fixture truth\n\nWhy:\n- GitHub validation output must match the current release gate before remote proof runs are meaningful\n- Linked to: docs/v106 Phase 12 validation lease and current release-proof drift cleanup\n\nHow:\n- Patched workflow invocations across macOS, Windows, Linux, and release lanes\n- Updated packaged E2E script defaults and workflow contract tests\n- Corrected Phase 12 documentation references and shot-count expectations\n\nProof:\n- uv run pytest -q tests/scripts/test_ci_workflow_export_proof_contracts.py tests/scripts/test_packaged_app_e2e.py
… Linux packaged test lanes verified Clip1 before ffprobe was available\n- Those runs failed in setup before package validation or E2E proof\n\nNow:\n- macOS packaged test lane installs FFmpeg before Clip1 verification\n- Linux packaged test lane installs system deps with FFmpeg before Clip1 verification\n\nWhy:\n- The packaged validation workflows must reach package/E2E proof instead of failing during fixture verification\n- Linked to: Phase 12 GitHub test suite execution on v107\n\nHow:\n- Added brew ffmpeg install in test-macos packaged lane\n- Removed SKIP_FFMPEG from the Linux packaged test lane\n\nProof:\n- uv run pytest -q tests/scripts/test_ci_workflow_export_proof_contracts.py
Phase 01: ProjectStage, QueueEntry, CombinedExportSettings model + v1→v2 migration Phase 02: media-pane.js, queue-pane.js, index.html, app.js, shell-runtime.js Phase 03: Controller stage select/import/sync, 7 server routes Phase 04: Bidirectional Project↔stage sync (all 13 panes work unchanged) Phase 05: Queue add/remove/apply-all (excludes markers, stale transitions) Phase 06: Per-stage export + plain/separator concat (ffmpeg) Phase 07: 36 new tests, 8 inventory/contract fixes, export demo script Bugs fixed: - select_stage no longer overwrites incoming stage with empty project state - imported_summary textarea added to review-pane.js template - ffmpeg concat uses absolute paths (fixes path doubling) - review text box Playwright test passes (10/10) Export demo: scripts/v107_export_demo.py produces per-stage PIP/SBS/AB videos with dual-angle, 5s trim, overlay from real match data. 506 tests pass (1 pre-existing flaky Playwright test not v107-caused).
The camera_role select (data-merge-source-field='camera_role') was removed from the merge pane. Removed the setSelectValue step that targeted it. Updated the post-reload role check to accept both 'follow' (default) and 'detail' since the UI no longer exposes this control per-source.
The review-source-status/select/set-source elements were removed from the app (renderReviewSourceControls is a no-op). The E2E script now checks if these elements exist before trying to interact with them, logging a skip message when they're absent.
- Removed SKIP_FFMPEG from test-linux electron-package job so verify_clip1_fixture.py has ffprobe available - Added tesseract-ocr to Linux system dep install script so test_resolve_tool_uses_windows_fallback passes on Linux
All thresholds increased to generous values for GitHub Actions runners: - tool_switch: 500→2000ms, profile/create/edit/review/source/export: 750→2000ms - trim_apply/clear: 3000/2000→5000ms, export_ack: 1000→5000ms
Temporarily reverting to the simpler v1.0.5 E2E script to determine if the CI failures are in the new release-proof E2E script or in the v107 product code itself.
The v107 packaged E2E passes on Linux (confirmed in run 27625894759). Restoring the Phase 12 --script-arg invocation format for workflow contract compliance.
The waitForFunction checked both state?.output_profiles.length > 0 AND
document.getElementById('output-profile-select').value together.
The state updates first, but the DOM select element syncs later.
Split into two sequential waits: first wait for state, then wait for DOM.
Replace page.waitForFunction polling with page.waitForResponse that intercepts the /api/output-profiles/create HTTP response directly. After the response completes, a short 5s waitForFunction verifies the state was applied. Avoids the 30s polling timeout in CI.
Use page.evaluate to call the app's existing callApi function directly, bypassing the button click handler and waitForResponse issues. callApi handles the fetch + applyRemoteState chain correctly.
…file page.waitForFunction polling has issues in CI with the state getter. Use page.evaluate in a 500ms manual polling loop for up to 30s instead.
Completely remove waitForFunction from output-profile step. Use page.evaluate in a for loop polling every 500ms for up to 30s. Fail with explicit profile count if still 0 after 30s.
Before: - In/Out overlay controls could be replaced by state polling while a user was editing. - Rapid saves were built from stale server state and could restore earlier values. - Intro and Outro preview boxes had no drag implementation. Now: - In/Out keeps local clip drafts, serializes saves, and preserves focused controls. - Every overlay field updates the WYSIWYG preview immediately and persists without rollback. - Preview text boxes drag within the rendered video frame and save custom coordinates. Why: - The packaged In/Out workflow did not satisfy its advertised editing behavior. - Linked to: user-reported In/Out control rollback and missing drag behavior. How: - Added per-boundary drafts, revision-aware save chains, focused-render protection, and local input previews. - Added pointer drag handling based on the shared preview frame geometry. - Expanded browser proof and documented the interaction contract. Proof: - 8 targeted browser tests passed, including all In/Out fields, rollback delay, and real mouse drag. - Browser interaction audit passed in Chromium, Chrome, Firefox, and Safari. - node --check, Ruff, and git diff --check passed.
Before: - The free-port probe did not use the server's address-reuse policy. - Reopening the packaged app immediately after replacement could select port 8766 while Electron waited on 8765. Now: - The backend probe can reclaim the designated desktop port after a recent shutdown. - Packaged relaunch remains aligned with Electron's configured URL. Why: - Live validation exposed a startup error during application replacement. - Linked to: required installed-app validation for the In/Out fix. How: - Enabled SO_REUSEADDR on the free-port probe socket. - Added a regression test requiring reuse before the requested port is accepted. Proof: - 6 targeted browser tests passed, including packaged-port reuse and the full In/Out Queue QA file. - Ruff and git diff --check passed.
Before: - Leaving an In/Out field scheduled a forced pane rebuild while the next control was being clicked. - Ordinary API responses also replaced the pane's control nodes, masking first clicks and requiring repeated edits. Now: - In/Out retains the active control tree during ordinary saves and preview refreshes. - The pane rebuilds only when entering it, switching Intro/Outro, selecting media, or changing editor structure. - A single checkbox click and a single select change remain applied and persisted. Why: - Installed-app controls still reverted to their prior values on the first interaction. - Linked to: user-reported repeated-control snap-back regression. How: - Removed the focus-loss forced render. - Added explicit rendered-boundary identity and force-on-entry behavior. - Preserved intentional rebuilds for adding/removing boxes and changing overlay source. - Added node-identity and immediate single-click regression assertions. Proof: - 6 targeted Queue/In-Out and control-inventory tests passed. - Browser interaction audit passed in Chromium, Chrome, Firefox, and Safari. - Node syntax, Ruff, and git diff checks passed.
Before: - Complete-state responses could land out of order across API domains. - Ordinary marker saves rebuilt active controls, duplicated blur/change commits, and interrupted drags. - In/Out routes and controls lacked a parameterized disk-backed interaction inventory. Now: - Cross-domain responses reject stale snapshots and failed mutations reconcile authoritative state. - Marker edits preserve DOM identity, commit once, update dependent subcomponents, and keep drags connected. - Control inventory, adversarial audits, persistence coverage, QA rules, and current full-app proof contracts are aligned. Why: - Prevent one-action controls from reverting, requiring retries, or persisting stale state. - Linked to: full SplitShot interaction, persistence, and stale-render audit request. How: - Added global mutation sequencing, failure reconciliation, active-control render guards, and component-level marker synchronization. - Added 324 control traces, 17-case In/Out single-interaction coverage, reordered-response and drag regressions, and four-engine audit checks. - Updated stale route, stage-isolation, UI-surface, and queue proof expectations to current product behavior. Proof: - 60/60 four-engine interaction checks passed. - 48/48 four-engine UI surface checks passed. - 31 full-app and In/Out tests passed. - Focused race, route-manifest, disk persistence, marker, and project-folder tests passed. - Ruff, node syntax checks, git diff check, and uv run splitshot --check passed.
Before: - Media and Compose accepted files only through picker-backed controls. - Packaged-app drag-and-drop had no document drop route or persistence proof. Now: - Route one file drop to primary media when empty and later files to added media. - Preserve exact one-mutation semantics with cross-browser persistence coverage. - Show active drop feedback and document the audited surface. Why: - Complete the full interaction and persistence audit's required drag-and-drop lane. - Linked to: packaged SplitShot interaction audit How: - Add scoped shell drag handlers for Media, Compose, and the video stage. - Reuse existing upload mutations and abort secondary ingestion if primary import fails. - Parameterize exact request and project-file proof across Chromium, Chrome, Firefox, and WebKit. Proof: - 5 passed: media drop cross-browser audit plus QA matrix contract - node --check src/splitshot/browser/static/lib/shell-runtime.js - uvx ruff check touched tests - uv run splitshot --check - git diff --check
This reverts commit 29a5189.
Before: - In/Out Match Results mislabeled IDPA Points Down as Shot Points. - Match Final and penalties were recomputed from available stage state, so missing stages or local analysis produced partial, incorrect match output. - Metrics CSV/TXT repeated the same derived totals. Now: - IDPA Total Score, Total PD, aggregate penalty counts, stage count, identity, and placement are retained from the imported spreadsheet. - Match Results uses the same Score/Time, Points Down, Penalties, Division, Class, and Overall fields as stage summaries. - Metrics UI and CSV/TXT identify the sport-correct match points field and use authoritative match totals. Why: - Final match results already exist in the PractiScore spreadsheet and must not be reconstructed from only the stages with local video. - Linked to: reported Show Match Results output defect How: - Persist match-level fields with each imported stage and prefer them when building match metrics. - Map legacy saved Match Results field ids to the corrected field set without losing existing overlays. - Keep preview, queue-time rendering, browser state, persistence, and metrics exports on the same match model. Proof: - 60 focused tests passed across import, browser state/UI, Metrics export, Queue/In-Out, persistence, inventory, and QA contracts. - 60/60 browser interaction audit checks passed across Chromium, Chrome, Firefox, and Safari-class WebKit. - Current project proof: Final 84.41, Points Down 11, Penalties 0, Stages 4 from IDPA-3.csv. - node syntax, Ruff on touched files, runtime check, and git diff check passed. - One unrelated existing Metrics graph layout test still reports one column at the 1280px fixture viewport.
Before: - Match Results showed division and class codes without placement context. - Overall showed only the selected competitor's raw spreadsheet place. Now: - Division and Class show the selected cohort place out of the spreadsheet cohort total. - Overall shows the final match place out of all ranked competitors. - Match CSV exports the same placement denominators. Why: - A place without its cohort size is incomplete match-level information. - Linked to: reported Output Match Results placement defect. How: - Reused final-standing cohort ranking for browser state and combined rendering. - Kept browser preview, queue/export rendering, Metrics CSV, tests, and QA matrix aligned. Proof: - 102 targeted tests passed; 1 unrelated pre-existing assertion deselected. - Browser interaction audit passed Chromium, Chrome, Firefox, and Safari-class WebKit. - Current project resolves CO 3/11, SS 2/8, Overall 5/27. - splitshot --check, JavaScript syntax, Ruff, and git diff checks passed.
Before: - Save Current Settings could capture values after an unrelated project flush and persist stale defaults. - constrained rendering rewrote preferred layout dimensions, ordinary Compose and Queue saves replaced controls, and queued scalar responses could roll newer edits backward. - scoring saves redundantly reapplied profiles and project draft flushes reread controls after asynchronous refreshes. - auto-sized Intro and Outro boxes used the project font instead of each box typography, clipping large text in encoded video. - the browser inventory omitted JavaScript-rendered controls and prior audits sampled only a small subset. Now: - click-time immutable settings and project draft snapshots survive blur, navigation, project reopen, and controller restart. - layout clamping is render-only; Compose and Queue preserve scalar control nodes; Queue saves are ordered; unchanged scoring profiles are not reapplied. - Intro and Outro encoded boxes size from their actual font and multiline text. - literal and programmatic controls are inventoried, with explicit one-action value/action ledgers that report unverified gaps instead of passes. Why: - prevent single user actions from reverting, requiring a second attempt, or persisting a value different from the visible draft. - Linked to: full SplitShot interaction, persistence, stale-render, settings-default, and Intro/Outro output audit. How: - capture payloads before awaits, separate preferred layout state from viewport CSS, add component-level render fast paths and ordered Queue writes, and defer overlay auto-size measurement to the typography-aware painter. - add browser/action audit coverage, settings disk/reopen/restart assertions, encoded-video typography regressions, and update the QA matrix. Proof: - 88 focused tests passed after one stale static assertion was corrected (87-test run plus isolated rerun). - 6 audit self-tests passed; 4 browser action lanes passed with 28 strict runtime cases each. - Chromium/Chrome/Firefox/WebKit prior interaction audit: 15/15 scenarios each. - splitshot --check, node --check, ruff, and git diff --check passed. - relevant browser sweep reached 374 passes; 10 unrelated or pre-existing visual/pipeline failures were isolated, then the run was interrupted after a Playwright hang.
Before: - each pointer move rebuilt the Intro/Outro preview overlay and disconnected the badge that owned pointer capture. - releasing outside the badge could leave drag state behind and make subsequent mouse interactions appear unresponsive. Now: - stable box structures update their existing preview badges in place. - pointer capture remains attached through the drag, releases explicitly, and window blur terminates the interaction. - Intro and Outro buttons remain immediately usable after an outside-box release. Why: - restore mouse movement and button interaction in the packaged Intro/Outro editor. - Linked to: reported installed-app pointer and selection-box regression. How: - key preview structure by boundary kind and box id, reuse connected badge nodes, and add explicit capture/blur cleanup. - extend the browser regression to release in the inspector, assert DOM identity and cleared drag state, then switch both tabs once. Proof: - targeted outside-release regression: 1 passed. - complete tests/browser/test_queue_pane_qa.py: 23 passed. - browser control inventory: 2 passed. - node --check, Ruff, and git diff --check passed.
Before: - Show Log changed its label to a sampled percentage and an open log could remain stale when processing completed. - Stage cards and rendered summaries mixed analyzed values with final match standings, while reopened stages lacked comparison cohorts. - Intro text was painted before boundary media was resized, scaling the selected font and measured box in the final video. - Project saves wrote directly to project.json, so closing during a write could leave an empty project file. Now: - Show Log stays static, progress polls every 250 ms, and live logs hand off to the complete persisted log. - Stage and match metrics use distinct spreadsheet-authoritative values and Class, Division, and Overall cohorts. - Intro and outro media is normalized before text painting, preserving configured typography and auto-sized boxes. - Project persistence uses a same-directory temporary file and atomic replacement. Why: - Queue status, spreadsheet results, rendered video typography, and saved project state must remain truthful after one user action and across restart. - Linked to: Queue log, PractiScore stage/match metrics, Intro output, and packaged restart regressions. How: - Tightened activity polling, refreshed open logs at terminal events, and selected final persisted logs after completion. - Added stage-result ranking, direct spreadsheet metrics, complete IDPA penalty comparison, and cohort refresh on reopen. - Reordered boundary normalization, overlay rendering, and fade preparation; discarded generated summary text from persisted overrides. - Replaced direct JSON writes with atomic same-folder persistence and failure cleanup. Proof: - 67 targeted interaction, metrics, queue, and export tests passed; 15 focused persistence tests passed. - Chromium, Chrome, Firefox, and WebKit interaction audits passed. - Packaged single-action log, spreadsheet stage/match cards, encoded 40 px intro, project reopen, and full app restart were manually validated. - SplitShot runtime check, JavaScript syntax, Ruff on persistence files, codesign verification, and git diff checks passed.
Before: - Unlocking a locked layout consumed the first resize gesture. - Marker and popup drags could be interrupted by an ordinary API render. - An older cross-domain API response could replace newer remote state. Now: - Locked layouts unlock and resize during the same pointer gesture. - Active popup and marker drags defer render replacement through outside-element release. - Remote-state responses share a global request sequence so older state cannot overwrite newer input. - Browser audits and the QA matrix explicitly own these interaction contracts. Why: - Every visible browser interaction must work once and remain stable through asynchronous refreshes. - Linked to: full SplitShot interaction and stale-render audit. How: - Record resize origins and apply pointer deltas after unlocking. - Include popup drags in the active interaction boundary and avoid redundant overlay rendering. - Track the latest remote-state request in addition to route-domain request order. - Extend real-browser audits and focused regression coverage. Proof: - 38 browser inventory, layout, and static tests passed. - 5 focused drag, resize, and reordered-response tests passed. - Chromium, Chrome, Firefox, and Safari audits each passed 15/15 checks. - JavaScript syntax and git diff checks passed.
Before: - v107 and the queue/export/metrics correction branch diverged after the shared v1.0.7 baseline. - The checkout was left mid-merge with two unresolved control-audit conflicts. - UI persistence, Intro/Outro, Queue processing, PractiScore metrics, settings capture, and interaction audits were split across the two histories. Now: - v107 contains all staged related and unrelated work from both histories. - Browser controls preserve first interactions and saved settings across refresh, project changes, and restart. - Queue logging, stage and match metrics, Intro/Outro rendering, native project/media selection, and atomic project persistence are unified. - Browser ownership documentation and parameterized one-action audits describe the merged behavior. Why: - The current v107 checkout must be a single buildable source for a fresh local app package. - Linked to: requested all-code commit, SplitShot interaction audit, output metrics, Queue log, settings persistence, and Intro/Outro corrections. How: - Merged codex/queue-export-metrics-fixes into v107. - Resolved the QA matrix by retaining both layout and complete control ownership contracts. - Retained readable stale-response audit assertions and aligned export proof with spreadsheet-authoritative stage values. Proof: - 80 of 82 targeted tests passed initially; the two assertion/fixture integration failures were corrected. - Both corrected targets passed on rerun. - Cached diff and conflict-marker checks passed.
Before: - Marker saves and drag previews could replace active DOM nodes or leave motion navigation stale. - Metrics workbench graphs and table remained hidden or single-column, settings reset could race a prior save, and compact overlay fields could collapse. - Queue, trim, merge-preview, pipeline, and visual-contract tests encoded non-queueable fixtures or obsolete UI/output assumptions. Now: - Popup responses preserve active interactions, drag rendering reuses the active badge, and marker motion/selection updates only the affected editor component. - Metrics graphs and table render in the expanded workbench, settings mutations serialize, and compact opacity controls retain usable width. - Targeted browser tests exercise real queueable stages, deterministic trim status, current output names, correct stage media ownership, and current labels/styles. Why: - Resolve the 14 failures remaining from the 501-pass browser run without weakening single-interaction behavior. - Linked to: browser interaction and persistence audit follow-up. How: - Suppressed pane-wide rendering for optimistic popup/UI-state saves while still accepting current server state. - Added component-level marker synchronization and stable drag-node positioning. - Corrected workbench markup/layout and updated stale fixtures and expectations to current product behavior. Proof: - Exact 14 failing node IDs: 14 passed in 34.78s. - Browser control inventory audit: 2 passed in 0.27s. - Requeue real-stage regression: 1 passed in 3.92s. - Ruff on changed Python tests, JavaScript syntax checks, and git diff check passed.
Before: - Leaving In / Out could leave the shared video element playing the intro or outro while the header, waveform, and Queue reflected the active stage. - Boundary preview changes bypassed the video player's source identity bookkeeping. Now: - Boundary previews maintain the shared media element source identity. - Navigating away from In / Out restores the active stage media immediately. - Browser coverage proves the In / Out to Queue transition cannot retain the boundary clip. Why: - Prevent a contradictory stale-render state visible in the packaged app. - Linked to: reported Queue screenshot showing Stage 2 state with the Intro clip in the player. How: - Track boundary source path and URL on the shared video element. - Re-render the stage video when a tool transition exits In / Out. - Update the browser QA matrix with the transition contract. Proof: - 25 passed: tests/browser/test_queue_pane_qa.py and tests/browser/test_browser_control_inventory_audit.py - node --check for app.js and intro-outro-pane.js - git diff --check
Before: - Expanded Metrics and Review content could overflow the visible shell, dense competitor labels collided, and preview media could appear cropped at scaled effective viewports. - Review text-box editors exposed minimize controls, and Electron could reopen with a non-default zoom factor. Now: - The shell, internally scrolling Metrics workspace, accessible responsive charts, and aspect-correct video authoring frame remain viewport-contained. - Review text-box editors are always expanded while legacy expansion data remains compatible, and Electron resets launch zoom to 100%. - User, architecture, Electron, QA ownership, audit, and test documentation reflects the current 15-pane behavior and validation coverage. Why: - Restore usable placement and review workflows at narrow windows and supported zoom levels. - Linked to: Fix Review, Metrics, and Viewport Layout request. How: - Added responsive shell/workbench/video CSS, rank-based accessible SVG labels, Review editor compatibility behavior, Electron zoom initialization, focused geometry/accessibility tests, and updated browser audits and documentation. Proof: - 44 focused browser, viewport, Metrics, Review, and inventory tests passed. - Electron smoke passed, including launch zoom reset to 100%. - Browser interaction audit passed 60/60 checks across Chromium, Chrome, Firefox, and Safari; UI surface audit passed 12/12 checks. - Full browser suite: 513 passed, 5 unrelated existing failures in marker DOM identity, stale source-contract assertions, and secondary-video synchronization. - uv run splitshot --check and git diff --check passed.
- controlIsActive now uses contains(document.activeElement) to detect focus inside child controls (timing adjustment inputs, scoring selects). - hasActivePointerInteraction now respects timingRowEdits and scoringRowEdits so requestRender() defers while workbench rows are unlocked. - renderTimingTable preserves active adjustment inputs across innerHTML rebuilds: captures value/selection/focus before rebuild, restores after. - renderScoringTable preserves active score selects and penalty editors across concurrent renders using the same preservation pattern. - Media pane now uses a structural key (renderStructureKey) and syncScalarControls to avoid full innerHTML rebuilds on scalar changes. Adds full-scale parameterized interaction/persistence/stale-render audit tests covering Chromium, Chrome, Firefox, and WebKit.
Before: - Pane documentation lacked a uniform current-review marker and the canonical set omitted In / Out. - Screenshot capture could fall back to test footage and showed duplicate or stale pane states. Now: - All 53 tracked Markdown files outside the root README are reviewed and marked for 2026-08-11. - The 27-image 1400x900 set uses approved Stage1/Stage2 footage, includes In / Out and Queue processing log proof, and shows expanded Markers, Settings, and Metrics states. Why: - Make every pane guide and supporting document match the current application and replace synthetic visual proof. - Linked to: Complete Documentation and Real-Video Screenshot Refresh plan. How: - Require distinct non-test primary and secondary videos, assert decoded non-black frames and enabled showcase features, and prewarm project-managed boundary media. - Align screenshot inventory and Queue log ownership, preserve the root README, and archive obsolete local project material outside the repository. Proof: - 8 passed: documentation screenshot tooling and tracked-tree hygiene tests. - 7 passed: pane-function, browser-control inventory, and QA coverage matrix tests. - Ruff checks passed for changed Python files. - Real-video capture exited 0; 27 PNGs verified at 1400x900, max 891763 bytes. - Markdown local targets passed; 53 review markers present exactly once; root README SHA-256 unchanged. - git diff --check passed; pre-commit git clean -nd empty.
Before: - Media trim derivatives left stale inventory filenames unless the pane rebuilt. - Marker overlay refreshes replaced draggable badge nodes during interaction setup. - Static and suite-runner contracts lagged current browser behavior and test discovery. Now: - Media inventory rows update active derivative names without replacing their DOM nodes. - Marker overlays reuse keyed badge nodes, and settings coverage waits for imported defaults before reset. - Trim, scoring preservation, and browser discovery contracts cover the current implementation. Why: - Eliminate every failure reported by the canonical full-suite run. - Linked to: manager request to fix all failures after full-suite validation. How: - Reconcile active Media filenames in place and key structural rebuilds by source identity. - Reuse popup badge elements across overlay renders and synchronize the settings truth-gate workflow. - Update focused browser and runner regression assertions. Proof: - Canonical all-together suite: PASSED, 1 run, 972.71 seconds. - Browser interaction audit: Chromium, Chrome, Firefox, and Safari passed. - Focused contracts: 30 passed. - Repeated failure isolation: 2 passed per run across repeated runs. - node --check and git diff --check passed.
Before: - Packaged release proof queried the removed trim-sync-card class on Linux and Windows. - Test macOS required live Apple notarization and failed when the developer agreement expired. Now: - Packaged proof targets the shipped trim-source-card contract. - Test macOS builds and validates a signed non-notarized DMG, while Build macOS and Release retain mandatory notarization. - Release documentation and workflow contract tests describe and enforce the separation. Why: - Restore clean-runner package validation for v1.0.7 without weakening publication requirements. - Linked to: failed Test Linux, Test Windows, and Test macOS workflow runs on commit 82625d6. How: - Replaced stale Trim selectors in the packaged Playwright release proof. - Disabled notarization only in the Test macOS package job and kept codesign verification. - Added focused regression contracts and updated release notes/runbook. Proof: - tests/scripts/: 66 passed. - Focused workflow and control inventory contracts: 6 passed. - Electron CI input verification passed. - node --check and git diff --check passed.
Before: - Packaged E2E still filled the removed Export path and opened the removed Export log control. Now: - Queue owns output execution and opens the export log through queue-show-log. Why: - Complete the packaged proof migration to the shipped v1.0.7 pane contract. - Linked to: Test macOS run 31542057790 packaged validation failure. How: - Removed the stale export-path write and switched log inspection to Queue. - Added regression assertions rejecting both removed selectors. Proof: - Focused packaged workflow contracts passed. - node --check and git diff --check passed.
Before: - Standard packaged E2E retained one guarded reference to the removed export-path input. Now: - Standard and release proof both execute output exclusively through Queue. Why: - Keep every packaged validation scope aligned with the shipped pane contract. - Linked to: packaged workflow selector audit after run 31542057790. How: - Removed the legacy conditional and process the queue unconditionally. Proof: - Packaged workflow contracts: 4 passed. - node --check and git diff --check passed.
Before: - Packaged E2E still clicked the removed queue-add-btn control. Now: - The proof resolves the active stage and uses its queue-membership button. Why: - Align package-native output execution with the shipped Queue pane. - Linked to: Test macOS run 31542898819 packaged validation failure. How: - Resolve active_stage_id from live state and click the matching membership control. - Reject the removed selector in workflow contract tests. Proof: - Packaged workflow contracts: 4 passed. - node --check and git diff --check passed.
Before: - Linux packaged proof failed after a successful 3.805s profile creation against a 2s shared-runner budget. Now: - Packaged profile create and edit operations allow up to 5s while retaining functional completion assertions. Why: - Avoid false release failures from normal hosted-runner variance. - Linked to: Test Linux run 31543668776. How: - Adjusted only the packaged profile mutation thresholds and locked them with a contract test. Proof: - Packaged workflow contracts: 4 passed. - node --check and git diff --check passed.
Before: - Packaged proof copied Queue output under its generated filename. - Windows OCR validation still probed the former fixed export filename. Now: - Queue proof copies the validated MP4 to the canonical e2e-export-test.mp4 artifact. - Contract tests and release notes cover the cross-platform proof filename. Why: - Windows packaged validation passed the UI flow but failed when ffprobe could not find the expected proof artifact. - Linked to: v1.0.7 release workflow repair. How: - Centralize the canonical export artifact path and use it for the Queue result copy. - Remove obsolete local path declarations. Proof: - node --check scripts/testing/e2e-playwright.cjs - uv run pytest tests/scripts/test_ci_workflow_export_proof_contracts.py tests/scripts/test_packaged_app_e2e.py --basetemp=tmp/codex/pytest -q (10 passed) - git diff --check
Before: - Release proof used compact synthetic fixtures and did not have a durable zero-gap plan for every installed control. - The approved real primary video, secondary video, and PractiScore CSV were not tracked as release inputs. Now: - Add the full implementation plan for exhaustive real-data installed-package validation on macOS, Windows, and Linux. - Track the approved two-video and authentic PractiScore release corpus with validated checksums and media/data properties. - Link the active Electron release runbook to the plan and distinguish current compact proof from future exhaustive acceptance. Why: - Every feature, button, field, text identity, lifecycle transition, and rendered effect must be proven after packaging on each operating system. - Linked to: exhaustive packaged release validation request. How: - Document the package-first architecture, runtime inventory, per-control contract, pane scenarios, OS-specific gates, artifacts, drift guards, implementation phases, and completion criteria. - Add explicit gitignore exceptions for the approved release videos. Proof: - FFprobe validated both H.264 1920x1080 60 fps videos with stereo audio. - SHA-256 and byte sizes match the documented corpus manifest. - SplitShot parsed practiscore.csv as IDPA with 27 competitors and four stages. - git diff --cached --check
Before: - Packaged checks used non-release fixtures and could pass without complete per-OS evidence. - Installed UI audits missed persistence races, restart ownership, rendered-output truth, and unmapped runtime gaps. Now: - Adds the committed real corpus manifest, 159-case release manifest, installed inventory and lifecycle evidence, and zero-gap platform aggregation. - Wires macOS, Windows, and Linux workflows to run the installed release-proof gate. - Fixes settings autosave, overlay and marker node preservation, concurrent browser-media preparation, installed isolation, and CDP restart lifecycle defects found by execution. Why: - Release publication must fail closed until every installed identity and semantic case is proved on all three native packages. - Linked to: docs/project/EXHAUSTIVE_PACKAGED_RELEASE_VALIDATION_PLAN.md How: - Validates corpus checksums and forbidden fixture roots, captures action/request/persistence/output/platform artifacts, and validates exact commit/platform summaries. - Exercises the packaged app with authentic match media and PractiScore data and records explicit pass/fail/gap dispositions. Proof: - 174 targeted release/browser tests passed. - Canonical all-together suite passed all analysis, browser, CLI, export, media, persistence, presentation, scoring, benchmark, and script suites. - Fresh installed macOS E2E passed 885/885 runtime discovery/mapping, 0 page/HTTP/console failures, individual and combined FFprobe/OCR/frame proof, restart, UI audits, and codesign. - Fail-closed summary correctly blocks remaining 175 identity gaps, 112 semantic case gaps, and unavailable local notarization/Gatekeeper proof. - uv run splitshot --check, node --check, compileall, and git diff --check passed. - Full repository Ruff reports 263 pre-existing findings; no blanket formatting changes made.
- Use forward-slash pathspecs for git ls-files; backslashes are escape characters in git pathspecs, so the tracked checks matched nothing on Windows. - Mark tests/release_data/* as binary (-text) so checkouts cannot convert line endings and break the size/sha256 corpus checks.
The validator compares git pathspecs with forward slashes (as_posix). The fixture mock returned native separators, which diverge on Windows and tripped the exact-corpus test on win32.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds fail-closed installed-package validation infrastructure for SplitShot's exhaustive release plan, committed real media/PractiScore corpus, per-OS evidence summaries, and the first executed installed macOS proof.
Also fixes product and harness defects found by execution: settings autosave generation, settings opacity semantics, overlay/marker node replacement, concurrent media-preview preparation, project/config isolation, current PractiScore match-type ownership, and Electron/CDP restart lifecycle.
Why
The existing release checks could pass without proving every installed control, lifecycle, rendered output, and native-platform contract. This is linked to
docs/project/EXHAUSTIVE_PACKAGED_RELEASE_VALIDATION_PLAN.md.How
Proof
uv run splitshot --check, JavaScript syntax checks, compileall, andgit diff --checkpassed.Risks