[davior]

Summary

This is Phase 0 of docs/gecko-video-creator.md. The video structure that segment() works out from a note on every render becomes a document: the VideoSpec. The new code runs alongside the existing code. segment() and renderer.py are untouched, and nothing a user sees changes.

• backend/app/video/spec.py: the VideoSpec as Pydantic models. A video is segments (scene / card / clip); the narration beats inside a segment (one per note block, each recording its block_id); overlays tied to beats; audio tracks tied to segments; and a registry of the media files they point at. It validates its own structure: unique ids, references to files that exist in the registry, and clips and cards that are well formed. Extra fields are rejected.
• backend/app/video/importer.py: import_note() walks a note step for step as segment() does, but records beats, overlays and which blocks each part came from. A quote or code block becomes an overlay on its own beat instead of a separate shot. segment_via_spec() has exactly segment()'s signature. The planned follow-up PR switches segment() to it.
• backend/app/video/compiler.py: compile_spec() turns a spec back into the Shot list and RenderOptions today's renderer uses. It refuses (UnsupportedSpec) anything that renderer can't draw:
  • media from GAM or not yet generated
  • timed overlays
  • per-segment motion, voice or transitions
  • more than one music track
• docs/gecko-video-creator.md:
  • The doc's worked example now validates against the models.
  • Corrections from the repo review:
    • GAM has no origin_* columns yet.
    • Read-aloud export keeps some speech code in Notes.
    • Putting a finished video back into a note needs designing.
    • GVC's speech calls need a way to fix the provider.
    • Notes and GAM both put a service named backend on the shared web network.
  • Records the decision on where GVC's code lives.

The proof (backend/tests/test_video_spec.py, 93 tests, about 8s, no real encodes)

• Every case in test_video_segmenter.py (68) is re-run with segment wrapped so both paths must return identical shots and warnings, field by field.
• 3,000 generated notes built to hit the edge cases are checked the same way: chapter screens, quotes taking over a section, sounded clips, missing and remote media, diagrams, intro/outro, pause markers.
• Real render() runs use the existing FakeFFmpeg stub and issue identical ffmpeg commands through both paths.
• 300 random RenderOptions survive a round trip on every setting the renderer reads.
• Every imported spec also survives a JSON round trip.

Two deliberate exceptions, both tested:

• Options that only decide the shape of the import (title_card, chapter_screens, …) are recorded as structure, not as settings.
• "Music on, nothing chosen" has no form in a spec, so the renderer's "music skipped" warning is the only difference.

Existing quirks, reproduced on purpose for now

The spec path keeps today's output identical, quirks included. These are worth fixing once the spec path is the only one:

• With headings not read aloud, a chapter mark is lost when a quote immediately follows the picture (segmenter.py:461).
• [pause:…] markers typed into a heading end up in the MP4's chapter list and the "title + chapter" overlay (segmenter.py:365, :384).

Next, as separate PRs

1. Switch segment() to segment_via_spec() and expose GET /api/video/spec?note_id=.
2. Make the dialog's Voice choice actually reach the renderer (worker.py:81).
3. Fix the two quirks above.

Test plan

• pytest -q from backend/, as CI runs it: 1,282 passed, 1 skipped (that skip was already there).
• The new tests on their own: 93 passed.
• CI green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PwaujUokXQAxnebBArpZuj

---

Generated by Claude Code