[davior]

M4 (AI drafting) and M5 (AI editing) from the plan of attack, built together as one assistant:

• it sits in a panel beside every page and collapses like the one in Gecko Notes;
• it knows the whole VideoSpec;
• it can draft a video from scratch or make any edit to the one open;
• every request runs in the background.

What you can do

• Settings (gear icon in the header): add an AI provider (Anthropic, OpenAI, DeepSeek, Ollama or a custom endpoint), test it, and make it the active one. The page also shows what the assistant has spent, always labelled an estimate.

• On the project list: describe a video, for example "A 3-minute video about the Colorado River water wars, using clips where people talk about it." The assistant:

  1. searches your GAM library, including what people say in videos, and cuts to the moment something is said;
  2. writes the whole draft;
  3. fixes any validation issues itself;
  4. lists what it made.

  Create this video turns the draft into a project marked as drafted from a prompt, with the conversation moved over to it.

• On a project: ask for any change, for example "dissolve into the river scene", "find footage of people talking about water for this scene", or "change the music after chapter 2 to something calmer".

  • It starts from the editor's draft, unsaved edits included.
  • "This scene" means the scene card you last worked in.
  • It answers with the list of changes it would make. Apply puts them into the draft as unsaved changes, so Save, Discard and History work as usual.
  • If the project was saved after you asked, you get a warning, with Apply anyway.
  • A plain question ("how long is it?") gets an answer and no proposal.

• It keeps working with the page closed. Close the project, the tab or the browser, and the request still runs to the end on the server.

  • Coming back to the project shows its live steps, or the finished proposal waiting for Apply or Discard.
  • A marker in the header, on any page, shows "The assistant is working" or "1 change to review", with a link to the video.

• Panel details:

  • a Stop button while it works;
  • an expandable list of steps (searching, editing, fixing);
  • earlier conversations for each video;
  • each turn's estimated cost and the conversation's total;
  • panel width and open or closed state are remembered.

How

• Part 1: providers, usage and Settings. Copied from GAM's M6, which came from Notes:

  • encrypted keys, returned to the browser only as api_key_configured;
  • the SSRF guard on custom URLs, and retries on 429/503;
  • pricing updated with the current Claude models;
  • UsageEvent attributed to a project and a turn, with no foreign keys, so deleting a project keeps its spend.

  New over GAM: multi-turn conversations that call tools. Each keeps its history in the provider's own wire format and only ever appends to it, so an Anthropic turn's content, thinking blocks included, is replayed exactly as it came. Anthropic requests cache the reference and the project as system blocks, plus top-level automatic caching. Ollama goes through its OpenAI-compatible /v1 endpoint.

• The tool set (app/assistant/tools.py): the Notes spec doc's §8.2 operations, addressed by id, never by list position (that doc's open decision 14).

  • Reading: get_spec, search_assets, get_asset, estimate.
  • Segments: add_segment, update_segment, move_segment, remove_segment (which reports anything still anchored to the removed segment).
  • Pictures and overlays: set_visual (a hit's spoken moment becomes the trim), add_placeholder, and add, update and remove for overlays.
  • Project and drafts: add, update and remove for audio tracks, plus set_project and replace_spec for whole drafts.
  • Edits are merge patches. Every edit is validated against the working copy as it is made, and the issues go back to the model. A valid video is never broken: an edit that would break one is refused and nothing changes. A draft with mistakes is repaired edit by edit. This generalises D2's "one repair round" to every step, which the plan doc now records.
  • A model that stops with issues left is told to fix them, at most twice. If they remain, nothing is proposed, and the reply says why.

• The system prompt (app/assistant/prompt.py) has two parts:

  • a fixed reference that makes the model an editor for this format: structure, anchors, transitions, Ken Burns paths, overlays, crossfades, credits, pacing, and "search before you choose, never invent an id";
  • this project's context, unchanged for the whole turn: the spec, the asset registry, the focused scene.

• Background jobs (app/assistant/worker.py), on the render queue's runner:

  • The prompt, starting draft, steps, reply, proposal and cost all live on the AssistantTurn row as they happen.
  • The user's token, needed for GAM searches, is held in memory only. A server restart ends a running turn with "Interrupted by a restart. Ask again." in the chat, as renders do.

• API (/api/assistant):

  • start, poll and stop a turn;
  • list the turns that need attention;
  • mark a proposal applied or discarded, or apply a draft as a new project;
  • list, read and delete conversations.

  Deleting a project deletes its conversations.

• Migrations: two, for providers and usage, then for assistant sessions and turns. Both were checked upgrading, downgrading and upgrading again.

Tests

• Backend: pytest -q passes, 609 tests. New ones cover:

  • Provider conversations: for both protocols, tool calls, verbatim replay, results sent in one message, the caching fields, refusals, unreadable arguments, and Ollama's URL with no key.
  • Every tool:
    • id addressing, merge-patch semantics, and refusing an edit that would break a valid video;
    • repairing a broken draft step by step, and keeping the project id;
    • a hit's moment becoming the trim, with the user's token forwarded to GAM;
    • placeholders, overlays, tracks and project settings;
    • bad calls returned as errors, never raised.
  • Whole turns against a scripted provider:
    • a draft from a prompt: search, a draft with a mistake, the fix, the moment cut in, the proposal, usage rows and the turn's cost, then applied as a new project with its source, its author and the conversation moved over;
    • an edit on a draft with unsaved changes that keeps them, with nothing reaching the project until Apply, and found again through the attention list;
    • a question that proposes nothing;
    • repair prompts, with nothing proposed when the issues remain;
    • conversation history, one turn at a time, no provider, Stop (queued and mid-turn), a restart, a provider failure, the token never stored, owner checks, and project deletion.

• Frontend: format, lint, npm test (119 tests) and build pass. New tests cover:

  • a waiting proposal found on return and applied into the draft;
  • the warning when the project was saved since;
  • the draft, version and focused scene sent with a request, with live steps and Stop;
  • a draft becoming a new video from the project list;
  • the panel collapsing and remembering it;
  • the Settings link when no provider is set up;
  • the provider and spend panels.

• End to end in a browser: the real backend and frontend, with stand-ins for GAM and for the model. The stand-in model is a scripted OpenAI-compatible server set up as an Ollama provider through Settings.

  1. From the project list, asked for a video and left for Settings mid-run. The header kept saying "The assistant is working", and the steps kept arriving.
  2. Came back: the draft was waiting, with four changes listed (a three-scene draft, a dissolve, the river interview cut at 41.5–48.0 s, a lower third). Create this video made the project.
  3. Asked "dissolve into the river scene", applied the change and saved: version 2.

  There are no Claude credentials in this environment, so no real model ran. The first run against your key is the real test of the prompt.

Deploy notes

• Two migrations, which run from the entrypoint as usual.
• After deploying, open Settings, add a provider and click Use this one. Any current Claude model works; claude-opus-5-5 is the placeholder.

🤖 Generated with Claude Code

https://claude.ai/code/session_016fiRJAt34ag62iPiuMkNhm