Skip to content

Generate placeholders' pictures and clips through GAM - #6

Merged
davior merged 1 commit into
mainfrom
claude/funny-knuth-en5fly
Oct 17, 2026
Merged

davior merged 1 commit into
mainfrom
claude/funny-knuth-en5fly

Conversation

@davior

@davior davior commented Oct 17, 2026

Copy link
Copy Markdown
Owner

A generate placeholder for a picture or a clip can now be made. GAM's M8 runs it on fal.ai with the user's own key. GVC has no fal code and no worker. No GAM change is needed.

How it runs

  • A generation row per run (new table and migration). Reading a project's generations refreshes the unfinished rows from GAM with the reader's token. A run started and then left still lands the next time the project is opened, and no token is stored.
  • Results go into the editor's draft as an unsaved edit: the placeholder's result is set and its status becomes ready. This happens once, and is recorded on the row. Any other outputs are offered beside it as small thumbnails to switch to.
  • Options come from the model's catalogue entry:
    • the aspect ratio, or FLUX's image size, follows the video's aspect;
    • durations are passed as GAM spells them;
    • the price shown is GAM's list price.
  • Music and sound placeholders are refused (cannot_generate), since GAM has no audio models. A clip with no base picture gets needs_base_image. A missing fal.ai key gives an error that links to GAM's Settings.

Routes

  • GET /api/gam/generation-models
  • POST / GET /api/projects/{id}/generations
  • POST /api/generations/{id}/use
  • DELETE /api/generations/{id}

The assistant

  • New generate_image tool. It adds the placeholder as a scene's picture or an on-screen image, and starts the generation at once. The cap is ASSISTANT_MAX_GENERATIONS per turn (default 6).
  • It never generates video, as decided. For footage it leaves a video placeholder for the user's Generate button. The prompt reference also says to search the library first.
  • Rows follow the turn into a new project. Applying a proposal reloads the project's generations, so the pictures fill in.

Frontend

  • Generate dialog:
    • the prompt, editable;
    • the model with its price;
    • 1–4 pictures, or for a clip its length and the picture it is made from;
    • an estimated cost.
  • On the placeholder: Generate…; while running, its progress with Stop; when done, the result with other outputs to switch to; on failure, the error with Try again.
  • "Still to make" card beside "Footage still to find".
  • stores/generations.ts: polls every 3 s while a run is going, with staleness tokens; it is reset on logout.

Tests

  • Backend: all 626 pass. New tests cover:
    • the catalogue: picking models, options, estimates;
    • starting runs, with the token forwarded;
    • refusals, following a job to done, use, cancel, a missing fal key;
    • owner checks and project delete;
    • the assistant: the tool's cap, an unstartable generation, and a whole turn that generates a picture but leaves the clip.
  • Frontend: all 129 pass, and lint and build are clean. New tests cover the spec helpers, the store, the dialog and the placeholder states.
  • End to end in Chromium, with stand-ins for GAM and the model:
    1. The assistant drafts a video and generates its picture, which fills in.
    2. Switching to the other output works.
    3. A clip is started from its Generate button.
    4. The page is left mid-run, and on return the clip is in the draft.

Generated by Claude Code

A `generate` placeholder for a picture or a clip now has a Generate button. GAM's M8
makes it on fal.ai with the user's own key; GVC has no fal code and no worker.

- A `generation` row per run. Reading a project's generations refreshes the
  unfinished ones from GAM with the reader's token, so a run started and left
  lands the next time the project is opened. No token is stored.
- A finished run's first output goes into the editor's draft (the placeholder's
  `result`, status ready), once; the other outputs stay offered beside it.
- Options come from the model's own catalogue entry (aspect ratio or FLUX image
  size from the video's aspect, durations as GAM spells them), with GAM's list
  price as the estimate.
- The assistant's new `generate_image` tool generates pictures, at most
  ASSISTANT_MAX_GENERATIONS a turn, and never video: for footage it leaves a
  video placeholder for the Generate button. Its rows follow the turn into a
  new project.
- Music and sound placeholders are refused: GAM has no audio models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fiRJAt34ag62iPiuMkNhm
@davior
davior marked this pull request as ready for review October 17, 2026 02:17
@davior
davior merged commit eee9de0 into main Oct 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants