Skip to content

MOVIE.md

Latest commit

 

History

History
313 lines (247 loc) · 14.4 KB

MOVIE.md

File metadata and controls

313 lines (247 loc) · 14.4 KB
Raw

Movie — multi-scene, character-consistent films

gflow movie turns a single TOML manifest into a sequence of generated video clips that share the same characters (face + voice) across every scene. It is the orchestration layer on top of gflow character (entity creation) and gflow video (R2V generation).

Status: shipped (since v0.14.0) and evolving. The CLI surface below is release-stable per SemVer; alpha (0.x) minor versions may still extend or adjust it, with changes noted in the CHANGELOG.

Quick start

# 1. Scaffold a manifest
gflow movie template movie.toml

# 2. Edit movie.toml (characters + scenes), then run
gflow movie run movie.toml --profile <chrome-profile>

movie run is generate-only by default: each scene becomes its own clip in the output directory. Pass --stitch for an optional ffmpeg hard-concat preview (no transitions — not a deliverable).

Manifest (movie.toml)

schema_version = 1
title = "E2E Stickman"
project = "6ba50219-…"            # reusable Flow project id
output_dir = "./out"

[style]
look = "cinematic, shallow depth of field"
palette = "warm golden tones"
negative = "no text, no watermark"
suffix = "Black and white cinematic street photography, photorealistic, film grain."

  [style.variants.warm]
  suffix = "Warm golden-hour cinematic grade."

  [style.variants.cool]
  suffix = "Cool blue-toned cinematic grade."

[[characters]]
name = "Stickman"
identity = "entity"               # reuse a Flow CHARACTER entity across scenes
face_prompt = "Simple round stickman, black ink lines, smiley face"
voice = "alnilam"                 # voice baked into the entity at creation

[[scenes]]
id = "summit"                     # stable key — used for resume
action = "stands on a clifftop at sunset, waves at the camera"
framing = "wide"
characters = ["Stickman"]         # which characters appear (drives R2V reuse)
speaker = "Stickman"
line = "We finally made it to the top!"
aspect = "9:16"
model = "omni-flash"              # 4/6/8/10s; Veo 3.1 supports 4/6/8
duration = 8

[[scenes]]
id = "close-up"
action = "smiles with excitement"
style_variant = "warm"            # selects [style.variants.warm]
style_suffix = "golden hour light" # appended after the variant suffix
aspect = "9:16"
duration = 6

Scene duration values depend on the model (issue #634). Veo 3.1 models support 4, 6, and 8; omni-flash also supports 10. Whether your account renders a duration control at all is cohort-dependent — where it does not, the run aborts pre-submit with no credits spent. Values are checked at parse time, before any scene generates. A scene with no named model inherits Flow's default and cannot receive a model-specific duration check at parse time; name the model whenever the exact length matters.

On first run, characters with identity = "entity" are created once (image generation — free, no credits) and cached. Each scene then generates a clip that reuses the same Flow CHARACTER entity so the character drives every scene from one identity.

Style variants

The [style] block defines a global visual style applied to every scene. For channel-format videos where the style changes across scenes (e.g. a monochrome → warm color arc), use named variants:

[style]
prefix = ""           # optional, prepended before the composed scene text
suffix = "Black and white cinematic street photography, photorealistic, film grain."

  [style.variants.warm]
  suffix = "Warm golden-hour cinematic grade."

  [style.variants.cool]
  suffix = "Cool blue-toned cinematic grade."

Per-scene selection:

Field Effect
style_variant = "warm" Selects [style.variants.warm] — its suffix replaces the base suffix.
style_variant = "none" Opts out of all style suffixes (base + variant). Prefix is still applied.
(omitted) Uses the base [style] suffix.
style_suffix = "sunset light" Scene-specific text appended after the variant/base suffix.

Style Configuration Errors

There are two style configuration errors that will raise a ConfigurationError:

  • Reserved none name: Defining [style.variants.none] is a configuration error, so the opt-out keyword can never shadow a real variant.
  • Unknown variant name: Specifying a style_variant in a scene that does not exist in [style.variants] (and is not "none") is a configuration error.

Composition rule (deterministic, no LLM):

final_prompt = [prefix] + composed_scene_text + [variant.suffix | base.suffix] + [scene.style_suffix]

The applied style is recorded per clip in <manifest>-handoff.json as style_applied (variant, prefix, suffix, scene_suffix), so downstream tools (Remotion, editors) can introspect it.

Resume: each completed scene's state records a hash of its composed prompt. If you edit the style (or anything else that changes a scene's prompt) and re-run, only the affected scenes are regenerated — unchanged scenes are still skipped and cost no credits. State files written by older gflow-cli versions fall back to comparing the stored prompt text; scenes with no stored prompt are never re-run automatically.

Consistency is best-effort, not pixel-exact. gflow guarantees the right entity rides the wire (consistency_method = entity); the final on-screen fidelity is Flow's Veo R2V model, which may reinterpret minimalist or hand-drawn references (e.g. a single round body can render as stacked circles). For tighter results, use clearer/richer reference images and a higher-tier model (veo-quality/veo-fast rather than veo-lite) — the same trade-offs you would hit driving Flow's UI by hand.

Run lifecycle — keep the browser open

gflow movie run drives the real Flow web UI in a headed Chrome window (your --browser chrome profile). For each scene the same browser window is used end-to-end:

  1. Attach the character entity to the generation (see below).
  2. Submit the prompt — this passes reCAPTCHA and spends 1 video credit.
  3. Poll Flow for completion (the clip renders server-side, ~30–90s).
  4. Download the finished mp4 into output_dir.

Do not close the browser window while a run is in progress. Steps 3 and 4 still use the browser page (polling even brings it to the foreground). If the window is closed before a scene finishes downloading, the in-flight scene aborts — you will see a "Target/page/context closed" or scene_failed error. This is expected, not a bug.

If a run is interrupted (closed window, crash, network drop), it is safe to re-run: the sibling <manifest>-state.json records completed scenes (keyed on scene.id) and they are skipped, so the command resumes where it left off. A versioned handoff manifest (<manifest>-handoff.json) is always written at the end.

Future — fire-and-forget. Flow's status-poll and download endpoints are non-generative and could be driven browser-free over Bearer REST (no reCAPTCHA, no extra credits), letting the browser close right after submit while gflow finishes over the API. That mode is not implemented yet; today the browser is required for the full generate → poll → download cycle.

Character consistency (how the entity rides)

A scene listing characters = ["Stickman"] is generated as R2V (reference-to-video) so the named Flow CHARACTER entity is reused. The entity is attached by driving Flow's resource picker:

  1. Click Add Media in the composer (references sub-mode).
  2. Switch to the Personagens (Characters) tab.
  3. Right-click the entity tile — addressed by entity id as data-tile-id="fe_id_<entityId>" — and choose the include-in-prompt action from the context menu (the add-ligature menu item; its caption is localized per account language, e.g. "Incluir no comando" on pt-BR or "Добавить в запрос" on ru — selectors are locale-free since issue #170). (A plain left-click navigates into the character editor; the inline include button on the Tudo tab attaches the character's thumbnail as a plain image, not the entity.)

This puts referenceEntities:[{entityId}] on the video:batchAsyncGenerateVideoReferenceImages request. Flow's response echoes the accepted entity at:

media[].mediaMetadata.requestData.videoGenerationRequestData
      .videoGenerationEntityInputs[].entityId

gflow asserts this on every entity scene (_assert_entities_attached) and refuses to report success if the entity did not ride — a text/image-only clip is never silently passed off as character-consistent. The recorded consistency_method in the run state is entity when the entity rode (vs text).

The same trap on the migrated host, and it is not ported yet

The bullet above — the inline include button attaches the character's thumbnail as a plain image, not the entity — is not a labs quirk. It is how Flow's reference surfaces work, and the migrated flow.google.com composer has its own version of it.

Measured 2026-09-07 ($0, scripts/dev/capture_migrated_mention_gestures.py): the migrated composer attaches by an @ mention picker that lists characters and media in one list, and the same query resolves to either depending on the keystroke.

gesture chip data-reference-type
@ + query + Enter Kael entity — carries data-entity-id
@ + query + ArrowDown + Enter kael_ref.jpg media — entity_id is null

So a name that matches both a character and a file is ambiguous at the picker, and Flow does not rank them. Whatever drives it must: the character wins, and the committed chip's data-reference-type is read back before submit — exactly the assertion _assert_entities_attached already makes on labs.

None of this is driven yet: --reference-entity / @Name on the migrated video path exits 36, so movie run cannot produce an entity-consistent scene there. Tracked in #723 with the working gesture recorded.

A costume change is a new character, not a variant

[characters.variants] appends a text delta to the prose appearance (composition.py:67-80), and the runner creates one entity per character name (cli_movie.py:589-597). That is coherent for identity = "text" and incoherent for identity = "entity": the entity's body reference fixes one outfit permanently, so a variant asking for another puts the reference and the prose in conflict inside one generation, with an undefined winner.

A Flow character entity bundles face and wardrobe. So model a costume change as two entities sharing a byte-identical face_prompt and differing only in body_prompt:

[[characters]]
name = "Kael_ridge"
identity = "entity"
face_prompt = "<the canonical face>"
body_prompt = "a plain worn canvas jacket in dust-brown, sand-coloured scarf"

[[characters]]
name = "Kael_coat"
identity = "entity"
face_prompt = "<the same canonical face, byte-identical>"
body_prompt = "a heavy oiled coat, hood down"

Scenes then name the costume state, and continuity is a lookup rather than a hope. Tracked in #724.

The full identity resolution ladder — entity, then the entity's own plate, then a plate cut from a take, then prose canon, with the rule that descending a rung is recorded next to the shot — lives in skills/video-production/SKILL.md step 4a. Read it before planning a multi-shot piece; a manifest that silently lands on the bottom rung produces a different actor in every scene.

See CHARACTER.md for the underlying entity model and CHARACTER_RECON.md for the reverse-engineered wire protocol.

Scene instructions (project brief cards)

You can manage a project's Agent-Mode brief cards per-scene. The movie manifest supports both global and scene-level instructions blocks:

[instructions]
# Applied to all scenes unless overridden.
[[instructions.card]]
title = "Cinematic Lighting"
text  = "Volumetric cinematic light from camera-left"
ref   = ["./refs/mood.jpg"]
enabled = true

[[scenes]]
id = "scene-1"
action = "hero emerges from fog"
[scenes.instructions]
disable = ["Cinematic Lighting"]  # disable a global card for this scene
[[scenes.instructions.card]]
title = "Fog Atmosphere"
text  = "Dense volumetric fog, low contrast"

For each scene, gflow movie merges global manifest cards with scene overrides (applying disable lists and adding/overriding card definitions), fetches the current project brief from the Flow server, uploads any local reference images to generate media UUIDs, preserves existing card IDs to prevent state drift, and PATCHes the brief to set the master switch and update cards before generating that scene's clip. See INSTRUCTIONS.md for the project-level instructions surface.

The manifest is authoritative — this is a full-sync, like gflow instructions apply. The PATCH replaces the project's brief with exactly the manifest's card set, so any card added out-of-band in the Flow web UI that is not in movie.toml is removed when a scene runs. Keep every card you want in the brief under [instructions] / [scenes.instructions]. Consecutive scenes that resolve to the same card set are synced only once per run (no redundant re-upload / PATCH).

movie run options

Flag Default Effect
--profile <name> default profile Chrome profile to drive (must be a chrome-strategy profile).
--out-dir <dir> manifest output_dir Override the output directory.
--dry-run off Print the pending video operation plan; make no API calls.
--fail-fast / --continue-on-error continue Stop on the first scene failure, or attempt the rest (default).
--stitch off After generating, hard-concat all clips into one preview mp4 (ffmpeg, no transitions).

Credits

  • Character creation (image generation) is free — no credits, no reCAPTCHA.
  • Each scene is one pending video generation operation, submitted at step 2 above. Current credit use varies by model, duration, account tier, and Flow policy — check Google Flow before submitting. An accepted operation may consume credits even if later polling or download fails. A --dry-run prints the plan without submitting anything.