Skip to content

Persist Studio manual edits via manifest#593

Open
vanceingalls wants to merge 1 commit intonextfrom
feat/studio-manual-edit-manifest
Open

Persist Studio manual edits via manifest#593
vanceingalls wants to merge 1 commit intonextfrom
feat/studio-manual-edit-manifest

Conversation

@vanceingalls
Copy link
Copy Markdown
Collaborator

@vanceingalls vanceingalls commented May 1, 2026

Summary

This PR changes Studio canvas geometry editing from direct source/timeline mutation to a manifest-backed edit layer.

Manual drag, resize, and rotation gestures now persist to .hyperframes/studio-manual-edits.json. Studio then reapplies that manifest in the places where users expect the edited layout to show up: live preview, undo/redo, thumbnails, frame capture, and producer renders.

The goal is to make visual canvas edits work mechanically without requiring the composition author or the model to pre-arrange every editable element with absolute positioning or GSAP variables.

Problem

The previous Studio manual-edit path was fragile for animated compositions:

  • Canvas geometry changes were source-patched directly for some elements, which was unreliable for relative layout, nested compositions, and authored CSS.
  • Some move attempts tried to mutate GSAP tween state or offsets, which could create jumps when the timeline rendered its original animated state again.
  • Preview and render could diverge because GSAP seek/render passes can rewrite animated transform state after Studio has visually adjusted an element.
  • Undo/redo had to reason about multiple write paths instead of a single project file representing the manual visual edit.
  • Agent handoff was ambiguous because visual edits were not represented as a clean artifact that an agent could read before making subsequent source changes.

This PR makes the manual visual edit itself explicit and project-local.

Approach

Manual edit manifest

Studio now stores manual canvas geometry edits in:

.hyperframes/studio-manual-edits.json

The manifest stores stable targets plus edit records:

{
  "version": 1,
  "edits": [
    {
      "kind": "path-offset",
      "target": {
        "sourceFile": "index.html",
        "selector": "#card",
        "id": "card"
      },
      "x": 32,
      "y": 18
    },
    {
      "kind": "box-size",
      "target": {
        "sourceFile": "scenes/intro.html",
        "selector": ".title",
        "selectorIndex": 0
      },
      "width": 420,
      "height": 96
    },
    {
      "kind": "rotation",
      "target": {
        "sourceFile": "scenes/intro.html",
        "selector": ".badge",
        "selectorIndex": 1
      },
      "angle": -8.5
    }
  ]
}

The target includes sourceFile, id, selector, and selectorIndex so Studio can keep edits attached to the intended element across preview, thumbnail, and render contexts.

CSS application model

Manifest edits are applied as a visual layer on the target element:

  • Move uses the CSS translate longhand backed by --hf-studio-offset-x and --hf-studio-offset-y.
  • Resize writes actual box dimensions: width, height, min-*, max-*, box-sizing, and flex sizing overrides where needed.
  • Rotation uses the CSS rotate longhand backed by --hf-studio-rotation.

This avoids rewriting GSAP timelines or requiring elements to be absolutely positioned. GSAP can still render its authored animation state, and Studio reapplies the manifest after seek/render passes.

Source-scoped target resolution

Manifest target lookup is now source-file scoped instead of global document scoped.

This matters because bundled previews can contain repeated IDs, classes, or selectors across scenes and nested compositions. Resolution now checks the owning [data-composition-id] root and its data-composition-file / data-composition-src metadata before applying an edit.

Covered cases include:

  • Duplicate id values across source files.
  • Duplicate class selectors across scenes.
  • Selector indexes that should be counted only within the target source file.
  • Nested composition DOM where the top-level document contains multiple source files at once.

Preview and render reapplication

Studio preview now installs seek hooks so manifest edits are reapplied after timeline seeking:

  • window.__hf.seek
  • window.__player.seek
  • window.__player.renderSeek
  • window.__timeline.seek
  • window.__timelines[*].seek

Producer renders also inject a small body runtime that applies the manifest and wraps render seek functions. The render wrapper uses a finite retry loop rather than an unbounded interval.

This addresses the class of bugs where an element appears correctly after dragging, but then jumps back or loses rotation/offset after a seek, scrub, capture, or render pass.

Undo/redo integration

Because the manual edit manifest is a normal project file, Studio's persistent edit history can record drag, resize, and rotation changes as file-level transactions.

Undo/redo no longer depends on click-away/blur timing for the manifest write path. Applying undo/redo for .hyperframes/studio-manual-edits.json updates the preview without a full source reload when possible.

Removed old manual geometry path

This removes the old source-patching/manual-movable path for canvas geometry:

  • No more "Make movable" / detach UI for this flow.
  • No more GSAP tween offset mutation for drag gestures.
  • No green/yellow debug editability highlighting.
  • Canvas geometry edits are represented by the manifest instead of trying to bake inline left / top / transform changes during the gesture.

User-facing behavior

After this PR, users should be able to:

  • Drag supported Studio elements without requiring absolute positioning.
  • Resize supported elements by changing their container dimensions, not by scaling.
  • Hold Shift while resizing to keep width and height uniform.
  • Rotate supported elements with the Studio rotation control.
  • Undo and redo manual visual edits through the existing Studio history controls.
  • Capture thumbnails/frame screenshots that include the manual manifest edits.
  • Render videos from Studio that include the manual manifest edits.

Agent handoff model

The agent-facing handoff is now explicit:

  1. Studio writes manual visual edits to .hyperframes/studio-manual-edits.json.
  2. Before an agent makes source edits, it should read the manifest as part of project context.
  3. The agent can either preserve the manifest, update source around it, or intentionally bake/remove manifest edits if the requested change calls for that.
  4. Studio and render continue to treat the manifest as the source of truth for manual canvas geometry until it is intentionally changed.

This keeps user-made visual edits from being hidden in transient DOM state and reduces the chance that later agent edits accidentally collide with the user's Studio adjustments.

Main files changed

  • packages/studio/src/components/editor/manualEdits.ts

    • Manifest parse/serialize/upsert helpers.
    • Runtime application for move, resize, and rotation.
    • Source-scoped target resolution.
    • Preview seek-hook installer.
  • packages/studio/src/components/editor/DomEditOverlay.tsx

    • Canvas drag/resize/rotation interactions now write manifest edits.
    • Overlay controls reflect the manifest-backed manual geometry model.
  • packages/studio/src/App.tsx

    • Reads/writes the manifest through Studio project file APIs.
    • Connects manifest writes to persistent undo/redo.
    • Applies manifest edits to preview and reinstalls seek hooks after preview load.
  • packages/studio/src/components/editor/manualEditsRenderScript.ts

    • Injected render-time runtime for applying manifest edits during producer renders.
    • Reapplies manifest edits after render seeks.
  • packages/studio/vite.config.ts

    • Applies manifest edits to Studio thumbnail generation.
    • Passes render body scripts into producer renders.
  • packages/core/src/studio-api/routes/thumbnail.ts

    • Includes manual edit manifest changes in thumbnail cache invalidation.
  • packages/producer/src/services/fileServer.ts

    • Supports additional render body scripts used by the manual edit runtime.

Test coverage added/updated

This PR adds or updates tests for:

  • Manifest parsing, serialization, and upsert behavior.
  • Path offset, box-size, and rotation application.
  • Clearing stale manifest-applied styles.
  • Avoiding manifest replay during active drag gestures.
  • Source-scoped target resolution with duplicate IDs/classes across nested compositions.
  • Preview seek-hook reapplication.
  • Render body script application after render seeks.
  • Thumbnail cache invalidation when the manifest changes.
  • Producer file server body script injection.
  • Overlay drag/resize/rotation behavior and undo/redo interactions.

Verification

Local checks run on the final restacked branch:

bun --bun run --filter @hyperframes/studio test \
  src/components/editor/domEditing.test.ts \
  src/components/editor/manualEdits.test.ts \
  src/components/editor/DomEditOverlay.test.ts \
  src/components/editor/manualEditsRenderScript.test.ts \
  src/hooks/usePersistentEditHistory.test.ts \
  src/player/hooks/useTimelinePlayer.test.ts \
  src/utils/clipboard.test.ts

Result: 7 files passed, 76 tests passed.

bun --bun run --filter @hyperframes/core test src/studio-api/routes/thumbnail.test.ts

Result: 1 file passed, 6 tests passed.

bun test packages/producer/src/services/fileServer.test.ts

Result: 23 tests passed.

bun --bun run --filter @hyperframes/studio typecheck
bun --bun run --filter @hyperframes/core typecheck
bun --bun run --filter @hyperframes/producer typecheck

Result: all passed.

bun node_modules/.bin/oxlint \
  packages/studio/src/App.tsx \
  packages/studio/src/components/editor/manualEdits.ts \
  packages/studio/src/components/editor/manualEdits.test.ts \
  packages/studio/src/components/editor/manualEditsRenderScript.ts \
  packages/studio/src/components/editor/manualEditsRenderScript.test.ts \
  packages/studio/vite.config.ts

Result: 0 warnings, 0 errors.

bun node_modules/.bin/oxfmt --check \
  packages/studio/src/App.tsx \
  packages/studio/src/components/editor/manualEdits.ts \
  packages/studio/src/components/editor/manualEdits.test.ts \
  packages/studio/src/components/editor/manualEditsRenderScript.ts \
  packages/studio/src/components/editor/manualEditsRenderScript.test.ts \
  packages/studio/vite.config.ts

Result: all matched files use the correct format.

git diff --check

Result: passed.

Manual render smoke test from the branch:

  • Rendered manual-editing-four-scenes_2026-05-01_12-02-16.mp4.
  • Extracted frame 60 and verified all three rotated scene-1 cards rendered with rotation.

Notes for reviewers

  • The PR is intentionally preserving manual canvas geometry as a separate Studio manifest instead of immediately baking it back into HTML/CSS source.
  • The manifest application is visual and additive. It does not rewrite authored GSAP timelines.
  • The target resolution must remain source-scoped because bundled previews can contain multiple source files with repeated selectors.
  • Producer package-level test uses its regression harness, so the focused producer file server test was run directly with bun test because that test file imports bun:test.

@vanceingalls vanceingalls force-pushed the feat/studio-manual-edit-manifest branch from 7bdc9b9 to 117f6ac Compare May 1, 2026 19:22
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not merge this as-is. The manifest direction looks good, but I found two correctness issues that can make Studio preview diverge from render or lose authored inline state.

  1. [P1] Preserve source metadata for rendered manual edits. Manual edits made inside a sub-composition are stored with target.sourceFile set to that composition path, but producer compilation inlines the sub-composition and removes data-composition-src without preserving an equivalent data-composition-file marker. The render-time manifest runtime only resolves nested targets through data-composition-file / data-composition-src and otherwise falls back to index.html, so those manifest edits cannot match in the compiled render DOM. I reproduced this with a compiled-style DOM: a manifest rotation targeting sourceFile="compositions/scene.html" left the nested card unrotated. This means Studio preview and final render can diverge for drilled-in manual edits. The affected render path is around packages/producer/src/services/htmlCompiler.ts where data-composition-src is removed during inline compilation, and the resolver is in packages/studio/src/components/editor/manualEditsRenderScript.ts.

  2. [P2] Restore authored translate when clearing offsets. Path offsets overwrite the inline translate longhand, but clearStudioPathOffset only removes translate instead of restoring any authored inline translate value. Undoing or clearing a manual offset on an element that already had style="translate: ..." leaves the Studio preview without the original translate until a full reload. I reproduced this by applying a manual path offset to an element with translate: 10px 20px, then applying an empty manifest; translate became empty. See packages/studio/src/components/editor/manualEdits.ts around clearStudioPathOffset.

Verification I ran locally on head 117f6acc: Studio focused tests passed, core thumbnail test passed, producer file server test passed. I also ran the two small reproduction snippets above; both reproduced the issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants