Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions docs/superpowers/plans/2026-09-05-remote-transcript-retention.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Remote transcript retention

Status: implemented and locally verified. Issue #805. Depends on #813 (fix/remote-output-backpressure,
61823921); separate PR against that branch. Main remains 5d641845. No edits to
A5 desktop rendering/worktree paths or the external operator toolkit.

## Intended behavior and implementation

1. Treat actual view subscriptions as transcript ownership. Unviewed sessions
retain lightweight status/file identity only; do not map or fold their JSONL
and semantic bodies. Release entries, indexes, UUID bookkeeping, mapper and
semantic state on last unsubscribe. Invalidate pending history responses and
backfill again when selected. Partial semantic turns wait for a fresh start.
2. Reuse the desktop pure live trim planner and marker helpers without editing
desktop code or using its global registries. Apply count/estimated-byte
targets to active live appends; preserve current/history semantic ownership,
cross-entry tool pairs, stable UUIDs and pagination anchors. Suspend trimming
while paging and briefly after older history is loaded.
3. Rebuild tool indexes from the retained window. Track trimmed UUID tombstones
per view so old live replays cannot reappear at the tail, but explicit older
pages can reload them. These small identity sets last for the viewed window;
all are released on unsubscribe. Correct replay dedupe takes precedence over
bounding UUID count during one continuously viewed session.
4. Carry byte offsets from history chunks into pagination and trim cursors.
Live records lacking offsets use provider markers. Preserve raw-record groups
at a trim boundary so multi-entry mapper output remains reloadable in order.
5. Add regressions for multi-session zero-view retention, detach/reselect and
pending responses, sustained count/byte trimming, replay dedupe, older-page
ordering/exact offsets, tool pair and semantic owner preservation. Adapt
transport tests to explicitly own a view where they assert live rendering.

## Checks, evidence and limitations

Run remote suites, relevant shared trim tests, typecheck, test contract, client
production build and diff check; full repository checks in CI. Compare retained
logical payload/cardinality against the audit's synthetic 4096 entries/32 MiB,
not production heap/latency claims. Safety constraints may pin an active window
above its target; never drop active ownership merely to hit a number. Explicit
older-history reading may exceed the live target during its grace period.
Synchronize issue/PR acceptance criteria and document these constraints. Review
checks/feedback, leave clean committed worktrees; do not merge.

## Implemented evidence and constraints

Ten remote retention regressions use real Claude/Codex/OpenCode mappers and
semantic folds. The zero-view regression fails against the parent #813 store
with 4096 retained entries; after the fix all three synthetic sessions retain
zero entries, tool indexes and seen UUIDs. Local remote plus shared window tests
pass (137 tests / 14 files). This is logical payload/cardinality evidence, not a
browser heap or production throughput benchmark.

A 2100-entry live burst trims to 1500 with matching indexes and totalEntries
unchanged. A 300-entry, 128 KiB-per-entry burst triggers the byte budget below
the count threshold and trims under the shared 24 MiB estimate target. Exact
history offsets survive trims and all-duplicate pages. Older pages reload
trimmed UUIDs in order, while live replay cannot append them at the tail.
Tool-result indexes rebuild chronologically before notifying subscribers so
historical duplicate tool ids cannot overwrite newer retained results.

The pure shared planner retains paired tool entries and semantic owners. The
remote adapter additionally refuses cuts inside one raw provider record; it
never adjusts a planned cut in a way that could invalidate pair safety.
Identity-only tombstones remain for a continuously viewed session, and safety
constraints/history-reading grace can exceed the nominal count/byte targets.
All body and identity state is released on last unsubscribe. Live frames lack
byte offsets and use the provider marker fallback until history supplies an
exact cursor. No protocol migration or desktop implementation change is needed.

Typecheck, test contract, client production build and diff check pass; the
existing client chunk-size/mixed-import warnings remain. Re-selection also
clears the authority of a file hint observed while unviewed, so a newer history
file can establish identity without waiting for another live append. New live
frames in the selected view still win over a stale history reply.

## Clean-build review correction

The first full CI run passed tests and coverage but failed the remote production
build: the shared planner introduces a runtime import of
`agent-transcript-parser/ghost`, whose package export points at unbuilt `dist/`.
A prebuilt parser in the original local dependency links had masked this.
The remote Vite config now aliases that pure leaf to pinned submodule source,
matching the desktop build convention without adding a ghost plane or changing
desktop files. The failure reproduced locally after pointing this worktree's
parser dependency at its own unbuilt checkout; the shared node_modules and
running app were left untouched. The same unbuilt checkout now passes the full `npm run test:package` gate.
The parser still has no dist directory. CI will rerun after rebasing onto
current main f7507980 (toolkit #812 and MCP repair #818).
Original file line number Diff line number Diff line change
Expand Up @@ -277,11 +277,11 @@ describe('WebSocketSessionFeed against a live RemoteServer', () => {
) as { input: { entries: Array<Record<string, unknown>> } }
const durableQueuedPrompt = recordedBundle.input.entries[13]!
await writeFile(transcript, disk.map(d => JSON.stringify(d)).join('\n') + '\n', 'utf8')
;(manager.resolveTranscriptFile as ReturnType<typeof vi.fn>).mockResolvedValue(transcript)

const f = makeFeed()
const store = new TranscriptStore(f)
await waitForOpen(f)
store.subscribe('s1', () => {})

manager.emit('started', { sessionId: 's1', kind: 'claude', projectDir: '/repo' })
// Live entry arrives FIRST (before backfill) — the desktop-order case.
Expand Down Expand Up @@ -311,6 +311,8 @@ describe('WebSocketSessionFeed against a live RemoteServer', () => {
)

// Backfill prepends the older records and skips the duplicate.
;(manager.resolveTranscriptFile as ReturnType<typeof vi.fn>).mockResolvedValue(transcript)
await vi.waitFor(() => expect(store.getSnapshot('s1').loadingOlderHistory).toBe(false))
await store.loadInitialHistory('s1')
await vi.waitFor(() =>
expect(store.getSnapshot('s1').entries.map(e => e.uuid)).toEqual([
Expand Down
3 changes: 3 additions & 0 deletions src/remote-client/src/transcript/store.reconnect.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ function fixture() {
}
},
}) as unknown as WebSocketSessionFeed
getHistory.mockResolvedValue({ ok: false, error: 'No transcript yet' })
const store = new TranscriptStore(feed)
return { store, getHistory, list, emit: (name: string, value: unknown) => { for (const cb of listeners.get(name) ?? []) cb(value) } }
}
Expand Down Expand Up @@ -62,6 +63,7 @@ describe('remote transcript reconnect recovery', () => {

it('ignores an in-flight old history reply after disconnect, even for the same file', async () => {
const f = fixture()
f.store.subscribe('s', () => {})
const old = deferred<{ ok: true; chunk: HistoryChunkResult }>()
try {
f.getHistory.mockReturnValueOnce(old.promise)
Expand All @@ -78,6 +80,7 @@ describe('remote transcript reconnect recovery', () => {
it('uses committed content after reconnect until a complete new semantic turn starts', () => {
const f = fixture()
try {
f.store.subscribe('s', () => {})
f.store.getSnapshot('s')
f.emit('onConnectionState', 'closed')
f.emit('onSessionSemanticEvent', { sessionId: 's', event: { type: 'block_started', source: 'proxy', turnId: 'lost-prefix', blockId: 'b', blockType: 'text' } })
Expand Down
Loading