feat(documents): offload unpinned documents to their digests when the re-write is free (offload PR-4)

[philmerrell]

Stacked on #1139 (PR-3) → #1138 (PR-2) → #1137 (PR-1). Merge in order; this PR's diff is the one commit on top. CI runs only for PRs targeting develop/main, so no checks show here until the stack is re-targeted — the suite result below is the local run. This is the cost work and is independently revertible (kill switch, or drop this commit); PRs 1–3 stand on their own as the correctness fix.

Why

A document enters the cacheable prefix on the turn that attaches it and is re-written in full on every cold turn for the life of the session: cacheWrite is 48–50% of attachment-session spend, and the 5-minute TTL cliff is plainly visible in the cold re-write rate (3% under a minute, 66% past five). The digest (PR-2) is a small fraction of a large PDF and the model can pull any page back with document_read (PR-1), so once the document has had its turn the bytes can leave the prefix. The evaluation spec's recoverable envelope is the document share of every cold re-write, which PR-1's documentTokens now measures per row.

Why head-of-turn, not update_after_turn

The spec's §4C put the trigger post-turn. The compaction stack has since moved to "decide post-turn, apply at the head of the next turn only when the prefix re-write is free or unavoidable" (apply_pending_compaction, thresholds spec §3.5) — and that predicate is exactly this spec's cache-aware rule (spec §8 item 21). So the offload runs in the same slot, right after the parked cut is applied, on the same facts: cache_expired (past the TTL since the last turn), prefix_changed (model/agent key differs), or over_ceiling (the last turn's input exceeded the compaction ceiling). Otherwise nothing moves. One decision point for every prefix mutation.

What

• session/document_offload.py — pure functions over the message list: pinning (§4D: the attach turn and the next, DOCUMENT_OFFLOAD_PIN_TURNS=2; the incoming or previous prompt names the document's stem; a document_read result for it sits in the recent turns), the size floor (DOCUMENT_OFFLOAD_MIN_TOKENS=5000, bytes/4), the rollout bucket (crc32(session_id) % 100 < DOCUMENT_OFFLOAD_ROLLOUT_PERCENT, default 100; DOCUMENT_OFFLOAD_ENABLED=false kills), the in-place replacement, and slice ageing.
• The replacement is PR-3's restore transformation applied early — same matcher, same persisted digest, same renderer — so an offloaded block is byte-identical to what the next cold restore would render for it. Tested. Unmatched blocks stay inline (the live path may keep bytes; restore may not).
• TurnBasedSessionManager.apply_document_offload owns the cache-gap decision (reading updated_at, last_prefix_key, last_input_tokens, or an in-process last-turn stamp when compaction is off), mutates in place (never rebinds agent.messages, the @-mention forks conversation history: the mention turn and the plain turns run on two different cached agents #741 alias), records document_offload on the ledger with documents, documentTokens (evicted), digestTokens, slices, sliceTokens, cacheGapSeconds, and emits DocumentOffloaded / DocumentOffloadedTokens / DocumentSlicesAged / DocumentOffloadCacheGapSeconds with the reason as a property. Never raises.
• document_read page slices are aged on both paths: older than DOCUMENT_SLICE_MAX_TURNS=2 turns they become a deterministic stub ([Retrieved pages placeholder: name=…]) on the live path under the same gate and on restore in initialize, so a warm agent and a cold restore agree.
• The stream coordinator calls the new method right after apply_pending_compaction, passing the incoming prompt so a document the user just named stays pinned. CompactionEvent (py + TS) gains the offload fields; the anatomy's event title shows them.
• Spec: PR-4 row marked built; §8 decisions 21–26.

What this does not change

• The attach turn: the full document still goes inline, and it stays inline on the next turn whatever the cache state.
• Nothing moves while the cache is live and the prefix is unchanged and the context is under the ceiling (tested) — the 72%-while-live violation the compaction PR-3 rule fixed cannot recur here. over_ceiling and prefix_changed offloads carry a short cacheGapSeconds by design and carry their reason.
• Pinning does not use Strands' pin_message metadata (spec §8 item 23): our slice is not SummarizingConversationManager, and message metadata is persisted to AgentCore Memory.
• PR-5 (the existing truncation deferral's cache-live guard), PR-6, PR-7 are untouched.

Tests

tests/agents/main_agent/session/test_document_offload.py (22): pin release after two turns, prompt-name pinning (incoming and previous prompt), document_read-result pinning, short stems ignored; candidate selection (unpinned + over the floor; digests and s3Location never candidates); rollout bucket stability, percent and kill switch; in-place replacement that equals the restore output byte-for-byte, unmatched stay inline, lookup failure leaves everything inline; slice ageing (old stubbed, recent kept, idempotent, too few turns); the session manager's three reasons and the wait, pinned documents never move even when free, compaction-off path, kill switch and bucket, slices under the same gate, never-raise; restore ageing through initialize.

Local backend suite (stub openpyxl on PYTHONPATH, verification only): 8901 passed, 3 skipped; the only 33 failures are the two spreadsheet-preview test files, which need the real openpyxl the stub cannot stand in for. CI's install settles those.

🤖 Generated with Claude Code