feat(compaction): model-relative thresholds, floor-seeking cut, hysteresis (PR-1)

[philmerrell]

Why

Compaction fires at a fixed 100k regardless of the model's window, picks its cut by turn count, re-cuts on every over-threshold turn (the #833 spiral: 56 consecutive cuts in one session), and compares a slice-relative index against the persisted absolute checkpoint (spiral-spec D3). Underneath all of it, Strands' default 40-message SlidingWindowConversationManager was running — AgentFactory never set one — and past 40 messages it slid the front of agent.messages every turn: a full prefix re-write per turn, and the reason the checkpoint/anchor coordinates never agreed (ANCHOR_MISMATCH on 14 of 20 sessions in the 2026-09-15 prod cost audit).

Spec rides with this PR: docs/specs/compaction-model-relative-thresholds.md. It answers the original question — should the threshold vary with the context window? — with measured data: the window should scale the ceiling down, never up. A 200k ceiling on Sonnet 5 replayed at 43% more input-side cost than 100k on the 20 audited sessions, because 36% of cache-write dollars are cold re-writes after a >5 min pause and their size is the context at the pause.

What

• Policy (compaction_policy.py): ceiling = min(0.5 × window, 100k), floor = 0.25 × ceiling, hard ceiling = min(0.7 × window, 1.5 × ceiling), from the catalog's maxInputTokens. Unknown window → the fixed threshold. Kill switch AGENTCORE_MEMORY_COMPACTION_MODEL_RELATIVE_ENABLED=false → legacy behavior exactly (fixed threshold, turn-count cut, no hysteresis). Every ratio/cap is env-backed.
• Floor-seeking, token-aware cut: oldest tool-pair-safe cut whose retained estimate is ≤ floor; the last protected_turns are always kept; estimates calibrated to the context-breakdown messages partition.
• Hysteresis: CompactionState.armed (persisted; legacy rows default armed). A cut disarms, a turn under the ceiling re-arms, only the hard ceiling forces a cut while disarmed (compaction_forced log = the previous cut did not take).
• One coordinate system: _live_offset = absolute index of agent.messages[0]; persisted checkpoint = offset + relative cut.
• Explicit conversation window: SlidingWindowConversationManager(window_size=2000, should_truncate_results=True) via AgentFactory.build_conversation_manager() (AGENTCORE_CONVERSATION_WINDOW_MESSAGES; 40 restores the SDK default). Kept rather than NullConversationManager because its reduce_context is the stack's only ContextWindowOverflowException recovery, and that path is independent of window size.
• compaction SSE payload and the persisted compaction map carry the policy fields (additive; the SPA validator ignores unknown keys, the TS interface gains optionals).

What this does NOT change

When history bytes change. The slice still applies only at restore (_apply_compaction). Bounding the summary (8k cap, persisted verbatim) is spiral-spec PR-2 and is next: all 16 over-100k sessions in the audit were AGENT_CACHE_BYPASS, so the restore slice already runs for them and the unbounded 23–40k-token summary is what defeats it. In-place apply on warm agents with paid-when-free scheduling is PR-3.

Behavior change to know about

Conversations between 40 messages and the ceiling now go to the model whole: more 0.1× cache reads per turn, far fewer 1.25× re-writes, and the model sees the conversation rather than its last 40 messages. Above the ceiling the policy bounds it.

Tests

• New: test_compaction_policy.py (table incl. unknown-window and kill-switch rows, estimator, floor-seeking cut, disarm / forced / re-arm, spiral shape cuts exactly once, live-offset coordinates, window flows into policy), test_conversation_window.py.
• Updated: compaction model key set + armed/policy round-trip; SSE emit-once stub accepts the new kwargs and asserts the extended payload.
• Byte-stability suite unchanged. Backend suite: 8622 passed, 3 skipped.

Follow-ups filed

• Hourly system-prompt tick (%H:00 in get_current_date_pacific()) re-writes every session's prefix once an hour (2.6% of September cache-write spend) — a measurement confounder for everything here; spawned as a separate task.
• OVER_COMPACTION_THRESHOLD diagnosis should read the persisted compaction.policy map instead of the fixed default (small, after this lands).

🤖 Generated with Claude Code

[philmerrell]

Threshold replay on 20 audited prod sessions (2026-09-15)

Replay method. Per session, tools+system = min non-zero cacheReadTokens (never compacted); new material per call = positive delta of (cacheRead + cacheWrite + input), or prior call's output + 300 when the real context dropped; a pause > 300 s is cold; PR-3's scheduling rule applied exactly (cut only at a turn start, and only when cold or at the hard ceiling; armed/disarmed hysteresis; hard ceiling 150k for 100k/25k, 300k for 200k/50k). Priced at Sonnet 5 global rates, cache write $2.50 / cache read $0.20 per MTok. Output tokens excluded. Source rows: GET /admin/costs/sessions/{id}/calls on prod, content-free.

Sample: the top-5 September users' three most expensive conversations plus the fleet's seven largest September conversations. All Sonnet 5, all heavy. "Actual" already benefits from the 40-message Strands window bounding contexts, which PR-1 removes, so "no compaction" is the baseline for what PR-1 replaces.

session	peak ctx	actual $	no compaction $	100k/25k $ (cuts)	200k/50k $ (cuts)	Δ 200k vs 100k
54fac08d	88,947	1.86	2.24	1.42 (1)	2.24 (0)	+0.82
0d8ba8f8	88,888	4.18	5.85	3.59 (2)	5.85 (0)	+2.26
e7e75953	78,555	2.56	4.21	2.57 (2)	4.21 (0)	+1.64
c81565a1	231,787	0.72	0.72	0.72 (0)	0.72 (0)	+0.00
39f99c3e	409,318	4.81	5.56	2.45 (3)	2.61 (2)	+0.16
65b6d4ab	203,001	10.93	4.38	2.21 (2)	3.15 (1)	+0.94
a7c6732b	150,345	2.32	3.39	1.89 (2)	2.95 (1)	+1.06
9ee498e2	114,698	2.35	2.43	2.43 (0)	2.43 (0)	+0.00
cbb97a84	126,037	4.27	4.55	4.55 (0)	4.55 (0)	+0.00
4e119c99	131,744	1.75	2.70	2.70 (0)	2.70 (0)	+0.00
56398d91	144,798	4.01	11.15	3.75 (4)	5.77 (2)	+2.02
cbc37993	65,603	0.90	1.05	1.05 (0)	1.05 (0)	+0.00
044c7bd8	103,761	2.40	3.61	2.78 (1)	3.61 (0)	+0.83
a3b884b1	284,476	12.22	17.68	8.16 (3)	13.14 (2)	+4.98
24a74cd9	285,994	9.38	13.92	5.62 (4)	6.93 (2)	+1.30
163c441a	712,695	7.53	9.48	3.40 (2)	3.65 (2)	+0.26
cef8dee6	190,049	8.67	25.86	7.01 (5)	11.13 (3)	+4.12
f00003ba	351,131	7.69	25.42	2.91 (6)	4.50 (5)	+1.59
4193bfa8	173,543	6.58	49.20	8.52 (8)	14.27 (4)	+5.75
1e5b82b9	164,788	7.51	15.44	5.96 (4)	9.83 (2)	+3.87
total		102.64	208.81	73.70	105.30	+31.60

200k/50k is worse than 100k/25k on 13 of 20 sessions and never better, and lands at roughly today's spend. Mechanism: cold re-writes after a > 5 min pause are ~36% of write dollars in this sample and their size is the context at the pause; a 150k-context return costs ~$0.375, a 30–50k one $0.08–0.12, and under 200k/50k most of these sessions never reach the ceiling so they run at 100–190k throughout.

Cohort note for PR-2 vs PR-3 ordering: all 16 over-100k sessions carry AGENT_CACHE_BYPASS (spreadsheet/word/ppt tools enabled), so they rebuild the Agent every turn and the restore slice already runs for them; 65b6d4ab had 17 live full re-writes before the 40-message window ever pinned. e7e75953 and 0d8ba8f8 pinned at 39–41 messages with every turn under 100k — pure window effect.