feat(compaction): bound the summary at 8k tokens and emit per-cut metrics (PR-2)

[philmerrell]

Stacked on #1125 (PR-1). Merge that first; this PR's diff is the one commit on top.

Why

The compaction summary is an unbounded join of AgentCore Long-Term Memory ConversationSummary records. In the #833 incident it was 165k chars (~40k tokens) against a 100k threshold, so compaction could never get back under the ceiling and every turn re-wrote a ~200k prefix. The 2026-09-15 prod audit found 23k–40k-token summaries in 6 of 20 audited sessions, and all 16 over-100k sessions were on the agent-cache bypass path where the restore slice already runs and is defeated by exactly this. Spiral-spec PR-2; thresholds spec §3.6.

What

• compaction_summary.bound_summary() holds the persisted summary at COMPACTION_SUMMARY_TOKEN_BUDGET (8,000 tokens, chars/4 — the same estimate the admin SUMMARY_OVER_BUDGET diagnosis uses).
  • Within budget → unchanged (small sessions see no difference).
  • Over budget → one Nova Micro converse call, side-channel like titles and tool-batch summaries (never touches agent.messages), with a prompt that keeps standing user instructions, decisions, the current state of the work, open items and exact identifiers, and drops narration and superseded drafts.
  • Model failure, a ceiling-hit generation or an overshoot → newest-first truncation: keep the newest records that fit; if none fit, the tail of the newest. Never oldest-first.
  • Runs once at checkpoint advance (the turn already paying a prefix re-write) and the result is persisted verbatim, so the byte-stability contract is unchanged.
• Kill switch AGENTCORE_MEMORY_COMPACTION_SUMMARY_MODEL_ENABLED=false (truncate only); AGENTCORE_MEMORY_COMPACTION_SUMMARY_MODEL_ID selects the model.
• Provenance on the persisted compaction.policy map: summarySource, summaryOutcome, summaryTokensBefore/After, summaryTokenBudget.
• One content-free EMF record per cut in AgentCoreStack/Compaction: CompactionCut, CompactionForced, CompactionInputTokens, CompactionRetainedTokens, CompactionSummaryTokens, CompactionSummaryOverBudget, with policy source / window / ceiling / floor / summary outcome as queryable properties. No conversation or summary text is emitted. Silenced by PROMPT_CACHE_OBSERVABILITY_ENABLED=false with the rest of the cost observability layer.

Tests

Newest-first truncation; within-budget passthrough; model compression; model failure, ceiling-hit and overshoot fallbacks; kill switch; env loading; an oversized LTM join bounded and persisted through update_after_turn with the restore prepending the same bytes; EMF record shape and kill switch. Full backend suite: 8643 passed, 3 skipped.

Not in this PR

Applying the pending cut in place on warm agents with paid-when-free scheduling (PR-3), and offload escalation when the protected tail alone exceeds the floor (PR-4). The spec's §7.2 lists the further data points worth collecting.

🤖 Generated with Claude Code

[philmerrell]

Landed on develop via #1129, not through this PR.

Merging #1125 with --delete-branch removed this PR's base branch (feature/compaction-model-relative-thresholds), which auto-closed this PR. GitHub won't reopen a PR whose base branch is gone, so it can't be retargeted to develop and merged on its own.

No work was lost: this stack was linear, so PR-2's commit 667d43c4 ("bound the summary at 8k tokens and emit per-cut metrics") was already contained in #1129's branch and merged with it. Verified on develop:

• git merge-base --is-ancestor 667d43c4 origin/develop → true
• summary_token_budget / COMPACTION_SUMMARY_TOKEN_BUDGET (default 8000) present in compaction_models.py

Closing as superseded by #1129.