feat(cost-diagnostics): per-call context ledger — prefix split, window trims, compaction events - #1130
Merged
philmerrell merged 2 commits intoSep 16, 2026
Conversation
…w trims, compaction events
Three content-free facts the 2026-09-15 prod cost audit had to reconstruct
by scanning and guessing, now stored on each model call's `C#` row:
- `prefixTokens: {system, tools}` — the agent's stable static prefix, from
the split the context-attribution hook already computes. The fleet
question "how big is the prefix and how much of it is tool schemas" (19%
of September spend; 46% for 16+-tool users) becomes a query.
- `windowRemovedMessages` — the conversation manager's cumulative
`removed_message_count` at the call. A rise between consecutive rows is a
trim (the sliding window or a compaction slice), which re-writes the
cached prefix. Pure-window sessions and compaction spirals took an hour
of fingerprint reading to tell apart; the anatomy now marks the call.
- `compactionEvents: [{kind, checkpoint, summaryTokens, ...}]` — the
decisions the session manager took since the previous call, each with
the summary's token size at that moment: `applied` (restore-time slice
ran), `checkpoint` (a new checkpoint was cut), and `forced` /
`floor_unreachable` reserved for the scheduling policy (#1125), which
calls `TurnBasedSessionManager.record_compaction_event(kind, **ints)`.
This is what will show a summary cap shrinking summaries and a
scheduling rule stopping re-writes without another table scan.
`ContextLedgerHook` mirrors the tool census: per-turn, per-model-call, read
(never drained) at turn end, off with `COST_DIAGNOSTICS_ENABLED=false` so an
absent field reads "not tracked", never 0. Session rows gain
`compactionAppliedCount` / `compactionForcedCount` /
`compactionFloorUnreachableCount` via atomic ADD; `checkpoint` stays on
`compactionCount`, which `_save_compaction_state(record_event=True)`
already bumps, so one event never feeds two counters.
The admin API projects the fields (content policy widened; numbers only),
derives `windowTrimmed` per row so readers need not diff rows, and the
profile reports the latest prefix split, trim calls, event counts by kind
and the last summary size, with `dataCoverage` flags for each. The anatomy
page adds a static-prefix tile (system · tools), a window-trims tile, the
compaction kinds with the latest summary size on the compactions tile, and
per-row `trim −N` / event badges with the numbers in the expanded row.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Resolves the one integration conflict with the compaction stack (#1125, #1128, #1129, #1131, #1132) in ``update_after_turn``. Both sides append to the same post-cut block: the stack added ``_emit_compaction_metrics`` and refactored to a local ``state`` alias (``state = self.compaction_state``, so the two save calls were already equivalent); this branch added the ledger's ``checkpoint`` event. Keep both, and route the event through the stack's ``_record_ledger_event`` seam instead of calling ``record_compaction_event`` directly, so the recorder stays resolved-by-attribute like every other cut decision. ``test_no_ledger_is_a_noop`` asserted the recorder was *absent* — true only while this branch was unmerged. It now simulates a ledger-less build via ``monkeypatch.delattr`` so it still guards the getattr seam. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The 2026-09-15 prod cost audit (memory + PR #1125 comment) needed three facts that nothing persists, so each took a DynamoDB scan and a proxy:
metadataevent. Fleet answer via scan: tools+system carrying cost is $132 of $699 September Sonnet 5 spend (46% of spend for 16+-tool users).compaction_forced/compaction_floor_unreachableas logs; proving PR-2 (summary cap) and PR-3 (scheduling) work needs them as stored, queryable facts.What
Per-call
C#row extras (all numbers, gated byCOST_DIAGNOSTICS_ENABLED, absent = "not tracked"):prefixTokens: {system, tools}— read from the attribution hook's cached split at store time.windowRemovedMessages—conversation_manager.removed_message_countat the call (newContextLedgerHook, same lifecycle asToolCensusHook).compactionEvents: [{kind, checkpoint?, summaryTokens?, summarizedTurns?, retainedMessages?, truncatedToolResults?, inputTokens?}]— drained fromTurnBasedSessionManager.record_compaction_event(kind, **ints). Wired today forapplied(restore-time slice) andcheckpoint(post-turn advance);forced/floor_unreachableare reserved for feat(compaction): model-relative thresholds, floor-seeking cut, hysteresis (PR-1) #1125 to call (contract sent to that branch's author).Session row counters via atomic ADD:
compactionAppliedCount,compactionForcedCount,compactionFloorUnreachableCount.checkpointdeliberately stays on the existingcompactionCountso one event never feeds two counters.Admin API: projections widened (
CALL_ROW_PROJECTION,SESSION_ROW_PROJECTION);SessionCallRowgains the three fields plus a derivedwindowTrimmed(delta vs the previous ledger-bearing row);SessionProfilegainsprefixTokens,windowTrimCalls,windowRemovedMessages,compactionEventCounts,lastSummaryTokens;ContextTrajectoryPointgainswindowTrimmed/compaction;DataCoveragegainsprefixTokens/windowTrim/compactionEvents.SPA anatomy page: "Static prefix" tile (total, system · tools), "Window trims" tile (calls preceded by a trim, messages removed), compaction kinds + latest summary size under the Compactions tile, and per-row
trim −N/ event badges with the numbers in the expanded row.Content-free by construction: the recorder keeps ints only (floats truncated, strings and bools dropped), the ledger is bounded (8 events per call), and every new path is in the content-policy projections.
Tests
test_context_ledger_hook.py(7),test_compaction_event_ledger.py(4),test_context_ledger_attach.py(4),test_context_ledger_persistence.py(3, moto: row extras + ADD counters + kill switch + malformed), profile service (+3), route coverage dict updated.tests/architecture tests/costs tests/apis/app_api/admin/costs tests/agents/main_agent/session tests/agents/main_agent/streaming tests/shared.tsc --noEmitclean; anatomy page spec 21 passed, util spec 8 passed.Related: #1125 (compaction thresholds), #1127 (deleted conversations in the drill-down).
🤖 Generated with Claude Code