feat(compaction): offload oversized tool results at intake (PR-4) - #1131
Merged
Merged
Conversation
…rics (PR-2) The compaction summary was an unbounded join of AgentCore LTM ConversationSummary records (165k chars / ~40k tokens in the #833 incident): a summary that is 40% of the threshold guarantees compaction can never get back under it. Spiral-spec PR-2; thresholds spec §3.6 / §7.1. - compaction_summary.bound_summary(): hold the persisted summary at COMPACTION_SUMMARY_TOKEN_BUDGET (8,000 tokens, chars/4 — the same estimate the admin SUMMARY_OVER_BUDGET diagnosis uses). Within budget → unchanged. Over budget → one Nova Micro converse call (side-channel, never touches agent.messages) with a prompt that keeps standing instructions, decisions, current state of the work, open items and exact identifiers, and drops narration and superseded drafts. Model failure, a ceiling-hit generation or an overshoot → newest-first truncation (keep the newest records that fit; if none fit, the tail of the newest). Runs once at checkpoint advance — the turn that already pays a prefix re-write — and the result is persisted verbatim, so the byte-stability contract is unchanged. - Kill switch AGENTCORE_MEMORY_COMPACTION_SUMMARY_MODEL_ENABLED=false skips the model and truncates; AGENTCORE_MEMORY_COMPACTION_SUMMARY_MODEL_ID selects the model. - Provenance on the persisted compaction.policy map: summarySource (ltm|fallback), summaryOutcome, summaryTokensBefore/After, summaryTokenBudget. - One content-free EMF record per cut in AgentCoreStack/Compaction: CompactionCut, CompactionForced, CompactionInputTokens, CompactionRetainedTokens, CompactionSummaryTokens, CompactionSummaryOverBudget, with policySource/window/ceiling/floor/ summaryOutcome as queryable properties. Silenced by PROMPT_CACHE_OBSERVABILITY_ENABLED=false with the rest of the layer. - forced flag narrowed to "ran while disarmed" (same hunk as the PR-1 fix). Tests: newest-first truncation, within-budget passthrough, model compression, model failure / ceiling / overshoot fallbacks, kill switch, env loading, oversized LTM join bounded and persisted through update_after_turn, EMF record shape and kill switch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…te is free (PR-3) Compaction decided WHAT to cut (PR-1) and bounded the summary (PR-2), but a cut still landed at the next restore regardless of whether the prompt-cache prefix was warm, and never landed at all on a warm (cached) agent. Under Bedrock caching a cut costs one re-write of what survives, so the cheapest turn to pay it on is one that was going to re-write anyway. Thresholds spec §3.5. - update_after_turn PARKS the cut: pendingCheckpoint / pendingSummary / pendingHardCeiling / pendingSince on CompactionState. `checkpoint` stays the APPLIED value (what _apply_compaction slices at on restore). A second over-ceiling turn while a cut is parked is a no-op, never a deeper cut. - apply_pending_compaction(agent, prefix_key) runs at the head of every turn (stream coordinator, right after the turn lease is stamped), on cached and freshly restored agents alike. It promotes the pending cut and slices agent.messages IN PLACE (slice assignment, never rebinding — the #741 alias) when, in order: cache_expired (more than cache_ttl_seconds since the previous turn), prefix_changed (model|agent key differs from the persisted lastPrefixKey), or hard_ceiling (previous input reached the hard ceiling the cut was computed under). Otherwise it waits. - The in-place result is byte-identical to what _apply_compaction derives from stored history under the promoted state (pinned by test_live_apply_matches_a_cold_restore_of_the_same_state), so a cold restore after a live apply reads the same prefix. - _adopt_session_conversation copies _live_offset when it points a new agent at the live list — the list's coordinate system travels with it. - Each application persists the reason, cacheGapSeconds and pendingSince on compaction.policy, logs rewrite_scheduled vs rewrite_forced, and emits CompactionApplied / CompactionAppliedForced / CompactionCacheGapSeconds; the cut record gains CompactionDeferred. - Kill switch AGENTCORE_MEMORY_COMPACTION_DEFERRED_APPLY_ENABLED=false applies immediately (PR-1/2 behavior); legacy mode is always immediate. The shared test fixture pins the immediate path explicitly; deferral has its own suite. Not in this PR: context_window_limit on the Strands model config (needs the window at agent construction). Tests: parking, no-deeper-cut, kill switch, legacy; warm/same-prefix waits; cache-expired / prefix-changed / hard-ceiling apply; first-ever key is not a change; live apply == cold restore parity; pending beyond the live list is dropped; re-arm and cut again after apply; metrics with reason; state round-trip; offset sync on alias. Full backend suite: 8659 passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The apply is the moment the bytes the model sees change, so it is the event the cost anatomy should mark. Same attribute-resolved helper as the PR-1 events; no-op until the ledger lands. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
philmerrell
force-pushed
the
feature/compaction-deferred-apply
branch
from
September 16, 2026 05:05
6e7d3a3 to
eab5119
Compare
philmerrell
force-pushed
the
feature/compaction-offload-escalation
branch
from
September 16, 2026 05:05
cffc32d to
f8aecce
Compare
promoted=1 distinguishes the pending-cut promotion from the restore slice's own applied event without a schema change; both are real byte changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The case that defeats the compaction floor is a huge tool result inside the
protected tail: the last N turns are kept whole, so no cut can bring the
session back under the ceiling (compaction_floor_unreachable). Cutting
inside the tail would drop the turn the user is working on; the right move
is to keep the reference and drop the bytes. Thresholds spec §3.6.
Built as intake offload rather than cut-time escalation: Strands 1.55's
vended ContextOffloader bounds an oversized tool result on AfterToolCallEvent,
before it is appended to the conversation, so the persisted message already
carries the bounded form and restore reproduces it byte-for-byte — nothing
in the prefix is ever mutated after the fact. Cut-time escalation would have
mutated protected turns and needed a restore-replay of every edit, and the
workspace tools it would have escalated into (workspace_files) are granted
to no prod role.
- core/tool_result_offload.py: BoundedToolResultOffloader (the vended plugin
plus a chars/4 pre-filter so its per-result CountTokens round trip only
runs for results near or over the gate, and a content-free
ToolResultOffloaded EMF record per offload). Storage: unified
strands.storage.S3Storage in the user-files bucket under
compaction-offload/{userId}/{sessionId}/ — one namespaced storage per
agent, so references are session-scoped by construction and an @-mention
agent resolves the same ones. evict_after_cycles=None: eviction from the
model path would turn a retrieval into a miss mid-turn. Gate 4,000 tokens
/ preview 1,000 (AGENTCORE_TOOL_RESULT_OFFLOAD_MAX_TOKENS / _PREVIEW_TOKENS);
AGENTCORE_TOOL_RESULT_OFFLOAD_ENABLED=false removes the plugin. Fail-open
everywhere: no bucket, construction failure, storage or CountTokens error
all keep the original result.
- ChatAgent._create_agent appends the plugin. It registers
retrieve_offloaded_content itself (pattern / line range / full) — one
stable spec in toolConfig, not an RBAC-gated tool, on the read_skill_file
precedent.
- CDK: 90-day lifecycle expiry on the compaction-offload/ prefix of the
user-files bucket (merged into its existing rules; shortest expiration
wins for the prefix). The runtime role already had Put/Get on the bucket.
- Cut record gains CompactionFloorUnreachable — the residual after intake
offload (attachments, sub-gate results).
- User attachments are deliberately not touched: the digest + page-range
read tool in document-context-offload.md is the right shape for those.
- Kaizen review-queue [2026-07-19] ContextOffloader spike closed as adopted.
Tests: flag / no-bucket / per-session prefix / config / preview clamp;
small result skips CountTokens; borderline result is measured; oversized
result is offloaded with preview + reference and a content-free metric;
storage and CountTokens failures keep the original; ChatAgent wiring with
and without a bucket. Full backend suite: 8670 passed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
philmerrell
force-pushed
the
feature/compaction-offload-escalation
branch
from
September 16, 2026 05:09
f8aecce to
49e610f
Compare
philmerrell
changed the base branch from
feature/compaction-deferred-apply
to
develop
September 16, 2026 14:46
philmerrell
added a commit
that referenced
this pull request
Sep 16, 2026
Resolves the one integration conflict with the compaction stack (#1125, #1128, #1129, #1131, #1132) in ``update_after_turn``. Both sides append to the same post-cut block: the stack added ``_emit_compaction_metrics`` and refactored to a local ``state`` alias (``state = self.compaction_state``, so the two save calls were already equivalent); this branch added the ledger's ``checkpoint`` event. Keep both, and route the event through the stack's ``_record_ledger_event`` seam instead of calling ``record_compaction_event`` directly, so the recorder stays resolved-by-attribute like every other cut decision. ``test_no_ledger_is_a_noop`` asserted the recorder was *absent* — true only while this branch was unmerged. It now simulates a ledger-less build via ``monkeypatch.delattr`` so it still guards the getattr seam. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1129 (PR-3) → #1128 (PR-2) → #1125 (PR-1). Merge in order; this PR's diff is the one commit on top.
Why
The case that defeats the compaction floor is a huge tool result inside the protected tail. The last N turns are kept whole, so no cut can bring the session back under the ceiling (
compaction_floor_unreachable). Cutting inside the tail would drop the turn the user is working on; the right move is to keep the reference and drop the bytes. Thresholds spec §3.6.Why intake, not cut-time escalation
Strands 1.55's vended
ContextOffloaderbounds an oversized tool result onAfterToolCallEvent, before it is appended to the conversation. The persisted message already carries the bounded form, so a restore reproduces it byte-for-byte and nothing in the prefix is ever mutated after the fact. Cut-time escalation would have mutated protected turns and needed a restore-replay of every edit. It would also have escalated into the workspace tools, whoseworkspace_filescatalog key is granted to no prod role, so the recovery path would have shipped dark.What
BoundedToolResultOffloader(core/tool_result_offload.py): the vended plugin plus a chars/4 pre-filter so its per-resultCountTokensround trip only runs for results near or over the gate, and a content-freeToolResultOffloadedrecord per offload.strands.storage.S3Storagein the user-files bucket undercompaction-offload/{userId}/{sessionId}/. One namespaced storage per agent, so references are session-scoped by construction and an@-mention agent resolves the same ones.evict_after_cycles=None: eviction from the model path would turn a retrieval into a miss mid-turn; a 90-day S3 lifecycle rule expires the objects instead (CDK, merged into the bucket's existing rules).AGENTCORE_TOOL_RESULT_OFFLOAD_ENABLED=falseremoves the plugin. Fail-open everywhere: no bucket, construction failure, storage orCountTokenserror all keep the original result.ChatAgent._create_agentappends the plugin. It registersretrieve_offloaded_contentitself (pattern / line range / full): one stable spec intoolConfig, not an RBAC-gated tool, on theread_skill_fileprecedent.CompactionFloorUnreachable, the residual after intake offload.document-context-offload.mdis the right shape for those and stays that spec's PRs 1–4.Behavior to know about
Tool results over ~4k tokens now reach the model as a 1k-token preview plus references, and the model has a tool to read any span back. The runtime role already had Put/Get on the user-files bucket.
toolConfigHashflips once per session shape when the new tool appears; that is expected and stable afterwards.Tests
Flag, no-bucket, per-session prefix, config and preview clamp; small result skips
CountTokens; borderline result is measured; oversized result is offloaded with preview + reference and a content-free metric; storage andCountTokensfailures keep the original;ChatAgentwiring with and without a bucket. CDK type-checks. Full backend suite: 8670 passed, 3 skipped.🤖 Generated with Claude Code