feat(efficiency): cut prompt-cache misses and background token spend - #555
Merged
Merged
Conversation
Sessions and threads were being rotated, and background lanes were paying for premium or invisible requests, in ways that never reached the user as a feature. This change removes those costs without changing what a turn does. Cache-preserving session identity - MCP change detection compares a content hash of the MCP-relevant sections (`*mcp*` keys, `[mcp_servers.*]` tables, manifest/.mcp.json) instead of mtime+size, so CLI permission saves, state rewrites, and manifest rewrites with identical content no longer discard every task session. - Codex thread keys no longer include the developer-instructions hash. Codex only re-renders `developer_instructions` when it builds a new context window, so the runtime now remembers the instruction profile each thread last saw and prepends a one-time `[Stave Instructions Update]` block to the next turn when it differs, instead of cold-starting the thread. - Auto routing keeps the previous model when the primary route would step down to a weaker rung of the same provider, so a cheaper-looking turn does not throw away the cached context. Per-turn injection - `stave:current-task-awareness` joins the dedup set and collapses to a one-line pointer when unchanged; the duplicated plan-directory lines leave workspace guidance; the ACP runtime applies the same dedup and commits after the prompt is accepted. - The local MCP server publishes a short `instructions` string naming the tool families and the injected-context rule, so hosts that defer tool schemas can still pick the right family without probing. Background lanes - Auxiliary lanes, turn summaries, and kickoff suggestions no longer request provider fast modes. - Inline completion defaults to off for new policies. - The prompt-suggestions setting can only turn suggestions off; callers that pass `false` (aux lanes, secondary runs, control queries) stay off even when the setting is on. Observability - Message usage badges tag a cache miss with its likely cause (provider, model, or native session change; otherwise a rebuilt context) and the number of tokens re-cached. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
astyfx
requested review from
JongSikLim and
png-1105
and removed request for
a team
September 17, 2026 08:21
|
| Commit | Scanned at | New | Resolved | Net |
|---|---|---|---|---|
8825e9d < |
2026-09-17 08:21 UTC | 0 | 0 | 0 |
Last scanned: 8825e9d · 2026-09-17 08:21 UTC
|
| Commit | Scanned at | New | Resolved | Net |
|---|---|---|---|---|
8825e9d < |
2026-09-17 08:21 UTC | 0 | 0 | 0 |
Last scanned: 8825e9d · 2026-09-17 08:21 UTC
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes session/thread rotations and background requests that cost tokens without producing anything a user can see. Turn behavior is unchanged; what changes is how often a cached prefix is thrown away and which lanes get to spend a request.
Changes
Features
settings.json, CLI state, or an identical manifest no longer discard task sessions[Stave Instructions Update]block on its next turn instead of a cold startstave:current-task-awarenessis deduplicated to a one-line pointer when unchanged; the ACP runtime applies the same dedupinstructionsstring naming tool families and the injected-context ruleFixes
falseno longer get re-enabled by the user's settingRefactors / Chores
[Stave Workspace Context]Test Evidence
bun run typecheck✓bun run check:doc-paths,bun run check:max-lines-ratchet✓bun teston the touched areas (mcp-config-refresh, retrieved-context-dedup, current-task-awareness, auxiliary-inference-policy, auto-routing-profile, usage-cache, message-usage-summary, codex-app-server-runtime, codex-app-server-mcp-lifecycle, codex-app-server-secret-env, claude-sdk-runtime, worker-mode-adapters, stave-mcp-server-instructions, stave-collaboration-tools, provider-runtime-options, ipc-schemas): all pass, 0 failuresNotes
developer_instructionsare rendered only when a new context window is built (first turn / compaction), not onthread/resume; the refresh block exists because of that. The instruction profile is tracked in memory, so the first resumed turn after an app restart refreshes once.cheapestAtceiling behave as before.🤖 Generated with Claude Code