fix(agent-runtime): retry and right-size the compaction summary before falling back - #563
Merged
Merged
Conversation
…e falling back Issue #543: on real long sessions most automatic compactions ended as the ~112-token retained-tail recovery notice, and the transcript row labelled each one "summary ≈112 tokens". Three things made the ADR 0049 fallback the common outcome instead of the exception: - compact() was called without a RetryPolicy, so pi-ai returned the first failed response as-is. The main turn retries transient failures through the streamFn wrapper, but the summary goes through Models.completeSimple and never reaches it; one dropped stream or 503 discarded the summary. - The preflight guard summed estimateTokens over the raw messages, while pi-agent-core serializes the conversation to text and caps every tool result at 2 000 characters. Tool-heavy sessions looked several times larger than the prompt that would be sent and skipped the model for summaries that would have fit. - ContextCompactionMark could not tell a fallback from a summary, so the row rendered the recovery notice as a successful summary of N tokens. The summary request now carries a bounded pi-ai RetryPolicy (3 retries, 2s/4s/8s, abort-aware; pi-ai's classifier decides what is transient, so quota and auth failures still return at once). The guard sizes the prompt pi actually serializes, and when that still exceeds the window it tries exactly one reduced input (tool results cut to a 500-char prefix, thinking dropped, no message removed) before the fallback runs. The mark gains an additive `fallback?: "retained_tail"` derived from persisted details, and the row labels such checkpoints as a failed summary in all locales. Supersedes PR #554, which retried every recoverable failure including deterministic ones and shrank the input by dropping the oldest messages while the checkpoint still claimed to cover them. Records ADR 0282 / D445 and updates E2E-084, the runtime spec, and the IPC mark shape.
This was referenced Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #543. Supersedes #554 (direction kept, mechanisms replaced — see ADR 0282 "Alternatives").
Root cause
On real long sessions most automatic compactions ended as the ~112-token retained-tail recovery notice, and the transcript row labelled each one
summary ≈112 tokens. Three things inpackages/agent-runtimemade the ADR 0049 fallback the common outcome:compact()was called withretry: undefined, so pi-ai returned the first failed response as-is. The main turn retries transient failures through thestreamFnwrapper (D186), but pi-agent-core's summary goes throughModels.completeSimpleand never reaches it. One dropped stream or 503 discarded the whole summary.compactionSummaryWouldExceedBudgetsummedestimateTokensover the raw messages, while pi-agent-core serializes the conversation to text and caps every tool result at 2 000 characters. A tool-heavy session looked several times larger than the prompt that would be sent, and the guard skipped the model for summaries that would have fit. The reporter'stokensBeforevalues (200k–885k on a 200k window) are exactly this shape.ContextCompactionMarkonly distinguishedfresh_window; a retained-tail checkpoint rendered as a successful summary of N tokens.Change
generateCompactionpasses pi-ai a boundedRetryPolicy(3 retries, 2s/4s/8s, abort-aware). pi-ai's classifier decides what is transient; quota/auth/malformed requests still return at once.convertToLlm+serializeConversation). When that still exceeds the window, exactly one reduced input is tried (tool results cut to a 500-char prefix with a visible marker, thinking dropped, no message removed) before the fallback runs. The checkpoint'smessagesToSummarize/retainedTailare the originals; only the request payload is reduced.ContextCompactionMarkgains an additivefallback?: "retained_tail"derived from persisteddetails.fallback.CompactionRowrenders it as "summary generation failed · recent context retained" in all 8 locales. Historical fallback rows get the honest label too, since the mark is derived on session open.No record schema, protocol version, or host-core change.
fresh_windowuntouched. Manual/compactinherits retry + sizing and keeps fail-fast semantics.Why not #554 as-is
recoverablefailure, including deterministic ones ("no new context", 401, quota), sleeping 2s+4s each time.throughMessageIdstill covered them, so the summary silently claimed a range it had not seen.main; no docs/ADR.Docs
ADR 0282 (amends ADR 0049 decision 1) · D445 ·
02-agent-runtime.md·01-ipc-protocol.mdmark shape · E2E-084 · zh mirrors.Validation
pnpm --filter @pi-desktop/shared test— 748 passedpnpm --filter @pi-desktop/agent-runtime test— 589 passed (8 new runtime tests: retry-then-succeed, deterministic no-retry, retry budget exhausted, abort during backoff, guard sizes serialized prompt, reduce-once, fallback only when reduced still overflows, reduced prompt reaches the model with the checkpoint covering every message; 9 unit tests on the new module)pnpm --filter @pi-desktop/i18n test— 25 passed (key parity across 8 locales)apps/desktopcontext-compaction+assistant-turnscontract tests — 29 passedpnpm build:js, desktop + agent-runtimetypecheck,pnpm lint,pnpm docs:check,check-architecture,check:agent-policy— all greenNOT RUN: live-provider E2E (E2E-008/009) — no test API key in this environment; the retry path is covered by scripted
completeSimpleresponses through the real pi-airetryAssistantCall. Residual risk: a sustained provider outage now delays the fallback by up to ~14s (bounded, Stop-cancellable).