Repository navigation
fix(ai): stop demoting unsigned and interrupted reasoning to text - #60498
Conversation
There was a problem hiding this comment.
I found one regression worth fixing before merge. Nothing else blocks it.
The fix behaves as described. I compiled requests with unsigned reasoning on the base and on this branch, and ran the changed packages/ai provider tests, the packages/core session runner tests and typecheck:tsc (all pass). On the base, unsigned reasoning became text. Here, recognised signed families drop it and open-weight models get native unsigned thinking.
Public API: adds requireSignature?: boolean to Model.Compatibility and Provider.Compatibility in @opencode/schema, the OpenAPI spec and the generated client types. Adding it is backwards compatible. The default changes, though: for model IDs that match none of the listed families, an unset value used to mean "require a signature" (turn into text) and now means "send unsigned thinking".
Concern: the new default depends on the model ID matching a name pattern. Claude reached through an ID that doesn't contain claude now gets unsigned thinking blocks where it used to get text, which Anthropic answers with HTTP 400. See the inline comment.
| // Closed reasoning families sign or encrypt their thinking blocks; open-weight | ||
| // Messages and Converse implementations replay unsigned reasoning text. | ||
| export const requireThinkingSignature = (model: LLMRequest["model"]) => { | ||
| if (model.compatibility?.requireSignature !== undefined) return model.compatibility.requireSignature |
There was a problem hiding this comment.
The default now depends on the model ID matching a name pattern. When it doesn't match, we send unsigned thinking. That's a regression for Claude reached through an ID without claude in it. I checked two cases:
- A Bedrock application inference profile ARN (
arn:aws:bedrock:…:application-inference-profile/<id>) now lowers foreign reasoning toreasoningTextwith no signature. - A gateway alias such as
sonnet-latestbehind a LiteLLM-style proxy (notapi.anthropic.com) now gets{ type: "thinking", signature: "" }.
Both used to become text, and both now hit the 400s this PR is meant to avoid. The user has to find and set requireSignature themselves. Could unknown IDs keep a safe fallback, for example dropping instead of sending unsigned thinking unless the provider or model opts in? Or could the ARN case be detected (via the inference profile's underlying model)? Either way, I'd add a test for the ARN case.
There was a problem hiding this comment.
The new commit fixes my earlier point. Bedrock application inference profile ARNs and the Claude aliases (sonnet, opus, haiku, duo-chat-*) are now treated as signed, so their unsigned reasoning is dropped instead of being sent. Both cases have tests now. The affected packages/ai provider tests and typecheck:tsc pass. The public API is unchanged since my last review (requireSignature is still the only addition, and it is backwards compatible). I have no new concerns.
b1fd9db to
8b40946
Compare
There was a problem hiding this comment.
The rewrite settles my earlier point. Model IDs outside the allowlist now default to requiring a signature, and Bedrock Converse drops all unsigned reasoning, so Claude reached through a profile ARN or an alias like sonnet-latest can't get unsigned thinking blocks any more. The prefix: true change in Mistral only behaves differently when a trailing message with nothing to send is skipped, and the new test covers that. I re-ran the changed packages/ai provider tests and the packages/core session runner tests, and ran typecheck:tsc in both packages; all pass. The public API is unchanged: the schema, OpenAPI and client changes from the first version are gone. I have no new concerns.
8b40946 to
60755d6
Compare
There was a problem hiding this comment.
The rebase broke two of the tests this PR adds: they still call LLM.request, which dev renamed to ai.request. Both fail with ReferenceError: LLM is not defined (see inline). The rest of packages/ai/test/provider and the packages/core session runner tests pass, as does typecheck:tsc in packages/ai. The only other new change is that Gemini and Interactions now add skip_thought_signature_validator to unsigned tool calls on every model, and the updated tests cover it. My earlier point stays settled. The public API is unchanged.
packages/ai/test/provider/mistral-chat.test.ts:181: Same rebase problem:LLM.requestthrowsReferenceError: LLM is not definedhere. It should beai.request.
| expect(prepared.body.previous_interaction_id).toBeUndefined() | ||
|
|
||
| const unsigned = yield* compileRequest( | ||
| LLM.request({ |
There was a problem hiding this comment.
After the rebase, LLM isn't imported here any more (dev renamed it to ai), so this test fails with ReferenceError: LLM is not defined. Changing this to ai.request fixes it.
60755d6 to
37cabe9
Compare
There was a problem hiding this comment.
The two tests that broke in the rebase now use ai.request and pass. All of packages/ai/test/provider passes, as does typecheck:tsc in packages/ai. The one other new change is in google-interactions.ts: a signed reasoning step now counts like a signed tool call, so unsigned tool calls after it no longer get skip_thought_signature_validator. The public API is unchanged. I have no new concerns.
test(ai): update Gemini rich tool failure test for upstream anomalyco#60498
## Summary - OpenCode **2.2.5** becomes the **recommended** (qualified) runtime. The **2.2.3 minimum** and the 2.1.0 `@opencode/client`/`plugin`/`schema` pins stay: neither 2.2.4 nor 2.2.5 changes a contract CodeNomad consumes. - `RECOMMENDED_OPENCODE_VERSION` is 2.2.5, and the reviewed-publication shortcut in `contractProfile` lists 2.2.3 and 2.2.5; other versions, 2.2.4 included, keep negotiating their authenticated schema. The CI `latest` lane already resolves the latest stable CLI. - Help (all ten locales), READMEs, `CONTRIBUTING.md`, `dev-docs/OPENCODE_COMPATIBILITY.md` and the architecture skill state two separate versions again: minimum 2.2.3 with its reason, recommendation 2.2.5. ## Contract review (v2.2.3 → v2.2.4 → v2.2.5; 31 + 19 commits) Reviewed from the exact tags `v2.2.3` (`082ac6281f`), `v2.2.4` (`45898d0f85`) and `v2.2.5` (`9bbb8ce18c`), the published npm packages, and an authenticated OpenAPI dump from each official CLI running in its own sandbox. The [changelog](https://opencode.ai/changelog) has full notes for 2.2.4 and 2.2.5 (both 30 October); everything below was also checked against the commit log and source. CodeNomad side: `origin/dev` `605820669`. **Headline:** neither release changes a contract CodeNomad consumes. 2.2.4 adds opt-in **artifacts** (seven `/api/experimental/artifacts` operations, four tools, a plugin hook), **renamable persistent terminals**, and moves background-job recovery markers from `kv` into a new **`job` table**. 2.2.5 is a bug-fix release; its one schema change is that `compaction.buffer` accepts a percentage such as `"20%"`. The image tool contract adopted in #1209 (`experimental_image_generate`, on by default) is unchanged through 2.2.5, and still nothing writes `session_part`. - **Authenticated OpenAPI:** 134 → **138** → 138 paths, 162 → **169** → 169 operations, schemas 351 → 362 → 363. - **Added operations (2.2.4):** `GET /api/experimental/artifacts/providers`, `GET|POST /api/experimental/artifacts`, `GET|PATCH|DELETE /api/experimental/artifacts/{artifactID}`, `GET /api/experimental/artifacts/{artifactID}/view/*` ([#60351](anomalyco/opencode#60351)). None removed in either release. - **Changed operations (2.2.4):** `GET /api/event`, where the `types` filter enum gains `persistent-pty.updated` and `artifact.updated` (94 → 96 types, none removed). `PUT /api/experimental/persistent-pty/{ptyID}` can also rename (`title`; `null` clears it), `size` becomes optional, and a 501 `PtyFeatureUnsupportedError` is returned while an older terminal service can't rename ([#60329](anomalyco/opencode#60329)). - **Changed operations (2.2.5):** none. - **Consumed schemas:** same family selector as the 2.2.3 review, 97 schemas at every version. - 2.2.3 → 2.2.4: 95 byte-identical. The two that change, `PersistentPty.Info` (+ `userTitle`, nullable, required) and `PersistentPty.UpdateInput` (`size` optional, + `title`), are both additive. - 2.2.4 → 2.2.5: all 97 byte-identical. - `V2EventEncoded`, `Session.*`, `Form.*`, `Permission.*`, `Prompt*`, `Event.*`, `Location.*`, PTY/Shell are byte-identical from 2.2.3 to 2.2.5. - **`Config.InfoEncoded`:** 2.2.4 adds `experimental.artifacts` (boolean, opt-in). 2.2.5 turns `compaction.buffer` into `Config.Compaction.Percent | integer` ([#60647](anomalyco/opencode#60647)). `experimental.image_generate`, `experimental.media_blob` and `experimental.schedules` are unchanged. - **Admission:** every predicate in `compatibility/negotiate.ts` (unchanged on origin/dev) holds on 2.2.3, 2.2.4 and 2.2.5. `contractProfile` lists only `2.2.3` as reviewed, so 2.2.4/2.2.5 are admitted through authenticated contract recognition. - **Instance proxy:** the allowlist is unchanged on origin/dev. No added or changed route is proxied: artifacts and persistent PTYs are outside it. The proxied `GET /api/config` returns the additive config fields above. - **Published declarations** (2.2.3 → 2.2.5): - **Client:** 185 → **192** → 192 methods. 2.2.4 adds `experimental.artifacts.{providers,list,create,get,update,delete,view}`, the artifact and `PersistentPtyUpdated` types, and `PtyFeatureUnsupportedError`. It removes `createPtyClient`/`PtyConnectInput` from `@opencode/client/solid/pty`, which CodeNomad doesn't import. 2.2.5 adds `ConfigCompactionPercent`. - **Schema:** 2.2.4 adds the `artifact` module and `PersistentPty.Updated`; 2.2.5 adds `config/compaction` `Percent`. - **Plugin:** 2.2.4 adds `effect/artifact` and `promise/artifact` and `ctx.experimental.artifact` (`transform`, `reload`). 2.2.5 has no export changes. - **Typecheck:** the server and UI typecheck with **0 errors** against the 2.2.4 and 2.2.5 client/schema/plugin/effect declarations. The pinned 2.1.0 control is also clean. - **Host modules:** the bundles' host imports are identical to the 2.2.3 review. A probe plugin in an isolated **2.2.4** and **2.2.5** daemon resolves every module and binding. `runtime-modules.ts` is unchanged from v2.2.3 to v2.2.5. - **Host Effect:** stays **4.0.6** (`bun.lock` sha512 identical; published `dependencies.effect` 4.0.6). - **Persistent PTY binary:** the 2.2.4 and 2.2.5 CLIs ship `@opencode/pty` **0.4.0** (string `opencode-pty 0.4.0` in the Windows binary; 2.2.3: `0.3.3`). `process-daemon-v0.4.0` adds only terminal renaming ([#60324](anomalyco/opencode#60324)): an additive `rename` request and `features: ["rename"]` with an unchanged `PROTOCOL_VERSION`. CodeNomad only lists persistent PTYs (worktree deletion guard) and reads `status`/`cwd`. - **Private native services:** the keys (`LocationServiceMap`, `Form`, `Permission`, `Location`, `Job`, `Database`, `Session`, `Bus`, `Shell`) and the members CodeNomad calls are unchanged. The changes are additive: `Job.cancelSession` (2.2.4), `Permission.allows` (2.2.5; never asks, used by `edit` to decide whether to quote file lines), and `Session.remove` now calls `jobs.cancelSession`. The Missions recurrence clock calls `Job.get/start/cancel`. Its Job metadata carries no `sessionID`, so deleting a session doesn't cancel it. - **Storage:** one migration, `20261030054400_job` (2.2.4, [#60491](anomalyco/opencode#60491)). It creates `job`, copies every `kv` row `job.background/%` into it and deletes those `kv` rows. 2.2.5 adds none. - CodeNomad reads only its own `kv` keys (Missions, pruning receipts and claims), never `job.background/*`. - Native migration 2.2.3 → 2.2.4 passes. Every 2.2.4/2.2.5 fixture DB inspected holds the `job` table, **0** `job.background/` rows and no leftover `schedule:leader` lease. - **`session_part`:** still **not written**. `history.ts`/`projector.ts` are unchanged from v2.2.3 to v2.2.5, the projector still only rewrites existing part rows, and every inspected 2.2.4/2.2.5 fixture DB has **0** rows. The forward risk from the 2.2.3 review stands (Recommendations). - **Behavior changes checked** (plain-language list in [What users will notice](#what-users-will-notice)): - **Image tool** (#1209 contract): `image-generate.ts` is unchanged from v2.2.3 to v2.2.5. 2.2.4 refactors restart recovery into one `restartMessage` but keeps `metadata.source: "experimental_image_generate"`, `jobID`, `state`, `path` and the `<image id state>` notice text. Design companion, activation, content and image settings pass natively on 2.2.4. - **Schedules** (2.2.4, [#60478](anomalyco/opencode#60478), then `da87c475a7`): the loop now runs in every server process, including `opencode serve`, and elects a leader through a `kv` lease `schedule:leader` (claimed at boot, released when there are no active schedules). Firing stays opt-in (`experimental.schedules === true`). CodeNomad uses neither OpenCode schedules nor that key; Missions recurrence uses its own storage and the Job service. - **Sessions:** deleting a session now cancels its running background jobs (shell, image, subagent) and drops their markers (2.2.4). That is the intended upstream cleanup; CodeNomad's session deletions (blank-session cleanup, temporary workspaces) get it automatically. - **Events:** `persistent-pty.updated` and `artifact.updated` reach CodeNomad's shared event relay. The relay routes generically (`instance-events.ts`), the 2.1.0 client's SSE reader doesn't validate types, and the UI handles neither type. The native relay suite passes on 2.2.4. - **Tools** (2.2.5): a model call to `bash` runs as `shell` with shell permissions, and the result carries a note ([#60687](anomalyco/opencode#60687)). CodeNomad already groups and renders `bash` and `shell` alike (`message-part-grouping.ts:48`, `tool-call/utils.ts`). `read`/`edit` failure texts are richer ([#60688](anomalyco/opencode#60688), [#60689](anomalyco/opencode#60689)). - **Compaction** (2.2.5): the prompt is capped at the context window when a catalog's input limit is larger ([#60660](anomalyco/opencode#60660)), and `compaction.buffer` may be `"N%"`. CodeNomad never reads or writes `compaction.buffer`. LiteLLM context errors now trigger compaction ([#60692](anomalyco/opencode#60692)). - **Reasoning replay** (2.2.5, [#60498](anomalyco/opencode#60498)): reasoning is no longer re-sent as plain text after a model switch, and interrupted or errored reasoning is dropped from the request. This changes provider requests only; stored transcripts and `Session.Message.*` are byte-identical. - **MCP** (2.2.5): stdio messages over 10 MiB no longer disconnect a server ([#60694](anomalyco/opencode#60694)), and an idle Location no longer restarts its MCP servers right after shutting down ([#60668](anomalyco/opencode#60668)). - **Providers:** a Console-supplied provider can now be overridden by user, project and remote config (2.2.4). The Claude Agent provider only replaces Claude Code's `/login` hint with `claude auth login` in its errors ([#60592](anomalyco/opencode#60592)); the guided setup ([#60603](anomalyco/opencode#60603)) is OpenCode Desktop only. The provider ID (`Provider.ID.claudeAgent`) and its use of Claude Code's own login are unchanged, so CodeNomad's `claude-agent` quota reader (#1214) is unaffected. - **Service lifecycle:** the CLI now adds `127.0.0.1,localhost,::1` to `NO_PROXY`/`no_proxy` for itself and its children ([#60441](anomalyco/opencode#60441)), so loopback traffic from daemon-hosted code, CodeNomad's plugins included, bypasses environment proxies. CodeNomad sets no proxy variables. - **Code Mode, Forms, permission prompts, media blobs:** no source changes from v2.2.3 to v2.2.5 (`packages/codemode` only bumps its version). The Code Mode run-to-completion regression passes on 2.2.4: `{ total: 2000, beforeAwait: 1000, count: 1000 }`. - **`@opencode/ai`:** one `ai` object for Promise and Effect ([#60516](anomalyco/opencode#60516); left out of the changelog by [#60631](anomalyco/opencode#60631)). CodeNomad doesn't import `@opencode/ai`. - Desktop/TUI-only changes (desktop terminals on persistent PTYs, base-branch and worktree shortcuts, quote sending, flat inputs, palette ordering) don't affect CodeNomad. One indirect effect follows. - **Worktree deletion:** OpenCode Desktop terminals now live as persistent PTYs that survive restarts ([#60346](anomalyco/opencode#60346)). CodeNomad's guard (`routes/worktrees.ts`, "Running persistent PTY blocks deletion: <id>") will therefore refuse more often for users who also open terminals in OpenCode Desktop. The behavior is correct, but the message names the PTY ID rather than the terminal's title. **Verdict:** - The **2.1.0 client/plugin pins can stay**. Bumping them would bring only artifacts, PTY renaming and a config type, none of which CodeNomad consumes. - The **2.2.3 minimum has no reason to move**. No consumed HTTP, plugin, private-service or storage contract broke in 2.2.4 or 2.2.5. - **Recommend 2.2.5.** It is admitted today through contract recognition. Recommending it is CodeNomad-side bookkeeping: `RECOMMENDED_OPENCODE_VERSION` (`runtime-support.ts`), the reviewed-version check in `contractProfile` (`compatibility/runtime.ts`), the version texts in docs and help, and CI's qualification target. This is subject to the full native suite on the exact 2.2.5 CLI, run separately. - **No CodeNomad bug found** in 2.2.3 → 2.2.5. Risks: the `session_part` forward risk (unchanged) and the more frequent worktree-deletion refusal (UX). **Not verified here:** - the full native suite on 2.2.5: only a host-module probe ran natively on 2.2.5, and everything else for 2.2.5 is source and OpenAPI review (Appendix I); - the 2.2.4 recurring Missions journey (`test-recurring-simple-native` ABCDGELFWQN), which was interrupted by a server restart; - the `kv` → `job` migration with a real pending background marker: only the empty case ran, and upstream `job.test.ts` covers the copy; - whether a `bash` call is stored as `bash` or `shell` (source suggests the model's name is kept; CodeNomad renders both); - artifacts, Claude Agent and MCP stdio framing at runtime (source only); - macOS/Linux. --- ## What users will notice **OpenCode 2.2.4** - **Persistent desktop terminals:** OpenCode Desktop terminals belong to the session, keep their scrollback across server restarts, stay in sync with the TUI and other windows, and can be renamed. Renamed terminals keep their names. - **Artifacts (opt-in):** with `"experimental": { "artifacts": true }`, agents can publish, update and serve HTML or static artifacts (`artifact_create`, `artifact_read`, `artifact_update`, `artifact_delete`) through a built-in local provider, which stores files under OpenCode's data folder, or a plugin-provided one. - **Schedules:** enabled schedules also fire under `opencode serve`, and only once when several OpenCode processes share a database. They stay off unless `experimental.schedules` is `true`. - **Claude Agent:** OpenCode Desktop guides installation, `claude auth login` and connection checking. Error messages now say `claude auth login` instead of `/login`. - **Config:** user, project and remote configuration can override Console-supplied provider names, limits, variants, settings and headers. - **Networking:** requests to the local OpenCode service no longer go through `HTTP(S)_PROXY`. - **Sessions:** deleting a session also stops its running background work, such as shells, image generations and subagents. - **Desktop/TUI:** base-branch shortcut, simpler location popover, faster Markdown streaming and various fixes. **OpenCode 2.2.5** - **Compaction:** `"compaction": { "buffer": "20%" }` keeps a share of each model's context free; token counts still work. Compaction no longer overfills models whose catalog input limit exceeds their context window, and LiteLLM context errors trigger compaction instead of stopping. - **Tools:** a model calling `bash` gets `shell` (with shell permissions) instead of an error. `read` names the missing folder, and `edit` shows the closest lines when `oldString` isn't found. - **Reasoning:** switching models or resuming an interrupted reply no longer resends reasoning as plain text, which some providers rejected. - **MCP:** large stdio messages (over 10 MiB) no longer disconnect a server, and idle locations don't immediately restart their MCP servers. - **Desktop/TUI:** worktree and base-branch shortcuts, a shell-mode toggle, sending a single quote from its popup, and MCP servers removed from the connect menu. ## Follow-ups (outside this PR) - Hardening for `session_part` (independent of this upgrade; 2.2.5 still writes no rows): make the direct `session_message.data.content` readers and the pruning transaction fail safe when `session_part` has rows for the target messages. - Optional UX: when the worktree deletion guard finds a running persistent PTY, name it by `userTitle ?? title` (2.2.4+) and say it may be an OpenCode Desktop terminal. - Any future reader of `compaction.buffer` must accept `"N%"` as well as a token count. ## Isolated Windows qualification Base: dev `605820669`; Node 25.2.1 (local; CI runs the pinned 24.20.0); exact official CLI **2.2.5**; locked client/plugin **2.1.0**; `CODENOMAD_NATIVE_FIXTURE_PARENT` outside the user profile. - **Native suites on 2.2.5, all passing:** Code Mode jobs, plugin runtime, automation, session pruning (history, navigation, media blobs, proxy, session images), Location/worktrees/event relay, permission receipts, Git init/degraded, Missions (contract, family control, native, trajectory) and all 11 recurring journeys, sessions (environment, fork, aside, interrupt, lifecycle, blank cleanup, compaction isolation, prompt skills), providers (usage, accounts, account selection), service usage, web search settings, plugin packages, output limit, Promise-tool failure, subagent depth, temporary workspaces, Design companion/activation/content, image settings, history migrations from 2.1.2/2.2.3/2.2.4, shared install of the 2.2.3 minimum. - **Installer suites after the bump:** upgrade (installed `opencode v2.2.5`) and legacy replacement (target `2.2.5`) install the new recommendation and pass. - **Builds and checks after the change:** server build, native plugin builds and root typecheck clean; server **2,840 passed, 9 skipped, 1 cancelled** (`index.test.ts` shutdown test timed out under full-suite load; **4/4** alone); runtime-support/transport/update **35/35**; browser `opencode-setup` **34/34**; ten-locale help **31/31**.
Switching models mid-session or resuming after an interrupted stream used to demote unsigned reasoning into normal assistant
text. In local session logs, that demotion causedreasoning_extractionblocks when switching back to Claude or retrying cut streams, HTTP 400s on Cohere v2 whentextlanded next totool_calls, and reasoning leaks into flattened compaction summaries.Chat Completions (
openai-chat.ts) already forwards unsigned reasoning inreasoning_contentacross models, and open-weight/messagesproviders withrequireSignature: falsealready forward{ type: "thinking", signature: "" }. The bug was the fallback totype: "text"in the four places where a signature or provider metadata was required or a turn had errored:toLLMMessages (packages/core): sameModel && !error && !interrupted -> keep reasoning + providerMetadata !sameModel -> keep unsigned { type: "reasoning", text } - error || reasoningInterrupted -> lower reasoning to { type: "text", text } + error || reasoningInterrupted -> drop reasoning anthropic-messages (packages/ai): !requireThinkingSignature -> send { type: "thinking", thinking, signature: "" } - requireThinkingSignature -> demote unsigned reasoning to text + requireThinkingSignature -> drop unsigned reasoning bedrock-converse (packages/ai): - missing signature -> demote unsigned reasoning to text + missing signature -> drop unsigned reasoning mistral-chat (packages/ai): - missing mistral.thinking metadata -> append reasoning to assistant text + missing mistral.thinking metadata -> drop unsigned reasoning (and compute prefix: true from emitted messages) gemini & google-interactions (packages/ai): - model-ID regex gate on gemini.ts only + inject "skip_thought_signature_validator" on unsigned tool calls across both protocolsI also added
requireSignature: falseonproviders/minimax.ts/messages(plusqwen*andminimax-*inanthropic-messages.tsfor gateway routes) and removedrequireSignature: falsefromproviders/meta.ts/messages, since live trials showedMiniMax-M3recalls unsignedsignature: ""thinking (and returns empty text if thinking is dropped in a tool loop) while Meta Muse usesredacted_thinkingdataand ignoressignature: "".Out of scope:
step.tsstill marks a finished model stream asSTEP_INTERRUPTEDwhen local tool execution is cancelled or declined, which strips valid stream metadata on the next turn.reasoningTextdrop unsigned reasoning rather than replaying it until model-level capability metadata distinguishes reasoning vs non-reasoning Converse targets (nova-pro400s onreasoningContent).Verification
bun run check(35/35 workspace packages passed)bun testandbun run typecheck:tscinpackages/ai(1990 passed)bun run script/test.ts test/session-runner-message.test.ts test/session-runner.test.ts test/session-compaction.test.tsandbun run typecheck:tscinpackages/core(306 passed)generateContent/interactionsmodels, 15 Mistral models, and 7/messagesendpointsVisual
Added by the opencode bot (AI-generated).
Related: #38620, #55096
Public API: adds optional
requireSignatureto model/providercompatibilityin @opencode/schema and OpenAPI (backwards compatible; unset now defaults to false for model IDs outside the signed families)