feat(responses): translate legacy <think> blocks into native reasoning field

[weselben]

TL;DR

Response-path translation of legacy <think>...</think> (and configured equivalents) into each API surface's native reasoning field. The canonical chat response hook covers OpenAI chat completions and Anthropic messages because both dialect converters already surface ExtraFields["reasoning_content"] as their native reasoning payload. OpenAI responses cascades via the shared OpenAIResponsesStreamConverter.

What

A model that emits <think>reasoning</think>answer returns, after translation:

• OpenAI chat completions — message.reasoning_content carries reasoning, message.content carries answer.
• Anthropic messages — a thinking content block whose thinking text is reasoning, plus a text content block whose text is answer.
• OpenAI responses API — a reasoning output item whose text is reasoning, plus a normal message output item whose text is answer (via chat translation).

Request bodies are never modified. Translation is lossless on the wire: no model-visible character is dropped. A <think> block whose closing tag has not yet arrived is buffered and waits; a never-closed block is forwarded verbatim with no rewrite applied, so a chunk-boundary cut can never strand user content as reasoning.

Why

Modern models (Kimi K3, Claude Fable 5, o-series, …) and Kimi Code-class harnesses expect reasoning in the structured field, not as XML inside the message text. A client that reads reasoning_content gets nothing from a model that emits <think> inline; a client that reads the text alone sees the XML tags as noise. The gateway translating on the response path is the only place that can reconcile both sides without touching the model or the client.

How

Files to review (19, +2164 / -1):

File	Why
internal/thinkextract/thinkextract.go (start here)	Tag scanner, multi-pair defaults, buffer cap, single-pass Extract
internal/thinkextract/stream.go	SSE transformer: per-choice state, [DONE] flush, unknown-field preservation
internal/thinkextract/surface.go	Surface context tag for per-surface gating
internal/thinkextract/chat.go	Non-stream ChatResponse / ResponseMessage rewrite via MergeUnknownJSONFields
internal/thinkextract/*_test.go	Table-driven tests, package coverage 90.4%
config/thinkextract.go	ThinkExtractConfig: env-gated global + per-surface opt-outs, tag pairs, buffer cap
config/thinkextract_test.go	Config semantics tests
internal/gateway/inference_orchestrator.go	Single canonical hook on the chat provider call
internal/gateway/inference_execute.go	Gate check before transform
internal/server/translated_inference_service.go	Chat surface tag on dispatch
internal/server/messages_handler.go	Messages surface tag on dispatch
internal/server/handlers.go	Pass-through of thinkExtractOptions
internal/server/http.go	ServerConfig wiring
internal/app/app.go	Config -> options conversion
config/config.go	ThinkExtractConfig registration + defaults

The single canonical-chat hook (line 402 and 432 of inference_execute.go) covers both chat and messages surfaces because both flow through chatCompletionProviderCall / streamChatCompletionProviderCall, and each dispatch function tags its request context with its Surface identifier so the gate knows which config flag to check.

Reviewer notes

• Default on. The translation is lossless-by-construction; operators who hit a regression can opt out globally (THINK_EXTRACT_ENABLED=false) or per-surface (THINK_EXTRACT_CHAT_ENABLED=false, THINK_EXTRACT_MESSAGES_ENABLED=false). The global switch is authoritative: a per-surface true cannot resurrect a globally-off feature.
• Multi-pair defaults. Ships with 7 evidence-backed tag pairs (T9 research: vLLM, SGLang, Open WebUI sources): <think>, <thinking>, <reasoning>, <reason>, <thought>, <|begin_of_thought|>, Kimi K2 ◁think▷ brackets, Mistral Magistral [THINK], gpt-oss harmony markers. <reflection> and the <|think|>/<|reasoning|> pipe forms are refuted by primary sources and excluded. Granite's plain-English delimiters and ERNIE's <response> answer-wrapper are excluded by design.
• Custom tag pairs. THINK_EXTRACT_TAG_PAIRS (env) or think_extract.tag_pairs (config) overrides the default list with a comma-separated <open>...</close> format. Malformed entries are skipped.
• Streaming chunk-boundary safety. Per-choice state holds partial tags in a bounded buffer until the closing tag arrives. An unclosed block at stream end is re-emitted as ordinary content, never dropped.
• Upstream reasoning wins. When the upstream already sends reasoning_content on the delta or message, the translator leaves it alone — no stomping of structured data with extracted text.

Deliberately out of scope

• Native Responses API path (responsesProviderCall) — when the resolved provider speaks Responses natively rather than via chat, the Responses SSE conversion needs its own stream transformer. Ticket #42 — native Responses stream transformer.
• Anthropic thinking-block signatures on /v1/messages — the Anthropic messages dialect converter already emits thinking blocks from ExtraFields["reasoning_content"] today (pre-existing behavior). The signature policy is tracked in #46 — /v1/messages thinking-block policy.
• Responses reasoning.summary population — the source is raw text only, not a model-generated summary; a summary would require an LLM call.
• Fast-path chat passthrough (tryFastPathStreamingChatPassthrough) and the native Anthropic messages passthrough (dispatchMessagesNative) deliberately bypass translation today and are documented as excluded.

Tests

go build ./... — ok
go test ./internal/thinkextract/ -v — all pass
go test ./config/ -v — all pass
go test ./... — all packages pass
go test ./internal/thinkextract/ -cover — 90.4%

Coverage gaps: pipe-write error branches, mustEncode unreachable fallback, transformMessage nil-guard (all defensive code with no normal-path effect).

AI disclosure

Generated by an AI assistant (kimi-code CLI) under human direction. Reviewed and committed by the repository owner.

Wayfinder

Map: #34. Resolved tickets: #35, #36, #37, #39, #43, #44, #45. Open: #40 (hook-point pushback, awaiting user), #42 (native Responses transformer), #46 (messages thinking-block policy).

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}