feat: streaming support for coding agents (Claude Code & Codex)#586
Open
Davidnet wants to merge 9 commits into
Open
feat: streaming support for coding agents (Claude Code & Codex)#586Davidnet wants to merge 9 commits into
Davidnet wants to merge 9 commits into
Conversation
…g tests The rebase onto main brought in the dashboard metrics added in #547, which are recorded after the response is written. The SSE streaming path returns early, so streamed requests (Claude Code, Codex) were invisible to the dashboard; record them there too, with the request-masking time as the proxy-overhead latency (response restoration is interleaved with the upstream stream). Also adds the test coverage the streaming feature landed without: the SSE engine end-to-end over a wire (chunked framing, header rewrite, PII restore), both provider codecs (placeholder split across deltas, tool-call argument restore, per-channel carry buffers, passthrough events), stream sniffing, Codex host routing, and chatgpt.com interception gating.
shepherd.js is a frontend dependency already declared in src/frontend/package.json; the root entry was added accidentally. Regenerating the lockfile also syncs the recorded src/frontend version with main (1.4.0 -> 1.4.1).
Contributor
|
Warning This PR touches 3+ distinct areas of the codebase. Consider splitting into smaller, focused PRs — each covering a single semantic type. Categories found: docs:
code:
chore:
test:
|
Collaborator
Member
Author
Codex CLI startup and completions both broke when routed through the MITM proxy. Five fixes, verified end-to-end against a live Codex run: - Gate chatgpt.com interception by path: only /backend-api/codex/responses is masked; the streaming MCP transport, model list, and telemetry pass through verbatim via a new in-tunnel passthrough that streams incrementally and preserves framing across keep-alive requests. - Pin relayed responses to HTTP/1.1: upstream legs negotiate HTTP/2, and relaying resp.Proto verbatim produced "HTTP/2.0" status lines that strict clients (Codex's hyper) reject as malformed. - Create the MITM loop's bufio.Reader once per connection so buffered bytes of a pipelined next request are not dropped between iterations. - Sniff SSE when Content-Type is absent: the Codex backend streams SSE with no Content-Type header, which sent responses down the buffered path and skipped PII restoration. - Restore PII in nested .done/.completed Responses-API payloads (content_part.done, output_item.done, response.completed) — Codex renders its final message from these, not from the flat deltas. - Reject WebSocket upgrades on masked endpoints (fail closed): PII in switched protocols cannot be masked. Codex is steered to SSE via a provider config with supports_websockets=false, documented in docs/09-coding-agents.md.
The response-restoration logic replaced each masked (dummy) value with its
original via a sequence of strings.ReplaceAll calls. When the fake-value
generator produced a dummy that coincides with a real original from another
mapping (e.g. "Priya"->"Nicole" alongside "Claude"->"Priya"), the passes
chained: restoring "Nicole"->"Priya" and then re-replacing that "Priya"->
"Claude", so the client received the wrong PII ("Hi Claude" where the model
wrote "Hi Nicole" for Priya). The bug was also map-order dependent.
Replace both restore paths (streaming codecs and the buffered
ResponseProcessor) with a shared single-pass strings.Replacer built by
processor.BuildRestorer, keys ordered longest-first. strings.Replacer scans
the input once and never re-substitutes its own output, so restored text is
never re-replaced. The streaming restoreCore builds the replacer once per
stream instead of iterating the map per delta.
Adds regression tests for the exact collision and longest-match precedence.
Member
Author
Collaborator
|
@Davidnet Great work, but a huge PR. Wanna review it together on Monday? |
Member
Author
Yes, It was also a quite a bit of new things, and for sure |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
proxy/streaming.go) with provider-specific codecs for Anthropic (codec_anthropic.go) and OpenAI (codec_openai.go)docs/09-coding-agents.mdguide and README updatesMain work was provided in #532