Skip to content

feat: streaming support for coding agents (Claude Code & Codex)#586

Open
Davidnet wants to merge 9 commits into
mainfrom
feat/streaming-feature
Open

feat: streaming support for coding agents (Claude Code & Codex)#586
Davidnet wants to merge 9 commits into
mainfrom
feat/streaming-feature

Conversation

@Davidnet

@Davidnet Davidnet commented Jul 2, 2026

Copy link
Copy Markdown
Member

Summary

  • Add SSE streaming support to the proxy so coding agents (Claude Code, Codex) work through it
  • New streaming pipeline (proxy/streaming.go) with provider-specific codecs for Anthropic (codec_anthropic.go) and OpenAI (codec_openai.go)
  • Record dashboard metrics for streamed responses
  • Docs: new docs/09-coding-agents.md guide and README updates

Main work was provided in #532

hanneshapke and others added 7 commits July 2, 2026 16:01
…g tests

The rebase onto main brought in the dashboard metrics added in #547, which
are recorded after the response is written. The SSE streaming path returns
early, so streamed requests (Claude Code, Codex) were invisible to the
dashboard; record them there too, with the request-masking time as the
proxy-overhead latency (response restoration is interleaved with the
upstream stream).

Also adds the test coverage the streaming feature landed without: the SSE
engine end-to-end over a wire (chunked framing, header rewrite, PII
restore), both provider codecs (placeholder split across deltas, tool-call
argument restore, per-channel carry buffers, passthrough events), stream
sniffing, Codex host routing, and chatgpt.com interception gating.
shepherd.js is a frontend dependency already declared in
src/frontend/package.json; the root entry was added accidentally.
Regenerating the lockfile also syncs the recorded src/frontend version
with main (1.4.0 -> 1.4.1).
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Warning

This PR touches 3+ distinct areas of the codebase.

Consider splitting into smaller, focused PRs — each covering a single semantic type.
This makes reviews easier and keeps the git history clean.

Categories found:

docs:

  • README.md
  • docs/09-coding-agents.md
  • docs/README.md

code:

  • src/backend/config/config.go
  • src/backend/processor/response.go
  • src/backend/providers/anthropic.go
  • src/backend/providers/openai.go
  • src/backend/providers/provider.go
  • src/backend/proxy/codec_anthropic.go
  • src/backend/proxy/codec_openai.go
  • src/backend/proxy/cors.go
  • src/backend/proxy/handler.go
  • src/backend/proxy/passthrough.go
  • src/backend/proxy/router.go
  • src/backend/proxy/streaming.go
  • src/backend/proxy/transparent.go
  • src/backend/proxy/transparent_factory.go

chore:

  • package-lock.json

test:

  • src/backend/config/config_test.go
  • src/backend/processor/response_test.go
  • src/backend/providers/providers_test.go
  • src/backend/proxy/passthrough_test.go
  • src/backend/proxy/router_test.go
  • src/backend/proxy/streaming_test.go

@hanneshapke

Copy link
Copy Markdown
Collaborator

@Davidnet Is this PR in addition to #532 ?

@Davidnet

Davidnet commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

@Davidnet Is this PR in addition to #532 ?

Yes it was a rebase, and a modifications that I brought from my tests

Davidnet added 2 commits July 3, 2026 14:44
Codex CLI startup and completions both broke when routed through the
MITM proxy. Five fixes, verified end-to-end against a live Codex run:

- Gate chatgpt.com interception by path: only /backend-api/codex/responses
  is masked; the streaming MCP transport, model list, and telemetry pass
  through verbatim via a new in-tunnel passthrough that streams
  incrementally and preserves framing across keep-alive requests.
- Pin relayed responses to HTTP/1.1: upstream legs negotiate HTTP/2, and
  relaying resp.Proto verbatim produced "HTTP/2.0" status lines that
  strict clients (Codex's hyper) reject as malformed.
- Create the MITM loop's bufio.Reader once per connection so buffered
  bytes of a pipelined next request are not dropped between iterations.
- Sniff SSE when Content-Type is absent: the Codex backend streams SSE
  with no Content-Type header, which sent responses down the buffered
  path and skipped PII restoration.
- Restore PII in nested .done/.completed Responses-API payloads
  (content_part.done, output_item.done, response.completed) — Codex
  renders its final message from these, not from the flat deltas.
- Reject WebSocket upgrades on masked endpoints (fail closed): PII in
  switched protocols cannot be masked. Codex is steered to SSE via a
  provider config with supports_websockets=false, documented in
  docs/09-coding-agents.md.
The response-restoration logic replaced each masked (dummy) value with its
original via a sequence of strings.ReplaceAll calls. When the fake-value
generator produced a dummy that coincides with a real original from another
mapping (e.g. "Priya"->"Nicole" alongside "Claude"->"Priya"), the passes
chained: restoring "Nicole"->"Priya" and then re-replacing that "Priya"->
"Claude", so the client received the wrong PII ("Hi Claude" where the model
wrote "Hi Nicole" for Priya). The bug was also map-order dependent.

Replace both restore paths (streaming codecs and the buffered
ResponseProcessor) with a shared single-pass strings.Replacer built by
processor.BuildRestorer, keys ordered longest-first. strings.Replacer scans
the input once and never re-substitutes its own output, so restored text is
never re-replaced. The streaming restoreCore builds the replacer once per
stream instead of iterating the map per delta.

Adds regression tests for the exact collision and longest-match precedence.
@Davidnet

Davidnet commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Codex sample:

codex exec \
      --skip-git-repo-check \
      -c 'model_provider="kiji"' \
      -c 'model_providers.kiji={ name = "Kiji", base_url = "https://chatgpt.com/backend-api/codex", wire_api = "responses", requires_openai_auth = true, supports_websockets = false }' \
"Draft a short reply to David Nettleton at dnon@gmail.com confirming the meeting."
OpenAI Codex v0.142.5
--------
workdir: /Users/davidnet/sample
model: gpt-5.4
provider: kiji
approval: never
sandbox: read-only
reasoning effort: medium
reasoning summaries: none
session id: 019f295d-39d1-7971-a168-0b6dd53b08d6
--------
user
Draft a short reply to David Nettleton at dnon@gmail.com confirming the meeting.
codex
Subject: Re: Meeting Confirmation

Hi David,

Confirming the meeting works for me. Looking forward to it.

Best,
Jordan
tokens used
8,434
image

Claude code

set -x HTTP_PROXY http://127.0.0.1:8081
set -x HTTPS_PROXY http://127.0.0.1:8081
 set -x NODE_EXTRA_CA_CERTS "$HOME/Library/Application Support/Kiji Privacy Proxy/certs/ca.crt"
$ claude -p --model haiku "Draft a two-sentence email to Priya Raghavan (priya.raghavan@acme.io) confirming Tuesday's sync. Sign it from me."
Here's a draft email:

---

**To:** priya.raghavan@acme.io
**Subject:** Confirming Tuesday's Sync

Hi Priya,

I wanted to confirm our sync scheduled for Tuesday. Please let me know if you need to reschedule or if there's anything you'd like to go over beforehand.

Best regards,
Yuki Smith
image

@Davidnet Davidnet requested a review from hanneshapke July 3, 2026 19:26
@hanneshapke

Copy link
Copy Markdown
Collaborator

@Davidnet Great work, but a huge PR. Wanna review it together on Monday?

@Davidnet

Davidnet commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

@Davidnet Great work, but a huge PR. Wanna review it together on Monday?

Yes, It was also a quite a bit of new things, and for sure

@Davidnet Davidnet marked this pull request as ready for review July 3, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants