Skip to content

fix(agent): stop duplicate streaming with server-held keys - #99

Open
kannan983 wants to merge 3 commits into
CopilotKit:mainfrom
kannan983:fix/streaming-activity-and-duplicate-args
Open

kannan983 wants to merge 3 commits into
CopilotKit:mainfrom
kannan983:fix/streaming-activity-and-duplicate-args

Conversation

@kannan983

@kannan983 kannan983 commented Oct 10, 2026 •

Copy link
Copy Markdown

Summary

When the agent runs with server-held keys from apps/agent/.env, every interactive (generateSandboxedUi) answer is garbled or interrupted, whichever model is used. Text answers look fine. BYOK browser keys don't take this code path, which is probably why it went unnoticed.

Symptom

Widgets render with duplicated text and raw tag fragments, or the chat shows "Something interrupted the answer". The runtime logs:

[OpenGenerativeUI] JSON parse error in streaming args, resuming: Bad object

The raw AG-UI stream from the agent has two TOOL_CALL_START events for the same tool call, and every args delta appears twice, so the concatenated args aren't valid JSON:

{"initialHeight": 980{"initialHeight": 980, "placeholderMessages": [...], "placeholderMessages": [...]

Root cause

CredentialScopedModel delegates through the provider model's public astream(...) / invoke(...), after bind_tools wraps it in a RunnableBinding. Inside a LangGraph node, that nested call inherits the node's callbacks through var_child_runnable_config. So the provider model reports on_chat_model_stream events and the wrapper reports the same chunks again. Passing {"callbacks": []} doesn't help, because merge_configs adds a list onto an inherited callback manager instead of replacing it.

With browser keys, request_model returns a plain ChatOpenAI, which is called directly, so the duplication never happens.

Fix

CredentialScopedModel now unwraps the RunnableBinding (merging its bound kwargs, e.g. tools) and calls the provider's _generate / _agenerate / _stream / _astream directly. Only the wrapper's own run reports events. Models without streaming support fall back to a single generation chunk, as before.

Testing

  • New test_credential_scoped_model_streams_each_chunk_once: wraps the model in a RunnableLambda (as a LangGraph node does) with bound tools and collects on_chat_model_stream tokens. Without the fix it produces 'aa bb cc'; with the fix, 'a b c'.
  • uv run --directory apps/agent pytest: 112 passed
  • Raw agent stream: one TOOL_CALL_START per call, and the args parse as JSON with all six generateSandboxedUi fields.
  • End to end with server keys, through Docker, with anthropic/claude-sonnet-5.5 and deepseek/deepseek-v4.1-flash via an OpenAI-compatible gateway (LLM_MODEL routing from feat(agent): support OpenAI-compatible gateways via vendor/model names #100). Before the fix, "Make an interactive loan EMI calculator" was garbled or interrupted. After it, the widget streams and renders fully, its controls work, and a follow-up question about the widget's values is answered correctly.

Related observation (not changed here)

The runtime server also logs Failed to apply activity patch ...: a.applyPatch is not a function. Under Node, @ag-ui/client's import * as a from "fast-json-patch" resolves to the CommonJS entry, whose Object.assign(exports, ...) exports Node can't detect. I tested with and without a fix for this: widgets and follow-up turns behave the same, because the error is caught and the browser bundle resolves the ESM build. So it looks like log noise only, and I've left it out of this PR. I'm happy to open a separate issue if you'd like it tracked.

@kannan983 kannan983 changed the title fix: interrupted and garbled widgets with server-held keys fix(agent): stop duplicate streaming with server-held keys Oct 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant