Skip to content

fix(openai): emit incremental tool-call events during streaming - #7

Open
xisongtang wants to merge 1 commit into
igorls:mainfrom
xisongtang:fix/openai-incremental-tool-call-streaming
Open

fix(openai): emit incremental tool-call events during streaming#7
xisongtang wants to merge 1 commit into
igorls:mainfrom
xisongtang:fix/openai-incremental-tool-call-streaming

Conversation

@xisongtang

Copy link
Copy Markdown

Summary

The OpenAI-compatible provider only emitted a tool_call event once the stream finished (when finish_reason was tool_calls/stop). The Google, Anthropic, and Ollama providers all surface tool-call progress as it arrives during streaming, so OpenAI was the inconsistent outlier.

This emits the accumulated tool calls on every chunk that carries tool-call deltas, so consumers can render live tool-argument progress (e.g. streaming UI previews of structured tool output) instead of waiting for the entire response to finish.

Before / After

 // Accumulate streamed tool calls
 if (delta.tool_calls) {
     for (const tc of delta.tool_calls) { /* accumulate into toolCallAccum */ }
+    // Incremental tool-call progress: emit accumulated arguments as they
+    // stream in, matching the Google/Anthropic/Ollama providers.
+    if (toolCallAccum.size > 0) {
+        const calls = Array.from(toolCallAccum.values())
+            .map((tc) => this.normalizeToolCall(tc));
+        yield { type: 'tool_call', calls };
+    }
     if (loopGuard.detection) break;
 }

The existing finish-gated emit (Emit tool calls when stream finishes) is unchanged, so consumers relying on a single final event are unaffected.

Why

Discovered while building live streaming previews: the OpenAI provider gave no intermediate tool_call events, so partial tool arguments (e.g. a large JSON payload being generated) could not be previewed until completion — unlike the other three providers.

Notes

  • dist/ is gitignored; only the source change is included.
  • I couldn't run bun test in my environment (bun not installed locally). Please run the suite; happy to add a unit test (fetch/SSE mock) if you'd like coverage for this path.

The OpenAI-compatible provider only yielded a tool_call event once the
stream finished (on finish_reason), unlike the Google, Anthropic and
Ollama providers which surface tool-call progress as it arrives. Emit the
accumulated tool calls on every chunk that carries tool-call deltas so
consumers can render live tool-argument progress (e.g. streaming previews).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant