Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/fix-structured-output-timestamps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@tanstack/ai': patch
'@tanstack/ai-bedrock': patch
'@tanstack/ai-byteplus': patch
'@tanstack/ai-openrouter': patch
'@tanstack/openai-base': patch
---

Timestamp native and fallback structured-output events when they are emitted so their lifecycle remains chronologically ordered.
3 changes: 2 additions & 1 deletion docs/reference/interfaces/TextAdapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,8 @@ activity layer synthesizes a stream around the non-streaming
Implementations must emit standard AG-UI lifecycle events (RUN_STARTED,
TEXT_MESSAGE_*, RUN_FINISHED) carrying raw JSON text deltas, plus a final
`CUSTOM` event named `structured-output.complete` whose `value` is
`{ object, raw, reasoning? }`.
`{ object, raw, reasoning? }`. Events must be timestamped when emitted so
their timestamps follow stream order.

#### Parameters

Expand Down
20 changes: 11 additions & 9 deletions docs/structured-outputs/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Streaming Structured Output UIs
id: structured-outputs-streaming
order: 3
description: "Build a UI that fills in field by field as the model streams structured JSON. chat({ outputSchema, stream: true }) on the server, useChat({ outputSchema }) on the client — progressive partial state plus a validated terminal object."
description: "Build a UI that fills in field by field as the model streams structured JSON. chat({ outputSchema, stream: true }) on the server, useChat({ outputSchema }) on the client — progressive partial state plus a typed terminal object."
keywords:
- tanstack ai
- structured outputs
Expand All @@ -16,7 +16,7 @@ keywords:

You have an existing chat-style endpoint and you want the structured response to populate a UI _while_ the model is generating — a form filling in field by field, a card whose ingredients list grows as JSON streams in, a typewriter preview of a JSON-typed report. Blocking on `await chat({ outputSchema })` would leave the UI dark until the whole object is ready; this guide is the alternative.

By the end you'll have a server endpoint streaming structured JSON as Server-Sent Events, and a client that reads a typed `partial` (progressive object) and `final` (validated terminal object) from `useChat`.
By the end you'll have a server endpoint streaming structured JSON as Server-Sent Events, and a client that reads a typed `partial` (progressive object) and `final` (completed terminal object) from `useChat`.

> **Note:** This is the streaming counterpart of [One-Shot Extraction](./one-shot). If you don't need progressive UI updates, the one-shot path is simpler. If you want users to iterate on the object across multiple turns and keep history, see [Multi-Turn Chat](./multi-turn).

Expand Down Expand Up @@ -48,11 +48,11 @@ export async function POST(request: Request) {
}
```

That's the entire server side. `chat({ outputSchema, stream: true })` returns a `StructuredOutputStream<InferSchemaType<typeof PersonSchema>>` — an `AsyncIterable` of standard streaming events plus a terminal `structured-output.complete` event carrying the validated object. `toServerSentEventsResponse` knows what to do with it.
That's the entire server side. `chat({ outputSchema, stream: true })` returns a `StructuredOutputStream<InferSchemaType<typeof PersonSchema>>` — an `AsyncIterable` of standard streaming events plus a terminal `structured-output.complete` event carrying the completed object. `toServerSentEventsResponse` knows what to do with it.

## Client with `useChat`

Pass the same schema to `useChat`. The hook gives you a progressively-parsed `partial` and a validated `final`:
Pass the same schema to `useChat`. The hook gives you a progressively-parsed `partial` and a typed `final`:

```tsx
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
Expand Down Expand Up @@ -82,7 +82,7 @@ function PersonExtractor() {
<p>Name: {partial.name ?? "…"}</p>
<p>Age: {partial.age ?? "…"}</p>
<p>Email: {partial.email ?? "…"}</p>
{final && <pre>Validated: {JSON.stringify(final, null, 2)}</pre>}
{final && <pre>Completed: {JSON.stringify(final, null, 2)}</pre>}
</form>
);
}
Expand All @@ -91,8 +91,8 @@ function PersonExtractor() {
What the hook does for you:

- **`partial`** is `DeepPartial<z.infer<typeof PersonSchema>>` — every property optional, every nested array element optional. Updated from `TEXT_MESSAGE_CONTENT` deltas via the runtime's partial-JSON parser. The hook derives it from the latest assistant message's `structured-output` part (see [Multi-Turn Chat](./multi-turn) for why that distinction matters), so it reads `{}` between `sendMessage()` and the first chunk without any extra reset state.
- **`final`** is `z.infer<typeof PersonSchema> | null` — the validated terminal payload from the `structured-output.complete` event. `null` until the run completes successfully.
- **`outputSchema`** is used purely for client-side TypeScript inference. Validation still runs on the server against the schema you pass to `chat({ outputSchema })` on the server route — the client doesn't re-validate.
- **`final`** is `z.infer<typeof PersonSchema> | null` — the completed terminal payload from the `structured-output.complete` event. `null` until the run completes successfully.
- **`outputSchema`** is used purely for client-side TypeScript inference. The streaming path does not run Standard Schema validation; validate the completed object in the consumer when required.
- The same shape works for **non-streaming adapters too**. If an adapter (Anthropic, Gemini, Ollama) returns a single `structured-output.complete` event with no incremental deltas, `partial` stays `{}` and `final` populates when the event arrives. Same consumer code. Claude Code and Codex emit `structured-output.complete` from the harness event. OpenCode, Grok Build, and `acpCompatible` parse the last assistant text at the end. In both cases `partial` stays empty until `final` is set. See [Harness Agents](./harnesses).

`outputSchema` is optional: omit it and `useChat` returns its standard shape without `partial` / `final`.
Expand Down Expand Up @@ -163,7 +163,7 @@ The `structured-output` part fields:
type: "CUSTOM",
name: "structured-output.complete",
value: {
object: T; // validated, parsed, typed
object: T; // completed, parsed, typed
raw: string; // full accumulated JSON text
reasoning?: string; // present only for thinking/reasoning models
},
Expand All @@ -183,6 +183,8 @@ Streaming structured output works with **every adapter**, but only some support
| `@tanstack/ai-openrouter` | Native single-request stream (`response_format: json_schema`) |
| `@tanstack/ai-grok` | Native single-request stream (Chat Completions, `response_format: json_schema`) |
| `@tanstack/ai-groq` | Native single-request stream (Chat Completions, `response_format: json_schema`) |
| `@tanstack/ai-bedrock` | Native stream through Converse or an OpenAI-compatible API |
| `@tanstack/ai-byteplus` | Native single-request stream on supported models; unsupported models emit `RUN_ERROR` |
| Other adapters (anthropic, gemini, ollama, …) | Fallback: runs non-streaming `structuredOutput` and emits the final object as one `structured-output.complete` event |

The fallback path keeps the consumer code identical across providers — you always read the final object off `structured-output.complete` — but you won't see incremental deltas unless the adapter implements `structuredOutputStream` natively.
Expand Down Expand Up @@ -211,7 +213,7 @@ const stream = chat({

for await (const chunk of stream) {
if (chunk.type === "CUSTOM" && chunk.name === "structured-output.complete") {
// Validated and typed against PersonSchema.
// Typed against PersonSchema. Validate here when required.
console.log(chunk.value.object.name);
console.log(chunk.value.object.age);
}
Expand Down
23 changes: 11 additions & 12 deletions packages/ai-bedrock/src/adapters/converse-text.ts
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,6 @@ export class BedrockConverseTextAdapter<
options: StructuredOutputOptions<TProviderOptions>,
): AsyncIterable<StreamChunk> {
const { chatOptions, outputSchema } = options
const timestamp = Date.now()
const runId = this.generateId()
const threadId = chatOptions.threadId ?? this.generateId()
const messageId = this.generateId()
Expand Down Expand Up @@ -326,7 +325,7 @@ export class BedrockConverseTextAdapter<
runId,
threadId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
parentRunId: chatOptions.parentRunId,
}
}
Expand All @@ -347,7 +346,7 @@ export class BedrockConverseTextAdapter<
messageId,
role: 'assistant',
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
}
}
accumulatedRaw += fragment
Expand All @@ -357,7 +356,7 @@ export class BedrockConverseTextAdapter<
delta: fragment,
content: accumulatedRaw,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
}
}
continue
Expand Down Expand Up @@ -385,7 +384,7 @@ export class BedrockConverseTextAdapter<
runId,
threadId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
parentRunId: chatOptions.parentRunId,
}
}
Expand All @@ -395,7 +394,7 @@ export class BedrockConverseTextAdapter<
type: EventType.TEXT_MESSAGE_END,
messageId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
}
}

Expand All @@ -404,7 +403,7 @@ export class BedrockConverseTextAdapter<
type: EventType.RUN_ERROR,
runId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
message: `${this.name}.structuredOutputStream: response contained no content`,
code: 'empty-response',
error: {
Expand All @@ -423,7 +422,7 @@ export class BedrockConverseTextAdapter<
type: EventType.RUN_ERROR,
runId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
message: `Failed to parse structured output as JSON. Content: ${accumulatedRaw.slice(0, 200)}${accumulatedRaw.length > 200 ? '...' : ''}`,
code: 'parse-error',
error: {
Expand All @@ -442,15 +441,15 @@ export class BedrockConverseTextAdapter<
raw: accumulatedRaw,
},
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
}

yield {
type: EventType.RUN_FINISHED,
runId,
threadId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
finishReason,
}
} catch (error: unknown) {
Expand All @@ -461,7 +460,7 @@ export class BedrockConverseTextAdapter<
runId,
threadId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
parentRunId: chatOptions.parentRunId,
}
}
Expand All @@ -477,7 +476,7 @@ export class BedrockConverseTextAdapter<
type: EventType.RUN_ERROR,
runId,
model: chatOptions.model,
timestamp,
timestamp: Date.now(),
message: errorPayload.message,
...(errorPayload.code !== undefined && { code: errorPayload.code }),
error: {
Expand Down
5 changes: 2 additions & 3 deletions packages/ai-byteplus/src/adapters/text.ts
Original file line number Diff line number Diff line change
Expand Up @@ -334,21 +334,20 @@ export class BytePlusTextAdapter<
// Mirror the base's contract: failures inside structuredOutputStream
// surface as a RUN_STARTED → RUN_ERROR pair rather than a throw, so
// consumers keep a single error-handling path.
const timestamp = Date.now()
const runId = generateId(this.name)
yield {
type: EventType.RUN_STARTED,
runId,
threadId: options.chatOptions.threadId ?? generateId(this.name),
model: options.chatOptions.model,
timestamp,
timestamp: Date.now(),
parentRunId: options.chatOptions.parentRunId,
}
yield {
type: EventType.RUN_ERROR,
runId,
model: options.chatOptions.model,
timestamp,
timestamp: Date.now(),
message: unsupported,
code: 'unsupported-structured-output',
error: { message: unsupported, code: 'unsupported-structured-output' },
Expand Down
Loading
Loading