Skip to content

feat(ai-sdk): stream tool outputs for async generator handlers - #1981

Merged
dinwwwh merged 1 commit into
middleapi:mainfrom
dinwwwh:claude/async-generator-check-bfd26e
Sep 4, 2026
Merged

feat(ai-sdk): stream tool outputs for async generator handlers#1981
dinwwwh merged 1 commit into
middleapi:mainfrom
dinwwwh:claude/async-generator-check-bfd26e

Conversation

@dinwwwh

@dinwwwh dinwwwh commented Sep 4, 2026

Copy link
Copy Markdown
Member

createToolFactory now streams tool outputs whenever the procedure handler is an async function*, not only when the output schema is built with asyncIteratorObject. Previously a schema-less async generator handler (or one paired with a type<AsyncIteratorObject<...>>() schema) returned the iterator object as the tool result instead of streaming preliminary results, even though the inferred tool output type was already the yield type.

The detection lives in @orpc/shared as isAsyncGeneratorFunction, so other packages can reuse it.

Fixes

  • Async generator handlers stream each event as a preliminary tool result with or without an asyncIteratorObject output schema. Runtime behavior now matches the inferred InferToolOutput type.
  • Output schemas remain the source of truth; the handler check is a fallback, so plain async handlers that happen to return an iterator are unchanged.

Testing

  • New unit tests for isAsyncGeneratorFunction (bound functions and async generator methods included, sync generators and plain async functions excluded).
  • New AI SDK tests: streaming without an output schema, streaming with a non-asyncIteratorObject schema, and no streaming for a non-generator handler returning an iterator. Type test covers the schema-less yield type inference.
  • Docs "Streaming Tool Outputs" section mentions the async function* path.

Add isAsyncGeneratorFunction to @orpc/shared and use it in createToolFactory
as a fallback when the output schema is not an asyncIteratorObject, so
async function* handlers stream preliminary results even without a schema.
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 30 untouched benchmarks


Comparing dinwwwh:claude/async-generator-check-bfd26e (537e5ea) with main (0cd296a)

Open in CodSpeed

@pkg-pr-new

pkg-pr-new Bot commented Sep 4, 2026

Copy link
Copy Markdown
More templates

@orpc/ai-sdk

npm i https://pkg.pr.new/@orpc/ai-sdk@1981

@orpc/arktype

npm i https://pkg.pr.new/@orpc/arktype@1981

@orpc/bun

npm i https://pkg.pr.new/@orpc/bun@1981

@orpc/client

npm i https://pkg.pr.new/@orpc/client@1981

@orpc/cloudflare

npm i https://pkg.pr.new/@orpc/cloudflare@1981

@orpc/contract

npm i https://pkg.pr.new/@orpc/contract@1981

@orpc/experimental-effect

npm i https://pkg.pr.new/@orpc/experimental-effect@1981

@orpc/evlog

npm i https://pkg.pr.new/@orpc/evlog@1981

@orpc/hibernation

npm i https://pkg.pr.new/@orpc/hibernation@1981

@orpc/json-schema

npm i https://pkg.pr.new/@orpc/json-schema@1981

@orpc/experimental-msw

npm i https://pkg.pr.new/@orpc/experimental-msw@1981

@orpc/nest

npm i https://pkg.pr.new/@orpc/nest@1981

@orpc/next

npm i https://pkg.pr.new/@orpc/next@1981

@orpc/node

npm i https://pkg.pr.new/@orpc/node@1981

@orpc/openapi

npm i https://pkg.pr.new/@orpc/openapi@1981

@orpc/opentelemetry

npm i https://pkg.pr.new/@orpc/opentelemetry@1981

@orpc/pinia-colada

npm i https://pkg.pr.new/@orpc/pinia-colada@1981

@orpc/pino

npm i https://pkg.pr.new/@orpc/pino@1981

@orpc/publisher

npm i https://pkg.pr.new/@orpc/publisher@1981

@orpc/ratelimit

npm i https://pkg.pr.new/@orpc/ratelimit@1981

@orpc/server

npm i https://pkg.pr.new/@orpc/server@1981

@orpc/shared

npm i https://pkg.pr.new/@orpc/shared@1981

@orpc/swr

npm i https://pkg.pr.new/@orpc/swr@1981

@orpc/tanstack-query

npm i https://pkg.pr.new/@orpc/tanstack-query@1981

@orpc/trpc

npm i https://pkg.pr.new/@orpc/trpc@1981

@orpc/valibot

npm i https://pkg.pr.new/@orpc/valibot@1981

@orpc/zod

npm i https://pkg.pr.new/@orpc/zod@1981

commit: 537e5ea

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — Reviewed the full single-commit diff of feat(ai-sdk): stream tool outputs for async generator handlers (features, tests, type tests, and docs).

  • isAsyncGeneratorFunction in @orpc/shared — detects native async function* handlers via the built-in constructor; correctly handles bound functions and async generator methods, and excludes sync generators, plain async functions, and iterator values (the (async function* () {})() case).
  • createToolFactory streaming fallbackisIteratorOutput now also fires when the procedure handler is an async generator, so schema-less (and type<AsyncIteratorObject<...>>()-schematized) async generator handlers stream each event through the existing yield* await call(...) path instead of returning the raw iterator. Contract-based implementToolFactory is untouched.
  • Tests — streaming tests genuinely fail without the fix (for await over the old Promise-returning execute throws "not async iterable", verified empirically), and the non-generator test guards against over-broad detection. The getIteratorYieldSchemas(...) branch and the handler fallback are independently exercised.
  • Type test — pins InferToolOutput to the yield type for a schema-less async generator handler, matching the new runtime behavior.
  • Docs — the Streaming Tool Outputs sentence now names the async function* path.

Verified: affected unit tests pass, tsc -b (incl. .test-d.ts) passes, and eslint on the changed files is clean.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@dinwwwh
dinwwwh merged commit 88ee055 into middleapi:main Sep 4, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant