Skip to content

fix(streaming): settle usage after client disconnect - #43

Merged
drewstone merged 1 commit into
mainfrom
fix/background-api-settlement-20260902
Sep 2, 2026
Merged

fix(streaming): settle usage after client disconnect#43
drewstone merged 1 commit into
mainfrom
fix/background-api-settlement-20260902

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

What changed

  • add an opt-in background continuation callback for OpenAI-compatible streams
  • keep the sandbox iterator and payment settlement alive after a client closes its reader
  • retain cancellation as the default when the callback is absent
  • stop response writes after disconnect without stopping usage collection

Why

Agent-app keeps persisted chat turns alive after an API viewer disconnects. The gateway previously stopped consuming that turn, so it could miss the final usage receipt and retain payment ownership.

Proof

  • Node 24: 447/447 tests pass
  • Node 22.13: 447/447 tests pass
  • focused middleware: 66/66 tests pass
  • typecheck and build pass
  • the new end-user cancel flow records input, output, reasoning, and provider cost after the reader closes
  • the existing cancel-by-default flow still aborts the sandbox iterator

@drewstone
drewstone merged commit 408dac6 into main Sep 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant