feat(telemetry): one agent_turn span per speech handle by davidzhao · Pull Request #2500 · livekit/agents-js

davidzhao · 2026-09-15T06:32:15Z

Port of livekit/agents#7143. Stacked on #2499.

Description

A reply that calls a tool runs two generations (LLM steps) in two tasks. They were two agent_turn spans linked only by lk.parent_generation_id, so one response rendered as two turns. A speech handle is now exactly one agent_turn.

The speech handle owns the span. The first reply task (pipeline, realtime, or say) opens agent_turn under agent_session; the follow-up generation after a tool call continues the open span instead of opening a second one. It ends with the speech in SpeechHandle._markDone, recording the speech's error (redaction-aware) and the gen_ai.invoke_agent.duration metric for the whole turn.
Each generation is a generation event on the span with lk.generation_id and lk.parent_generation_id; the span's lk.generation_id names the latest generation and the new lk.generation_count how many there were. lk.speech_id is set at creation.
Every step's llm_node, function_tool, tts_node, realtime_inference and agent_speaking nest under the one turn.
A preemptive generation discarded for a successor answering the same user turn hands its open agent_turn to that successor (preemptive_generation_discarded event; lk.speech_id follows the speech that answered). An attempt cancelled with no successor still ends as its own turn. The queue-wait and interruption helpers tolerate a span already ended with the speech.

agent_turn  (lk.speech_id, lk.generation_count=2; events: generation ×2)
├─ llm_node          generation 1: the tool call
├─ function_tool
├─ llm_node          generation 2: the reply
├─ tts_node
└─ agent_speaking

Changes Made

voice/speech_handle.ts: span ownership, _generationId / _parentGenerationId, _takeAgentTurn / _continueAgentTurn, end in _markDone.
voice/agent_activity.ts: withAgentTurn and continueDiscardedTurn wrap the pipeline, realtime and say tasks; per-step span creation removed.
telemetry/otel_metrics.ts: recordInvokeAgentDuration (gen_ai.invoke_agent.duration, unit s, job attribution); telemetry/trace_types.ts: lk.generation_count, the metric name.

Adaptations from the Python source

say previously had no agent_turn span in JS; it now gets one like Python's TTS task.
lk.generation_id / lk.parent_generation_id were never stamped on agent_turn in JS before (the constants existed unused); they are now, as <speech_id>_<step>.

Testing

New voice/agent_turn_span.test.ts (6 tests): a tool-calling reply is one agent_turn with two generation events and both llm_nodes, the tool, tts_node and agent_speaking inside it; a plain reply is one generation; the discarded preemptive hand-off; an LLM failure fails the turn; the duration metric when sampled out.
Full agents suite green; build, typecheck, lint, API report updated.

Review follow-ups

A realtime tool reply, which the framework runs on a new speech handle (Python runs it on the same one as its next step), now continues the tool call's agent_turn instead of opening a second one: continueToolReplyTurn hands the open span over, the turn keeps the tool call's speech id, and the reply's generation is numbered after and parented to the tool call's (lk.generation_id / lk.parent_generation_id continue the original handle's numbering). The exported trace matches Python's: one turn, two generations, parent link intact.
A speech whose owned task fails now ends its turn with the error and reports it through exception(): the first non-cancellation rejection of an owned task is stored on the handle before it is marked done. The pipeline also stores a genuine LLM node failure the way Python's _on_llm_task_done does, since nothing awaits that task's rejection.
lk.generation_count counts the generation events on the turn, carried across handoffs, instead of the current handle's own step count: a turn continued from a discarded preemptive attempt or a tool call reports every generation it shows. Python's _continue_agent_turn needs the same carry.

🤖 Generated with Claude Code

changeset-bot · 2026-09-15T06:46:52Z

🦋 Changeset detected

Latest commit: 2cc06f9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-anthropic	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-azure	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-did	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-krisp	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-meta	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-protoface	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-soniox	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration

Devin Review found 2 new potential issues.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

devin-ai-integration · 2026-09-26T05:28:21Z

+    this._agentTurnSpan = span;
+    this._agentTurnContext = trace.setSpan(otelContext.active(), span);
+    this._agentTurnStartedAt = startedAt;
+    this._agentTurnAgentName = agentName;
+    this._agentTurnGenerations = carry.generations;


🟡 Discarded preemptive turn loses initial queue wait

After a preemptive handoff, _queueWaitRecorded remains false on the successor. Its authorization overwrites the first generation's queue wait on the shared span.

Learn more

The queue-wait attribute is meant to describe the first generation of a turn. recordQueueWait stamps it once per handle, guarded by _queueWaitRecorded. A discarded preemptive handle can already have recorded that attribute, but _continueAgentTurn copies only the span and generation count. When the replacement handle is authorized, its fresh flag permits another stamp on the same span.

Example: An initial preemptive attempt waits 300 ms before authorization; a replacement waits 10 ms. The exported turn reports 10 ms rather than the first generation's 300 ms.

Recommended fix: Carry _queueWaitRecorded with the turn and restore it on the successor, or guard the attribute at the span/turn level rather than per handle.

Was this helpful? React with 👍 or 👎 to provide feedback.

Port of livekit/agents#7143. A reply that calls a tool runs two generations (LLM steps) in two tasks; they were two agent_turn spans linked only by lk.parent_generation_id, so one response rendered as two turns. A speech handle is now exactly one agent_turn. - The speech handle owns the span: the first reply task (pipeline, realtime, or say) opens agent_turn under agent_session; the follow-up generation after a tool call continues the open span. It ends with the speech in SpeechHandle._markDone, recording the speech's error redaction-aware and the gen_ai.invoke_agent.duration histogram for the whole turn (new in JS: otel_metrics.recordInvokeAgentDuration and trace_types.METRIC_GEN_AI_INVOKE_AGENT_DURATION; the Python metric already existed). - Each generation is a `generation` event with lk.generation_id and lk.parent_generation_id (SpeechHandle._generationId / _parentGenerationId, `<speech id>_<step>` like Python); the span carries the latest generation id and the new lk.generation_count. - A preemptive generation discarded for a successor answering the same user turn hands its open agent_turn over (preemptive_generation_discarded event, lk.speech_id follows the speech that answered), both on a newer attempt and on the real reply after onUserTurnCompleted invalidated it. The queue-wait and interruption helpers tolerate an ended span. - `say` gets an agent_turn too, as in Python's _tts_task; JS had none. Tests: agent_turn_span.test.ts (tool call is one turn with two generation events and every step nested; plain reply is one generation; discarded preemptive hand-off; LLM failure fails the turn; duration metric when sampled out; sampled-out hand-off). The preemptive-guard stand-in handle gained _takeAgentTurn; the PII key test skips METRIC_* names, which are not attribute keys. The handoff runs inside generateReply, before the reply task is created: Task starts its body synchronously and the task opens the turn in its first statements, so a handoff performed after generateReply() returned came too late, leaving the successor's own agent_turn unended and its llm_node, tts_node and function_tool dangling from a span that was never exported (seen in a cloud export as bare nodes and turns whose generation count did not match their events). A regression test drives the real reply task. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

… the turn's speech id - endAgentTurn records a thrown string or object as an exception, so the turn does not read as a success while the handle reports a failure - the reply steps stamp the turn's speech id (the root speech's when a realtime tool reply continues its parent's turn on a new handle) instead of overwriting it with the step's own handle id Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

With queue waits measured per generation, a tool reply on the same handle would replace the turn's lk.speech_queue_wait with its own; the span keeps the wait before the reply started. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…rd; chained reply ids - a realtime model that answers its tool calls itself (autoToolReplyGeneration) opens the reply generation after the tool call's speech is done: the turn is now held open for it (AutoToolReplyTurnHold) and adopted by the model's next generation, ended after 5 s or at activity close when none comes - the pipeline task stamps lk.interrupted only while its speech still owns the turn, so a discarded preemptive attempt unwinding cannot write the successor's verdict - a handle that adopted a turn but never emitted a generation passes the numbering on from where it stood, so the next id follows an existing one; chained replies tested to three generations Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

davidzhao added this pull request to stack #2502 September 15, 2026 06:32

davidzhao mentioned this pull request Sep 15, 2026

tests: trace-shape schema and checker #2501

Merged

davidzhao force-pushed the dz/telemetry-agent-turn branch from 9a563af to 91cea4d Compare September 15, 2026 06:46

davidzhao force-pushed the dz/telemetry-agent-turn branch from 91cea4d to dd4e334 Compare September 16, 2026 04:28

davidzhao force-pushed the dz/telemetry-agent-turn branch 2 times, most recently from 241cd41 to 3c94ca8 Compare September 20, 2026 05:02

davidzhao force-pushed the dz/telemetry-agent-turn branch from 3c94ca8 to 47d445d Compare September 20, 2026 05:58

davidzhao force-pushed the dz/telemetry-agent-turn branch from 47d445d to c5d72a6 Compare September 20, 2026 06:11

davidzhao force-pushed the dz/telemetry-agent-turn branch from c5d72a6 to 2ee50d2 Compare September 20, 2026 07:14

davidzhao force-pushed the dz/telemetry-agent-turn branch from 2ee50d2 to 3c49f49 Compare September 20, 2026 07:15

davidzhao force-pushed the dz/telemetry-agent-turn branch 2 times, most recently from 548c6e5 to 308e57f Compare September 20, 2026 17:17

davidzhao force-pushed the dz/telemetry-agent-turn branch from 308e57f to e95ba38 Compare September 20, 2026 17:43

davidzhao force-pushed the dz/telemetry-agent-turn branch 2 times, most recently from 3d4201a to fb87ab8 Compare September 20, 2026 23:39

davidzhao force-pushed the dz/telemetry-agent-turn branch from 0aca960 to c150336 Compare September 21, 2026 05:05

davidzhao force-pushed the dz/telemetry-agent-turn branch from c150336 to b50cc56 Compare September 21, 2026 05:18

This comment was marked as resolved.

Sign in to view

davidzhao force-pushed the dz/telemetry-agent-turn branch from b50cc56 to 31a8d6d Compare September 21, 2026 15:35

davidzhao force-pushed the dz/telemetry-agent-turn branch from 31a8d6d to bb31be9 Compare September 21, 2026 15:37

chenghao-mou approved these changes Sep 23, 2026

View reviewed changes

Comment thread agents/src/voice/agent_activity.ts Outdated

Comment thread agents/src/voice/speech_handle.ts Outdated

davidzhao force-pushed the dz/telemetry-agent-turn branch from bb31be9 to 7ae16ae Compare September 23, 2026 15:10

davidzhao force-pushed the dz/telemetry-agent-turn branch from 7ae16ae to 291af61 Compare September 26, 2026 01:56

This comment was marked as resolved.

Sign in to view

davidzhao force-pushed the dz/telemetry-agent-turn branch from 291af61 to 6ebf417 Compare September 26, 2026 05:02

This comment was marked as resolved.

Sign in to view

davidzhao force-pushed the dz/telemetry-agent-turn branch from 6ebf417 to 31677aa Compare September 26, 2026 05:26

devin-ai-integration Bot reviewed Sep 26, 2026

View reviewed changes

davidzhao force-pushed the dz/telemetry-agent-turn branch from 31677aa to 81529d0 Compare September 26, 2026 05:40

davidzhao and others added 4 commits September 25, 2026 22:52

davidzhao force-pushed the dz/telemetry-agent-turn branch from 81529d0 to 2cc06f9 Compare September 26, 2026 05:54

davidzhao merged commit 72898e1 into main Sep 26, 2026
12 of 13 checks passed

davidzhao deleted the dz/telemetry-agent-turn branch September 26, 2026 06:04

Navigation Menu

feat(telemetry): one agent_turn span per speech handle - #2500

feat(telemetry): one agent_turn span per speech handle#2500
davidzhao merged 4 commits into
dz/telemetry-coveragefrom
dz/telemetry-agent-turn

davidzhao commented Sep 15, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Sep 15, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

devin-ai-integration Bot Sep 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

davidzhao commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes Made

Adaptations from the Python source

Testing

Uh oh!

changeset-bot Bot commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

devin-ai-integration Bot Sep 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

davidzhao commented Sep 15, 2026 •

edited

Loading

changeset-bot Bot commented Sep 15, 2026 •

edited

Loading