feat(telemetry): interruption detail, handoff span, fallback events, text input - #7137
Merged
Merged
Conversation
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
2 times, most recently
from
September 5, 2026 20:42
d274cf0 to
f35af04
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 5, 2026 20:48
f35af04 to
dd9b4df
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 5, 2026 21:00
dd9b4df to
2d2ae2a
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 5, 2026 21:07
2d2ae2a to
40ce794
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 5, 2026 21:10
40ce794 to
8f34d94
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
2 times, most recently
from
September 6, 2026 00:31
d491a27 to
9e40051
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
2 times, most recently
from
September 6, 2026 01:09
2bdd416 to
def301b
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 6, 2026 01:28
def301b to
87874d3
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 6, 2026 01:37
87874d3 to
14c2b7c
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
2 times, most recently
from
September 6, 2026 05:23
3731e28 to
9289b47
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 6, 2026 05:42
6113f9f to
895ab3d
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 6, 2026 23:04
4bedb0b to
02b24cd
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 7, 2026 00:10
a2849a9 to
b00f8eb
Compare
davidzhao
marked this pull request as ready for review
September 7, 2026 03:40
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 7, 2026 03:54
b00f8eb to
4801016
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 7, 2026 05:05
1f2a1e1 to
fc68466
Compare
davidzhao
force-pushed
the
dz/telemetry-coverage
branch
from
September 7, 2026 05:48
fc68466 to
28c70f8
Compare
chenghao-mou
reviewed
Sep 7, 2026
| self._authorized_at: float | None = None | ||
| # telemetry: who interrupted this speech, recorded by the caller that knows | ||
| self._interrupt_source: str | None = None | ||
| self._interruption_recorded = False |
Member
There was a problem hiding this comment.
nit: I don't think this is used?
chenghao-mou
approved these changes
Sep 9, 2026
| """Priority for important messages that should be played before others.""" | ||
|
|
||
| def __init__( | ||
| self, *, speech_id: str, allow_interruptions: bool, input_details: InputDetails |
Member
There was a problem hiding this comment.
I wonder if we should expose the source as a parameter here. We still call .interrupt() in the following cases:
- The realtime path and superseded-turn path still interrupt without setting
audio_activityoruser_turn. Both regressions failed.
| if output_emitter.pushed_duration() > 0.0: | ||
| logger.warning( | ||
| f"{tts.label} already synthesized of audio, ignoring the current segment for the tts fallback" # noqa: E501 | ||
| ) |
Member
There was a problem hiding this comment.
we should record here too if this has pushed some audio.
…text input Smaller coverage gaps on existing spans, each of which came up when reading a trace and not being able to answer a question from it. Interruptions: agent_turn carried a single lk.interrupted boolean. It now records an `interrupted` event with lk.interruption.source, set by the caller that knows the cause: `audio_activity` (barge-in), `user_turn` (a committed turn preempting the reply), or `programmatic` (session.interrupt(), a tool, teardown). The pipeline and say paths also stamp lk.playout.position, how many seconds had actually played when the user cut in. The false-interruption path records `playout_paused` when a barge-in pauses playout and a `false_interruption` event with lk.false_interruption.resumed when the timer resolves it. Agent handoff: update_agent() spans a new `update_agent` bar (parent: agent_session) with lk.previous_agent_label and lk.agent_label; the old agent's drain_agent_activity (with on_exit inside) and the new agent's start_agent_activity nest under it. The initial start stays under session_start. Fallback adapters (LLM and TTS): the request span gets a `fallback_provider_failed` event per provider that failed (lk.fallback.label, lk.fallback.index) and, on success, lk.fallback.label / lk.fallback.index for the provider that served the request. Previously the adapter only renamed the request span, so a silent failover was invisible. Text input: a `text_input` event on agent_session (participant, byte size) when a lk.chat text stream arrives, so text-mode conversations show their input edge. Tests: tests/test_coverage_spans.py covers barge-in source and playout position through a fake session, the update_agent span and its children, LLM fallback events, and interrupt-source precedence. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The keyterm-detection LLM pass ran in the reply task's context, so its llm_request appeared directly under agent_turn as if it were a second inference step. It is STT context for later turns, not part of any reply: a keyterm_detection span under agent_session, with model, provider and keyterm counts (the terms themselves stay in the session report as lk.pii.keyterms). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ecord the model that answered FallbackAdapter.model/provider (LLM, TTS, STT) now follow the active instance, the primary before any traffic and then whichever last served, as metrics_metadata already did. llm_node, tts_node, start_agent_activity and the adapter's own request span name a real model instead of the adapter. Since request attributes are stamped at request start, a failover mid-request is recorded on the response side: gen_ai.response.model and gen_ai.provider.name of the instance that answered, on the adapter request span and on the node when it finishes. Fallback events carry the instance's model and provider next to its label. Usage metrics needed no change: inner streams emit them with their own model and the adapters suppress their own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…the message The conversation event that starts a detection pass fires from the reply answering the user message, so the pass is that agent's work on the turn. Parent it to the current span at trigger time (the agent_turn), falling back to the session root when nothing is current (a skipped reply, user code editing the history). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ource before the pause cancel, handoff start under update_agent, text input size in bytes - fallback adapters stamp the span the request was made under (llm_node/tts_node) with the instance that served, at the moment it serves, instead of the node reading the adapter's shared active instance when it finishes: concurrent requests on one adapter can be served by different instances - a paused reply resolved by a user turn is interrupted by the pause cancel before its cause was named; name it first, and in the pause cancel itself - a handoff's start_agent_activity takes the update_agent span as its parent explicitly rather than relying on the session start context being cleared - lk.text_input.size counts UTF-8 bytes like the RPC payload sizes Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…alize the keyterm span's provider model/provider read the first instance marked available (the primary once all are down), so a recovered primary is reported again before it has served; the instance that actually serves is stamped per request by the stream. keyterm_detection now runs the provider through gen_ai_provider_name like every other span, omitting it when unknown. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…andoff span is a parent, not current The UI renders spans and attributes, so the interrupted, playout_paused, false_interruption, fallback_provider_failed and text_input events go; lk.interruption.source stays as an attribute on agent_turn, and a failover is readable from the request span's expected vs served model. update_agent was made current for the whole handoff, so the long-lived tasks the new activity spawned inherited it and their spans nested under an ended span; it is now current only around drain/pause/close and passed explicitly to start and resume. The playout position is stamped only when this speech's own audio played, since wait_for_playout returns the previous segment's event otherwise. Comments trimmed to what a reader of the code needs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…me barge-in; record partially serving tts instance - SpeechHandle.interrupt(source=...) and AgentActivity.interrupt(source=...) carry the cause of the interruption for lk.interruption.source; _set_interrupt_source removed - realtime input_speech_started interrupts as audio_activity; superseded turns as user_turn - tts fallback records the serving instance when it failed after pushing audio Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Smaller coverage gaps on existing spans, each of which came up when reading a trace and not being able to answer a question from it.
Interruptions
agent_turncarried a singlelk.interruptedboolean. It now also carrieslk.interruption.source, set by the caller that knows the cause:audio_activityuser_turnprogrammaticsession.interrupt(), a tool, teardownThe pipeline and
saypaths also stamplk.playout.position, how many seconds had actually played when the user cut in. No events: the UI renders spans and attributes, so the interruption detail lives on the turn.Agent handoff
update_agent()spans a newupdate_agentbar (parentagent_session) withlk.previous_agent_labelandlk.agent_label. The old agent'sdrain_agent_activity(withon_exitinside) and the new agent'sstart_agent_activitynest under it. The initial start stays undersession_start.Fallback adapters (LLM and TTS)
The attempt span carries
lk.fallback.label/lk.fallback.indexfor the provider that served the request; a failover shows asgen_ai.request.model(expected) differing fromgen_ai.response.model(served) on the request span. STT fallback is left for the deferred STT-events work since STT has no request span to hang these on yet.Text input
keyterm_detectionspan around the keyterm-detection LLM pass (stt_context_options.keyterm_detection), nested under theagent_turnthat answers the user message (the conversation event fires from that reply: it is the agent's work on the turn), falling back toagent_sessionwhen there is no reply. Itsllm_requestused to sit directly underagent_turn, looking like a second inference step. Attributes are counts only (lk.keyterms.count/added/removed) plus model and provider; the terms themselves stay in the session report aslk.pii.keyterms.Fallback adapters report who serves.
FallbackAdapter.model/.provider(LLM, TTS, STT) follow the active instance: the primary before any traffic, then whichever last served, likemetrics_metadataalready did.llm_node,tts_node,start_agent_activityand the adapter's own request span therefore name a real model instead ofFallbackAdapter/livekit. Because request attributes are stamped when a request starts, a failover mid-request is recorded on the response side:gen_ai.response.modelandgen_ai.provider.nameof the instance that answered are set on the adapter request span and onllm_node/tts_nodewhen the node finishes. The serving run carries the instance'sgen_ai.request.modelandgen_ai.provider.namenext tolk.fallback.label/index. Usage metrics needed no change: the inner streams already emit them with their own model and the adapters suppress their own.Tests
tests/test_coverage_spans.py: barge-in source and playout position through a fake session, theupdate_agentspan with drain/on_exit/start nested, the LLM fallback serving provider, and interrupt-source precedence.Not in this layer
STT events on the turn (deferred behind
eou_waitfeedback), realtime session lifecycle, and IPC high-ping / CPU pressure events.Stacked on #7134.
🤖 Generated with Claude Code