Repository navigation
docs: LiveKit guide fixes from the Dialog-RSN-1 review - #657
Merged
pmckeetx merged 2 commits intoOct 14, 2026
Merged
Conversation
- Troubleshooting quotes the plugin's real errors for a missing or wrong key. - "Why not the OpenAI plugin" no longer lists three failures that prod fixed on 13 Oct (session fields, metadata echo, call id length). It lists what still differs: audio by default, ignored OpenAI settings, rejected options. - Step 3 names the DIALOGUE_BASE_URL override the plugin reads. - The session snippet's TTS comment matches agent.py: Dialog-RSN-1 can now speak, so it says the plugin asks for text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 14, 2026
One name for the endpoint override across every Dialog-RSN-1 companion repo, matching the browser repo and the realtime-dev-kit examples. It holds the full realtime URL (wss://<host>/v1/realtime). No fallback to the old name, as with the DIALOG_API_KEY rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pmckeetx
added a commit
that referenced
this pull request
Oct 14, 2026
* docs: bring the Dialog-RSN-1 EAP docs onto docs.poly.ai Copies every page from the early access site (dialog-rsn-1__eap_docs main, c4fe9f3) into dialog-rsn-1/: overview, quickstart, prompt guide, prompt cookbook, evaluations, protocol reference and eight guides, plus the one image they use. Adds a Dialog-RSN-1 tab after ADK with the EAP site's groups. Changes from the EAP copy: - docs.poly.ai links are now relative - internal review-note comments with EAP feedback are removed - the protocol reference and the API reference link to each other Pages keep their Preview tags and the api.us.poly.ai host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: drop the Preview label and point Dialog-RSN-1 at the self-serve host Removes the Preview tag and the preview-API warning from every Dialog-RSN-1 page, and rewords the three "in this preview" phrases. Switches every api.us.poly.ai URL to api.studio.poly.ai. The quickstart and the Python guide now say the key is a workspace API key from Agent Studio. The quickstart notes that a studio.poly.ai key only works on api.studio.poly.ai and points regional accounts at the API reference's host table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: rename DIALOGUE_API_KEY to DIALOG_API_KEY in the Dialog-RSN-1 pages The product is Dialog API, and the companion repos use DIALOG_API_KEY. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 core pages: match prod as of 14 Oct (#655) * docs: match the Dialog-RSN-1 pages to prod as of 14 Oct Prod now starts sessions in output_modalities ["audio"] and speaks replies server-side (poly_core #47599), so text clients must set ["text"]. Fix the reference, overview, quickstart and API reference where they said audio out is refused, and add output_modalities ["text"] to the API reference voice samples, which printed an empty reply on prod as written. Also correct claims that no longer match prod or the server source: default input rate is 24 kHz, turn timing fields, semantic_vad and noise_reduction are accepted and ignored, MCP tools are refused, response.cancel with nothing active and a colliding response.create return errors, playback reports and an estimate set history after a barge-in, a mismatched call_id is never refused, audio history is bounded from the tenth turn, the gateway does not close a quiet connection after 60 s, and poly_confirmed and poly_resume_playback are not sent. Rename POLYAI_API_KEY to DIALOG_API_KEY in the API reference samples and make the error-handling sample runnable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Reference: poly_knowledge actions is a string An array is refused with invalid_request_error and the whole function_call_output is dropped (server model: actions: str | None). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Voice agent guide: DIALOGUE_URL, and match us-1 prod as of 14 Oct (#651) * Voice agent guide: say what DIALOGUE_URL is The Step 1 snippet reads config.DIALOGUE_URL next to DIALOG_API_KEY, and the guide never said what it holds. It defaults to us-1 production and .env can override it, as config.py and .env.example in the companion repo do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Voice agent guide: match us-1 prod as of 14 Oct - output_modalities: sessions now start with audio, so say why the demo sets text. - Turn timing: silence_duration_ms and threshold are accepted and ignored; idle_timeout_ms is still rejected. Troubleshooting row too. - Probe sample: a fresh run, including the heard line for a reply that played to the end, and the cut-off point it shows. - Latency: six more turns widen the lower bounds of first text and first audio. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rename the URL override to DIALOG_REALTIME_URL One name for the endpoint override across every Dialog-RSN-1 companion repo, matching the browser repo and the realtime-dev-kit examples. It holds the full realtime URL (wss://<host>/v1/realtime). No fallback to the old name, as with the DIALOG_API_KEY rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Python guide: declare the input rate, fix stale session claims (#652) - Send rate 16000 in audio.input.format. Without it prod assumes 24 kHz. - The 48 kHz fallback now changes only SAMPLE_RATE. - output_modalities: a session can start in audio, so text must be set. - turn_detection timing fields are now ignored rather than rejected. - Drop the reference citation for "five 32ms chunks", which it doesn't say. - Note PortAudio on Linux; only the first script runs on its own. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 browser guide review fixes (#653) * Browser guide: fix the rate mismatch claim, kHz spacing and relay diagram Hardcoding 48000 on 44.1 kHz hardware doesn't break recognition in testing; the server hears the audio about 9% fast. Say that instead of "breaks". Write kHz with a space, as the reference does, and realign the relay diagram after the host change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Browser guide: drop the claim that text output is locked Since 13 Oct, us-1 prod defaults sessions to output_modalities ["audio"] and streams response.output_audio.* when the field is left out. The guide's code sets ["text"] explicitly and still gets text only, so keep the code and say why the field must be set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * docs: match the migration and TTS guides to what prod accepts now (#654) Prod starts sessions in audio with a managed voice and ignores the OpenAI timing fields, semantic_vad, noise_reduction and voice names instead of rejecting them. List what is rejected and what is ignored, describe both reply routes, set text explicitly in the TTS guide, and name speech_started as the usual barge-in signal. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 speech to text guide: update for current prod (#656) * docs: update the speech to text guide for current prod behaviour Re-measured on us-1 prod: sessions now default to audio output, the model field and turn-timing fields other than idle_timeout_ms and interrupt_response are accepted, and the timing table was below what a mic-paced client sees. Also clarify the regional key and host, the WAV format, and the missing-key and 401 troubleshooting rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rename the URL override to DIALOG_REALTIME_URL One name for the endpoint override across every Dialog-RSN-1 companion repo, matching the browser repo and the realtime-dev-kit examples. It holds the full realtime URL (wss://<host>/v1/realtime). No fallback to the old name, as with the DIALOG_API_KEY rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * docs: LiveKit guide fixes from the Dialog-RSN-1 review (#657) * docs: update the LiveKit guide for the current plugin and prod behaviour - Troubleshooting quotes the plugin's real errors for a missing or wrong key. - "Why not the OpenAI plugin" no longer lists three failures that prod fixed on 13 Oct (session fields, metadata echo, call id length). It lists what still differs: audio by default, ignored OpenAI settings, rejected options. - Step 3 names the DIALOGUE_BASE_URL override the plugin reads. - The session snippet's TTS comment matches agent.py: Dialog-RSN-1 can now speak, so it says the plugin asks for text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rename the URL override to DIALOG_REALTIME_URL One name for the endpoint override across every Dialog-RSN-1 companion repo, matching the browser repo and the realtime-dev-kit examples. It holds the full realtime URL (wss://<host>/v1/realtime). No fallback to the old name, as with the DIALOG_API_KEY rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * docs: fix Dialog-RSN-1 prompt guide, cookbook and evaluation against prod (#658) - Prompt guide: poly_knowledge example used "actions": [], which prod refuses (actions is an optional string); the turn-timing warning now matches prod (timing fields and semantic_vad accepted and ignored; turn_detection null, interrupt_response false and idle_timeout_ms refused); "say X and call Y" describes what the model actually does; the complete-sequence pattern matches its recipe; two cookbook anchors used straight apostrophes where the rendered ids use curly ones. - Evaluation: first_token_latency paired a response.created with a text delta from a later response on tool turns; now pairs by response_id. Say to set output_modalities ["text"] for the listed events, and to grade poly_out_of_domain on the answering response. - Cookbook: backticks in two descriptions, session-object anchor on the turn-taking reference card. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * docs: match the Dialog-RSN-1 reference pages to dialogue_api master (#662) Usage is always present on response.done; create_response false skips the model's turn confirmation; capacity refusals close 1013; retrieve returns the transcript, not the audio; compacted item ids, close-code conditions, the split 429 rows, server voice settings, the missing error codes and refused fields, response.create overrides and held requests. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * docs: match the Dialog-RSN-1 prompt, evaluation, LiveKit and voice agent pages to the service code (#663) - Playback: without reports the server estimates playback at 15 characters a second and cuts history to the estimate on barge-in; with reports, history is cut to spoken_text. - create_response: false skips the model's end-of-turn check, so turns commit at the client's commit or the first VAD silence. - speech_started is model-confirmed only while a reply plays; a dropped noise leaves a start with no stop. - The input transcript comes last and only with a completed reply. - collect() no longer stops at the first response.done. - Prompt guide: instructions are placed verbatim into a prompt template; poly_spoken for hold lines; stale call_id is kept out of the prompt, not rejected; a corrected poly_knowledge resend is left out of the prompt. - Sneeze recipe works on the caller's turn only. - LiveKit: the plugin, not the service, drops per-reply instructions and non-function tools. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 guides: match the code on playback, transcripts and turn starts (#664) - Streaming TTS: without reports the server estimates playback at 15 chars/s; reports replace the estimate and spoken_text is written into history. The unspeakable-text check now pushes the stripped text. - Migration: list the response.create and audio item refusals; say when input_audio_transcription.completed does not arrive. - Speech to text: no .completed for a cancelled or failed reply, the transcriber keeps the partial line; the server resamples, the client doesn't. - Python: check for session.updated instead of discarding the reply; turn starts are only model-confirmed while a reply plays. - Browser: same turn-start correction, barge-in reason turn_detected, deltas can't interleave, full example disables the button and handles close. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 docs: get the key at studio.poly.ai, and name it POLY_API_KEY (#660) * Send readers to studio.poly.ai for their API key Assume every reader signs up or logs in at studio.poly.ai and creates the key in the API Keys tab, which is what api.studio.poly.ai accepts. Drops the regional-Studio detours from the setup steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rename DIALOG_API_KEY to POLY_API_KEY Same name the ADK uses: `poly apikey` writes the account key to POLY_API_KEY for the PolyAI APIs and Dialog-RSN-1, so one variable now works for both. No fallback to the old name, as with the earlier renames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 docs: US host only, English at launch, public example repos (#665) * Dialog-RSN-1 docs: US host only, English at launch, public example repos - API reference: drop the EU and UK hosts. Enterprise US accounts use api.us.poly.ai; Dialog-RSN-1 runs in the US only. - Limits and prompt guide: English at launch, more languages in a model update. poly_response_language stays documented as the mechanism. - API reference Examples: replace the internal realtime-dev-kit link with the public companion repos and dialog-rsn-1-realtime-agents. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 API reference and prompt guide: lead with spoken replies The API speaks replies by default, so the intro and the output_modalities row lead with ["audio"], the Run a turn samples say why they set ["text"], and the prompt guide no longer says the model answers in text by design. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Dialog-RSN-1 docs: lead with server voice, keep text as the bring-your-own-TTS option (#666) New sessions speak their replies by default, so the overview, reference and quickstart now start there. The quickstart script prints the audio transcript as it streams and saves the reply to reply.wav, and a new section covers output_modalities ["text"] for your own text-to-speech. Guides that set ["text"] now say why. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * docs(dialog-rsn-1): concurrency is 10 per account at launch; limits apply from 20 October (DEVP-994) (#649) * docs(dialog-rsn-1): concurrency is 10 per account at launch; limits apply from 20 October (DEVP-994) Go/No-Go 13 Oct: a flat cap of 10 replaces the 2/10/50 tiers and GA moves to 20 October. Drops the card-tier and 50-on-request wording from the limits page and points higher caps at contact us. * docs(dialog-rsn-1): source note says 20 October too --------- Co-authored-by: Peter McKee <peter.mckee@gmail.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andy Mohajeri <mohajeri.andysheh@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What I tested
All live tests ran against
api.us.poly.ai(viaDIALOGUE_BASE_URL=https://api.us.poly.ai/v1), because the demo key is refused onapi.studio.poly.ai(401 on both X-API-KEY and Bearer). The docs and repos still point atapi.studio.poly.ai; that host question is Peter's.livekit-plugins-polyai:uv sync && uv run pytest: 34 passed.ruff check,ruff format --checkandmypyclean.uv sync, thenuv run agent.py devagainstlivekit-server --dev(1.13.9). Onmainthe agent fails at once withDialog-RSN-1 needs an API key ... DIALOGUE_API_KEY:uv.lockpinned the plugin at its first commit (db64214), from before theDIALOG_API_KEYrename and theapi.studio.poly.aihost. After re-locking on plugin main (ebde908),uv run caller.pyran the whole call: greeting,look_up_reservation(GM40912), barge-in cut at "checking", fee confirmed,change_checkout_time(14:00),call.wavsaved. No errors in the agent log.session_config: sessions now start withoutput_modalities: ["audio"]. The plugin's firstsession.updateswitches to["text"], and replies arrive asresponse.output_text.deltaonly, with nooutput_audio.delta. So there's no server audio on top of ElevenLabs.livekit-plugins-openai1.8.3: I captured its real firstsession.updateand replayed it on prod, and it's now accepted (semantic_vadbecomesserver_vad, voice dropped).response.creatednow echoesresponse.metadata. Call ids are now 32 characters. A scratch copy ofagent.pyonopenai.realtime.RealtimeModel(base_url=..., modalities=["text"])ran the whole scripted call, tools and barge-in included. With default modalities it asks for audio, and LiveKit then plays the model's audio instead of running the TTS (agent_activity.py).reasoning,turn_detection.interrupt_response: false, and MCP tools.lk agent dev agent.py(whichagent.py devnow suggests in a deprecation warning) fails with "api-key is required" because it doesn't read.env, so the guide keepsuv run agent.py dev.agent.pyverbatim, at a 4-space dedent.npx mint@latest broken-links: no broken links in Dialog-RSN-1 pages. The 16 it finds elsewhere already existed (messaging-channel, voice-channel, release notes). External links in the guide all return 200.livekit-plugins-polyaiisn't on PyPI (404), and the guide installs from git.What changed in
dialog-rsn-1/guides/livekit.mdxaudio.outputrejected,generate_reply timed out, the 37-char call id) don't reproduce on prod since 13 Oct. The section now lists what still differs: the stock plugin asks for audio by default, so Dialog-RSN-1's voice replaces your TTS. It also sends OpenAI settings that Dialog-RSN-1 ignores, and it exposes options Dialog-RSN-1 rejects. The intro line now says the stock plugin "isn't built for" Dialog-RSN-1, instead of "doesn't work unchanged".Dialog-RSN-1 needs an API key,Dialog-RSN-1 refused the connection (401)). The stock-plugin row is now the symptom you'd actually see.DIALOGUE_BASE_URLto override the endpoint, as.env.exampleand the plugin do.agent.py(Dialog-RSN-1 can speak now).Left open for Peter
api.studio.poly.ai, so nothing here was tested on the documented host.livekit-plugins-polyai? I kept the plugin as the recommendation and made the comparison accurate.DIALOGUE_BASE_URLhere. Other repos use different names.Paired PRs: demo PolyAI-LDN/dialog-rsn-1-livekit#3, plugin PolyAI-LDN/livekit-plugins-polyai#11.
Also: URL override renamed to
DIALOG_REALTIME_URLPeter picked one name for the endpoint override across every Dialog-RSN-1 companion repo and guide:
DIALOG_REALTIME_URL, holding the full realtime URL (wss://<host>/v1/realtime). It matches the browser repo and the realtime-dev-kit LiveKit and Pipecat examples. No fallback to the old name, as with theDIALOG_API_KEYrename.Step 3 now says
RealtimeModel()connects towss://api.studio.poly.ai/v1/realtimeand thatDIALOG_REALTIME_URLoverrides it, matching PolyAI-LDN/livekit-plugins-polyai#11 and PolyAI-LDN/dialog-rsn-1-livekit#3.🤖 Generated with Claude Code