Skip to content

Say the plugin asks for text, and why replies match by order - #11

Merged
pmckeetx merged 2 commits into
mainfrom
docs-review-2026-10-14
Oct 14, 2026
Merged

pmckeetx merged 2 commits into
mainfrom
docs-review-2026-10-14

Conversation

@pmckeetx

@pmckeetx pmckeetx commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

What I tested

All live tests ran against api.us.poly.ai (via DIALOGUE_BASE_URL=https://api.us.poly.ai/v1), because the demo key is refused on api.studio.poly.ai (401 on both X-API-KEY and Bearer). The docs and repos still point at api.studio.poly.ai; that host question is Peter's.

  • livekit-plugins-polyai: uv sync && uv run pytest: 34 passed. ruff check, ruff format --check and mypy clean.
  • Demo, as a reader would: uv sync, then uv run agent.py dev against livekit-server --dev (1.13.9). On main the agent fails at once with Dialog-RSN-1 needs an API key ... DIALOGUE_API_KEY: uv.lock pinned the plugin at its first commit (db64214), from before the DIALOG_API_KEY rename and the api.studio.poly.ai host. After re-locking on plugin main (ebde908), uv run caller.py ran the whole call: greeting, look_up_reservation(GM40912), barge-in cut at "checking", fee confirmed, change_checkout_time(14:00), call.wav saved. No errors in the agent log.
  • Prod changed on 13 Oct. Probed with the plugin's own session_config: sessions now start with output_modalities: ["audio"]. The plugin's first session.update switches to ["text"], and replies arrive as response.output_text.delta only, with no output_audio.delta. So there's no server audio on top of ElevenLabs.
  • The stock livekit-plugins-openai 1.8.3: I captured its real first session.update and replayed it on prod, and it's now accepted (semantic_vad becomes server_vad, voice dropped). response.created now echoes response.metadata. Call ids are now 32 characters. A scratch copy of agent.py on openai.realtime.RealtimeModel(base_url=..., modalities=["text"]) ran the whole scripted call, tools and barge-in included. With default modalities it asks for audio, and LiveKit then plays the model's audio instead of running the TTS (agent_activity.py).
  • Still rejected whole on prod (nothing applies, instructions included): unknown fields, reasoning, turn_detection.interrupt_response: false, and MCP tools.
  • lk agent dev agent.py (which agent.py dev now suggests in a deprecation warning) fails with "api-key is required" because it doesn't read .env, so the guide keeps uv run agent.py dev.

What changed

Docs only, no behaviour change. The plugin already asks for text and matches replies by order, and both still work on prod after 13 Oct.

  • README.md: "Audio output: Not supported" now read like a service limit. Dialog-RSN-1 can speak now, so the row says the plugin asks for text.
  • realtime_model.py docstring: it said response.created carries no metadata. Prod echoes it now. Order matching still holds.

Left open

  • Prod now echoes metadata, so the plugin could match by response.metadata as well as by order. I left that alone: it's a design choice.
  • If a session.update were ever rejected, the session would stay on the new ["audio"] default. The plugin only reads output_text deltas, so the agent would go silent. It sends only fields prod accepts today, so this can't happen now.

Paired PRs: demo PolyAI-LDN/dialog-rsn-1-livekit#3, docs PolyAI-LDN/polyai-mintlify-doc#657.

Also: URL override renamed to DIALOG_REALTIME_URL

Peter picked one name for the endpoint override across every Dialog-RSN-1 companion repo and guide: DIALOG_REALTIME_URL, holding the full realtime URL (wss://<host>/v1/realtime). It matches the browser repo and the realtime-dev-kit LiveKit and Pipecat examples. No fallback to the old name, as with the DIALOG_API_KEY rename.

Plugin: reads DIALOG_REALTIME_URL instead of DIALOGUE_BASE_URL. protocol.realtime_url() already passes a full wss://…/realtime URL through unchanged, so only the env lookup, docstring and README change. uv run pytest 34 passed; ruff, format and mypy clean; with DIALOG_REALTIME_URL=wss://api.us.poly.ai/v1/realtime the model resolves that URL.

Merge this before PolyAI-LDN/dialog-rsn-1-livekit#3, which then needs a re-lock so the demo picks the rename up.

🤖 Generated with Claude Code

Dialog-RSN-1 on prod now defaults to audio output and echoes response
metadata. The README's "Audio output: Not supported" now reads as a service
limit, so say the plugin asks for text. The module docstring said
response.created carries no metadata; it does now, and order matching still
holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One name for the endpoint override across every Dialog-RSN-1 companion repo,
matching the browser repo and the realtime-dev-kit examples. It holds the full
realtime URL (wss://<host>/v1/realtime). No fallback to the old name, as with
the DIALOG_API_KEY rename.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pmckeetx
pmckeetx merged commit 4706bba into main Oct 14, 2026
7 checks passed
@pmckeetx
pmckeetx deleted the docs-review-2026-10-14 branch October 14, 2026 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant