Update AssemblyAI skill: universal-3-5-pro across the board (streaming + async), Speech Understanding summarization/action items, Voice Agent BYO LLM + webhooks#17
Open
dlange-aai wants to merge 2 commits into
Conversation
…arization/action items, streaming language_codes/Opus, Voice Agent BYO LLM + webhooks
Sync the skill with AssemblyAI docs/spec changes since the last update:
Streaming:
- u3-rt-pro (Universal-3 Pro Streaming) removed from the model picker and
the streaming spec speech_model enum (July 2026); reframed as legacy and
superseded by universal-3-5-pro. Enum now: universal-3-5-pro,
universal-streaming-english, universal-streaming-multilingual.
- Replace the (nonexistent) singular language_code input param with the
actual language_codes (plural, list) steering param.
- Add opus / ogg_opus encodings (sample_rate ignored for Opus).
- Correct defaults: prompt/agent_context <=1750 chars,
previous_context_n_turns default 5 (range 0-100), vad_threshold 0.2/0.5,
interruption_delay mode-dependent + fixed 256ms; max_speakers hard cap.
Speech Understanding:
- Add native summarization (summary_type + effort) and action_items
(include_decisions + effort) features under speech_understanding.request.
- Clarify that top-level summarization/auto_chapters stay deprecated; these
are the modern replacements (or the LLM Gateway).
Voice Agents:
- Add Bring-your-own-LLM (llm field on stored agents; OpenAI-compatible).
- Add Voice Agent Webhooks API (/v1/webhook-subscriptions).
- HTTP tool headers are now a list of {name,value} (not a dict); reads
return name + last_set_at only (not masked "***").
- Correct input languages to the 18 Universal-3.5 Pro Streaming languages.
…ing + async) universal-3-5-pro is now GA for both realtime/streaming and pre-recorded/async transcription and is the default on each. The async speech_models enum and default were updated in the spec to ["universal-3-5-pro", "universal-2"], and universal-3-pro was removed from the pre-recorded model list/enum (superseded). - SKILL.md: lead the Models section with "universal-3-5-pro across the board"; drop the Universal-3 Pro row and the stale "opt-in / enum still lists universal-3-pro" async caveat; async speech_models now defaults to ["universal-3-5-pro", "universal-2"]; note the speech_model_used response field. - api-reference.md: speech_models default/enum updated; temperature, remove_audio_tags, language_codes, and Medical Mode retargeted to Universal-3.5 Pro; warning example uses universal-3-5-pro. - speech-understanding.md: Translation models now Universal-3.5 Pro + Universal-2. - audio-intelligence.md: Medical Mode pre-recorded model is Universal-3.5 Pro. - python-sdk.md: speech_models fallback example uses universal-3-5-pro.
dlange-aai
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated docs-sync run (2026-07-06), plus a follow-up repositioning. Compared the current skill against the AssemblyAI docs/spec and applied targeted updates.
universal-3-5-prois now the model to use across the boardIt is GA for both realtime/streaming and pre-recorded/async, and is the default on each. The spec updates confirm this:
speech_modelsnow defaults to["universal-3-5-pro", "universal-2"]and the enum accepts only those two —universal-3-prowas removed from the pre-recorded model list/enum (superseded), mirroring theu3-rt-proremoval on streaming.temperature,remove_audio_tags,language_codes), Medical Mode, and Translation to Universal-3.5 Pro; updated the Python SDKspeech_modelsexample; noted thespeech_model_usedresponse field.Streaming (
references/streaming.md,SKILL.md)u3-rt-proremoved from the streaming model picker andspeech_modelspec enum (docs PR #177). Enum now lists onlyuniversal-3-5-pro,universal-streaming-english,universal-streaming-multilingual. Reframed as legacy and updated every "universal-3-5-pro / u3-rt-pro" annotation.language_codes(plural, list) is the real language-steering connection param (docs PR #170); removed the inaccurate singularlanguage_codeinput param.encodingnow acceptsopusandogg_opus(sample_rateignored for Opus).prompt/agent_context≤1750 chars;previous_context_n_turnsdefault5(0–100);vad_threshold0.2/0.5;interruption_delaymode-dependent + fixed 256ms;max_speakershard cap (docs PR #180).Speech Understanding (
references/speech-understanding.md,SKILL.md)summary_type+effort) and Action Items (include_decisions+effort) underspeech_understanding.request.<feature>(docs PRs #171, #176).summarization/auto_chaptersstay deprecated; these are the modern replacements (or the LLM Gateway).Voice Agents (
references/voice-agents.md,references/api-reference.md,SKILL.md)llmfield on stored agents (OpenAI-compatible;base_url/model/api_key; one entry; streamed;[]reverts to managed; can target the LLM Gateway)./v1/webhook-subscriptionsendpoints.headersshape change (docs PR #151): list of{name, value}(write) /{name, last_set_at}(read), not a dict; values never returned. Fixed the stale"***"masking claim.Only information clearly documented in the spec was changed; existing structure and style preserved. The untracked local
evals/dir is out of scope.