ElevenLabs support: expressive tags, pre-v3 prompt markup, v3 audio-tag dialect - #4
Merged
Merged
Conversation
…ause]) Mirrors the speechmarkdown-js reference grammar: a fixed keyword list parsed into a new NodeType::Expressive AST node. Text and SSML base formatters keep the tag verbatim, matching the JS TextFormatter and W3cSsmlFormatter behavior. Unknown bracketed content continues to pass through as plain text, which preserves Eleven v3's open-ended natural-language audio tags.
Ports the speechmarkdown-js ElevenLabsFormatter: <break time> for pauses (strength mapped to approximate durations), <phoneme> for IPA modifiers, expressive audio tags verbatim, everything else degrades to plain text. No <speak> wrapper, no XML escaping — a bare prompt, as the shared corpus .elevenlabs.ssml fixtures specify. Also fixes a parser bug: is_time_break accepted any token ending in 's'/'ms', so plain-text words like [apps] were parsed as breaks; it now requires a numeric body. The corpus runner now enforces the .elevenlabs.ssml fixtures (9 cases). Model notes: <break> works on multilingual_v2 / flash_v2_5 / flash_v2 (max 3s); <phoneme> only on flash_v2 / turbo_v2, English only; eleven_v3 parses neither — it needs the audio-tag dialect (next).
New Platform::ElevenLabsV3 ("elevenlabs-v3") formatter for eleven_v3 /
eleven_v3_conversational, which parse no SSML at all. SpeechMarkdown
constructs map to Eleven's native direction mechanisms:
- breaks -> ... / [short pause] / [pause] / [long pause] (three
approximate steps; v3 has no exact durations)
- emphasis -> [emphasized] / [stress on next word] / [understated]
prefix tags (never CAPS-mutates user text)
- whisper/excited/disappointed and rate/volume modifiers ->
[whispers]/[excited]/[rushed]/[drawn out]/[softly]/[loudly]
- IPA -> native " slash form; sub speaks the alias
- #[style] sections -> verbatim prefix tags (v3 tags are open-ended
natural language); audio/mark dropped
capabilities.rs rewritten for both dialects to match the documented
per-model truth: pre-v3 = break (<=3s) + flash-only phoneme; v3 =
audio tags + inline IPA, everything else unsupported with pointers to
API-level alternatives (voice_id, voice_settings.speed).
The constant said "eleven-labs" but Platform::from_platform_str only
accepts "elevenlabs" — every dotnet ElevenLabs conversion failed as
unsupported. Also adds the ElevenLabsV3 ("elevenlabs-v3") constant.
… comment's eleven-labs)
4 tasks
…tolerance corpus gate - pre-v3 formatter clamps <break> values above ElevenLabs' documented 3s limit (in-limit values keep the caller's unit formatting, matching the corpus fixtures' 3s/250ms) - is_time_break now rejects malformed numbers ([1.2.3s], [1..5s]) — at most one decimal point — closing the gap the previous fix left - phoneme attribute values escape double quotes (tag integrity; the prompt body stays unescaped by design) - corpus runner panics on ANY failure instead of a >50% threshold — the enforced fixture families are the regression net - both ElevenLabs formatters honor FormatterOptions.preserve_empty_lines (was hardcoded true behind a wrong #[allow(dead_code)])
Loop-2 review caught leading/trailing-dot forms ([5.s], [.5s], [0.s]) still parsing as breaks. The dot must now be surrounded by digits, matching the speechmarkdown-js grammar the code cites. Also drops two unused use-super imports flagged by clippy --all-targets.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Real SpeechMarkdown support for ElevenLabs. Previously
Platform::ElevenLabsfell through to the plain-text formatter:to_ssml(input, "elevenlabs")silently returned stripped text, andcapabilitiesadvertised elements ElevenLabs cannot parse (ElevenLabs has no SSML engine at all).ElevenLabs markup is model-dependent, so this adds two dialects:
elevenlabs(pre-v3: multilingual_v2 / flash_v2_5 / flash_v2)elevenlabs-v3(eleven_v3*)<break time="x.xs"/>(≤3s).../[short pause]/[pause]/[long pause]<phoneme>(flash_v2/turbo_v2 only, English only)"/IPA/"[emphasized]/[stress on next word]/[understated][whispers],#[excited], …)Cross-dialect emission is a correctness bug, not a degradation: v3 reads stray XML aloud; pre-v3 models read audio tags aloud (per ElevenLabs help center). Sources: help.elevenlabs.io articles 13416374683665 & 24352686926609, docs best-practices, v3-audiotags blog.
Changes
NodeType::Expressivefor the speechmarkdown-js keyword list ([laugh],[sigh],[applause]…), kept verbatim by Text/SSML-base formatters; unknown bracketed content still passes through as plain text (preserving v3's open-ended natural-language tags)elevenlabs.rs: pre-v3 formatter ported from the JSElevenLabsFormatter— breaks (strength→duration table), IPA→<phoneme>, no<speak>wrapper, no escaping; matches the shared corpus.elevenlabs.ssmlfixtures, which the corpus runner now enforceselevenlabs_v3.rs: newPlatform::ElevenLabsV3("elevenlabs-v3"), sections → prefix tags, sub speaks alias, audio/mark droppedcapabilities.rs: rewritten for both dialects to match documented reality (was: prosody/audio/mark/say-as falsely advertised)is_time_breakaccepted any token ending in "s" (plain-text[apps]became a break); dotnet binding'sElevenLabsconstant was"eleven-labs"which never parsedNotes
Test plan
.elevenlabs.ssml) passcargo clippy --all-featuresclean