RustTts floravox catch-up: ABI canary, pinned DLL, floravox routing, sidecar G2P (issue #15) - #16
Open
willwade wants to merge 7 commits into
Open
Conversation
… drop SSML debug dump Issue #15 step 1. CI and the local release build picked whichever rust_tts_wrapper.dll sorted first in the NuGet cache, decoupled from the RustTtsWrapper.Bindings pin — a latent callback-ABI mismatch. Both now resolve via scripts/Get-RustTtsWrapperDll.ps1, which reads the csproj pin and fails on floating versions. CI also copies the win-x86 DLL (previously x64-only) and hard-fails when the pinned DLL is missing instead of shipping without it. Also removes the hardcoded vg_ssml_debug.txt dump from SpeakSsml.
Issue #15 step 2. The thunk code has targeted the consolidated 7-arg boundary callback since e80eca2, but the shipped 0.3.20 DLL exports the old 4-arg signature under the same symbol name - GetProcAddress cannot tell them apart, and the mismatch read garbage registers into the boundary lambda. - RustTtsWrapper.Bindings 0.3.20 -> 0.5.3 (combined sherpaonnx + floravox-lexicons build; see rust-tts-wrapper publish change) - Loader now resolves tts_set_on_mark and refuses any DLL without it: the symbol only exists in DLLs built with the consolidated callback ABI, so it discriminates exactly the builds that are safe to call - Engine: SetOnMark plumbing (thunk + registration, used in the next commit for SPEI_TTSBOOKMARK) - Engine header: fix the stale 6-arg OnBoundaryThunk declaration (it silently overloaded the real 7-arg definition) - scripts/test-rust-abi.ps1: export-table check for the pinned DLL, canary, and a negative control against a cached pre-consolidation 0.3.x DLL
… marks, lexicon G2P Issue #15 step 4 (SPD parity). Local voices whose model dir carries a piper sidecar (*.onnx.json) now create a 'floravox' engine instead of 'sherpaonnx', keeping sherpaonnx as fallback when floravox is missing (the 32-bit DLL ships without it) or fails to build: - credentials: modelsDir + relative modelId, plus 'lang' (fetches the published gruut-lexicon/Phonetisaurus/ByT5 G2P bundle for that language; cached, degrades to the model phonemizer offline) and misaki us/gb for English heteronyms and numbers - SSML path enabled for floravox: BuildSSML output goes through tts_speak_ssml; boundaries and marks map back to SAPI offsets via m_offsetMappings (MapTextOffset) instead of the plain-text map. Bookmarks are emitted as SSML-standard <mark name='...'/> for floravox (the <bookmark/> dialect is Azure-only); marks fire real SPEI_TTSBOOKMARK events and retire the simulated ones - boundaries: charOffset < 0 now emits an audio-time-only event instead of being dropped; the estimated flag is surfaced in logs - measured timings (floravox duration tensor) are trusted verbatim and never shifted by the online silence compensation Adds VoiceGarden.UI.Tests (xunit): floravox integration through the shipped 0.5.3 DLL - SSML synthesis with boundaries + marks, missing voice error surfacing, OOV lexicon G2P.
… hardening port) Issue #15 step 5. MMS/coqui voices ship in sherpa layout (model.onnx + tokens.txt) with no *.onnx.json, which is what the C++ adapter uses to route piper-family voices to floravox - so they could never take the measured-timing/lexicon-G2P path. PiperSidecarGenerator builds the sidecar from tokens.txt (skipping count headers, merging duplicate symbol ids), case-folding phoneme map keys (G2P looks phonemes up case-insensitively), deriving a language.code sidecar from the model id (floravox 0.8.5 language routing), and reading sample_rate from the catalog or a sibling MMS config.json. Sidecars carry a generator marker: - only generated sidecars are ever refreshed (tokens.txt newer -> re-run, per the SPD rule); shipped/patched sidecars are authoritative and never overwritten - writes are atomic (.tmp + move) so concurrent scans never see a partial json Wired into ScanInstalledModels and the post-extraction download path (catalog sample rate available there). 15 new xunit tests cover generation, casefold, stale re-run, shipped-sidecar protection, language derivation, and the tokens parser; smoke-verified end to end: mms_eng speaks through floravox with boundaries via the generated sidecar.
…test suites README predated floravox routing (issue #15): engine count said 21+, the offline row only listed SherpaOnnx, the architecture diagram showed a 22MB/21-engine DLL with estimated-only local boundaries, and the testing section missed the new xunit suite and the DLL-pin/ABI-canary scripts. Also notes the Win10 1903+ floor for floravox on x64 (falls back to SherpaOnnx) and the sidecar generator + tests in the component table.
…tcha too floravox-core auto-detects backends from the graph (piper/MMS VITS, Matcha +vocoder, Kokoro) and self-serves sherpa layout: KokoroBackend reads tokens.txt + voices.bin directly, MatchaBackend finds a sibling vocoder .onnx, VitsBackend falls back to tokens.txt - none of them need a generated sidecar. The sidecar-presence routing gate therefore kept Kokoro/Matcha on sherpa-onnx for no reason (verified locally: kokoro speaks through floravox from pure sherpa layout, misaki included). - ModelSupportsFloravox replaces HasPiperSidecar: allowlist by layout (voices.bin / vocoder .onnx / piper sidecar / tokens.txt) with a name-based exclusion for the flow families floravox cannot load (zipvoice, supertonic, pocket, kitten - they carry tokens.txt too) - floravox credentials: kokoro-en voices get lang en + misaki us (Kokoro's own phonemizer) - sidecar generator: same flow-family exclusion; download hook also gates on the catalog model_type - tests: kokoro-through-floravox (sherpa layout, no sidecar), flow families never get sidecars (23 total)
…e fallback
floravox without a 'lang' credential has no phonemizer and its G2P
chain ends in letter spelling - coqui-en-ljspeech read every word
spelled out ('a r e w e ...', 2.9s vs 1.3s for the same sentence).
Verified against the shipped DLL: with lang=en + misaki the voice
speaks normally.
- coqui-en-* now derives lang like piper-* (id carries the locale)
- general fallback for families whose ids carry no language (matcha,
...): read the promoted token's Attributes\\Locale (written from the
catalog at promotion time) and use its primary subtag
- without a derivable language floravox letter-spells silently, so this
must be derivable for every routed voice
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #15. All five stages from the issue's suggested order, one commit each. Includes wrapper-side work in AACTools/rust-tts-wrapper (main + tag v0.5.3).
What changed
1. CI/DLL hygiene (
883bbb5)vg_ssml_debug.txtdump fromSpeakSsmlscripts/Get-RustTtsWrapperDll.ps1resolves the DLL from the csproj pin (never "whatever is cached"); CI and local release builds use it. CI now also ships the win-x86 DLL (previously x64-only) and hard-fails when the pinned DLL is missingscripts/test-rust-dll-pin.ps1— unit tests for the resolver incl. floating-version rejection2. ABI canary (
763f363)RustTtsWrapper.Bindings0.3.20 → 0.5.3 (combined sherpaonnx + floravox-lexicons DLL)tts_set_on_mark— the symbol only exists in DLLs with the consolidated 7-arg boundary callback, so the 0.3.x register-garbage hazard can't loadSetOnMarkplumbing; fixed the stale 6-argOnBoundaryThunkdeclarationscripts/test-rust-abi.ps1— export-table check with a negative control against a cached 0.3.x DLL3. Wrapper side (rust-tts-wrapper
6c8fb21+d1b3489)test-windowsCI job covers the exact NuGet feature combohashFiles()in a job-levelifinvalidated the file, so no tag since v0.5.0 ever published (the root cause behind "NuGet stops at 0.5.0"). Tag v0.5.3 is now publishing.4. Floravox routing (
03e3629)charOffset < 0emits audio-time-only events instead of being dropped<prosody>/<break>, bookmarks emitted as<mark name=.../>, marks → realSPEI_TTSBOOKMARK(retiring the simulated ones), offsets mapped back to SAPI positionslang(published G2P bundle, cached, graceful offline) + misaki us/gb for English5. Sidecar generation (
735a857)PiperSidecarGenerator: builds*.onnx.jsonfor sherpa-layout voices (MMS/coqui) from tokens.txt — casefolded phoneme maps,language.codederivation, catalog/MMS-config sample rate; marker ensures shipped/patched sidecars are never overwritten, stale re-run per SPD rules, atomic writesTests
VoiceGarden.UI.Tests(xunit, 18 tests): floravox integration through the shipped 0.5.3 DLL (SSML + boundaries + marks, OOV lexicon G2P, missing-voice error surfacing) and 15 sidecar-generator unit testsNotes for review
estimated=false) boundaries require duration-patched voices (floravox'sadd_durations_output.py); unpatched voices get audio-length-scaled estimates — same shape as today but through the SSML/mark path