Skip to content

Better multilingual speech support - #3

Draft
jajandio wants to merge 2 commits into
benryanx:masterfrom
jajandio:codex/better-multilingual-support-clean
Draft

Better multilingual speech support#3
jajandio wants to merge 2 commits into
benryanx:masterfrom
jajandio:codex/better-multilingual-support-clean

Conversation

@jajandio

@jajandio jajandio commented Apr 13, 2026

Copy link
Copy Markdown

Summary

This PR improves Navi's multilingual speech behavior for bilingual and non-English usage.

What changed

  • added a language selector in settings with Auto plus the supported Whisper/TTS languages
  • switched local TTS from Chromium speechSynthesis to native Windows System.Speech voice selection
  • routed speech recognition through the English-only Whisper model for English and a stronger multilingual Whisper model for Auto and non-English languages
  • upgraded the multilingual transcription default from Xenova/whisper-base to Xenova/whisper-small for better non-English recognition
  • logs ElevenLabs API failures before falling back to local TTS, so provider issues are visible during debugging
  • nudged Navi's guidance prompt so labels, explanations, and steps match the detected or selected language
  • added focused tests for language routing helpers and exposed them through npm test

Why

Before this change, local TTS often inherited the wrong browser or system voice and could read English with a Spanish accent. Speech recognition also behaved inconsistently for bilingual usage because it was effectively biased toward the English-only transcription path, while the smaller multilingual model struggled more with non-English accents and short utterances.

Validation

  • npm test � 6 tests passing
  • prior manual local verification on Windows that English uses an English voice and Spanish uses a Spanish voice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants