Repository navigation
feat: transcribe while recording on the Intel GPU (#139) - #140
Open
raphaelm22 wants to merge 4 commits into
Open
raphaelm22 wants to merge 4 commits into
raphaelm22 wants to merge 4 commits into
Conversation
…cure#131) faster-whisper (CTranslate2) only runs on the CPU or NVIDIA GPUs. A new "Intel GPU (OpenVINO)" compute device runs Whisper through OpenVINO GenAI's WhisperPipeline on Intel's int8 model builds. On a Core Ultra 7 265U, large-v3-turbo transcribed a 7.4 min meeting in ~45 s instead of 7.7 min on the CPU. - openvino_whisper.py (no Qt): model repos, pipeline cache with a compiled-model cache, faster-whisper's Silero VAD (WhisperPipeline has none and hallucinates over silence), 60 s speech batches as the only progress/cancel points, language detected once then pinned. - TranscriptionWorker._run_openvino: any OpenVINO failure, at load or mid-run, falls back to the CPU; unreadable audio still errors. - Settings: new device entry with inline status; openvino-genai is installed on demand, asked before any config write since Config.set() saves immediately. - System Status: Intel GPU row when selected, and the Whisper model check looks for the OpenVINO build. - openvino extra in pyproject (uv.lock: only the four new packages), commented line in requirements.txt, module in the CI import smoke.
) Opt-in (transcription.live, intel_gpu only). A LiveTranscriptionWorker reads the track files ChunkWriter streams to disk, mixes them as DualAudioCapture.stop does, and transcribes speech once it has settled. At Stop it finishes from the final combined file and takes the recording's job slot with TranscriptionWorker's signals; any failure falls back to the normal transcription. A replayed 7.4-minute meeting had its transcript 7.8 s after Stop instead of 60 s. No capture-path changes.
Author
|
Ready for review (stacked on #132). Used daily since 15/10 on a Core Ultra 7 265U (Intel GPU selected, live transcription on): 7 real Teams meetings, 11 to 115 min, pt-BR, 4 to 6 speakers. No fallbacks or errors in the log. The note in the description about a real call is now answered: no audio or video stutter in Teams, and the text keeps up during the meeting (batches every ~55 s). After Stop the transcript appears in ~5 s on a 40 min meeting. |
raphaelm22
marked this pull request as ready for review
October 22, 2026 13:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #139.
Summary
An opt-in setting, Settings → Transcription → Transcribe while recording (Intel GPU), transcribes each recording as it is recorded. At Stop only the last stretch of speech is left, so the transcript is ready in seconds instead of minutes.
No capture-path changes, which was the main risk raised in #78. The worker reads the track files
ChunkWriteralready streams to disk; audio callbacks and writers are untouched.app/transcription/live_transcriber.py(new)TrackFollowerreads the frames appended to a WAV that is still being written, up to the frame count in its last flushed header.LiveMixermixes mic(s) and system the wayDualAudioCapture.stopbuildscombined_audio.wav: by sample index, at 0.5/0.5. It reads only as far as every track has data, because a track that is behind may still catch up.LiveSession, plain logic with no Qt:LiveTranscriptionWorker(QThread) has the same signals asTranscriptionWorker, plusfallback.openvino_whisperdecode_batchis factored out oftranscribe()._GENERATE_LOCKserializesgenerate()on the shared pipeline, because a live recording and the queue's job can both be transcribing.MainWindowliveon,intel_gpuselected and auto-transcribe on.RecordingState.STOPPINGmakes the workers stop reading before the writers close and the mix runs._take_live_workerandfinish()hand it the job slot as_transcription_worker, so the busy check, cancel, diarization and the session binding all work as before.finish()and a worker failure share a lock, so a failure afterfinish()was accepted emitsfallback, and a normalTranscriptionWorkerstarts in the same slot._shutdown_workerscancels and waits for live workers.Settings and config
A checkbox, enabled only for the Intel GPU device.
transcription.livedefaults toFalse.Measured (Core Ultra 7 265U, large-v3-turbo)
Through the real
MainWindow, headless. The recorder's capture was replaced by realChunkWriters fed from a 16-minute meeting at 3x, and Stop was driven asRecorder.stop_recordingdoes:There were no dialogs and the app closed cleanly. The voice-profile scores were the same as in the original CPU run.
Replay of a 7.4-minute meeting:
Text agreement with the normal transcript was 0.886 word-sequence similarity, 1,082 vs 1,099 words. This is within Whisper's own variance on the same recording:
The residual comes from the live level following the running peak; the final mix's global peak is only known at Stop.
Not yet tested: a real Teams call. The question is whether GPU use during the call (about 7 s of generate per ~60 s of speech) affects call video. That is why the setting is opt-in, and why this is a draft.
Tests
43 new tests:
test_live_transcriber.py(24): reading a growing WAV, mixing up to the shortest track, settle window, short-batch hold, silence skipping, recording-time offsets, running-peak level vs. a lone track left as is, language pinning, stopping between batches, finish from the final file, and the worker's finish/failure/fallback/cancel handshake.test_live_transcription_window.py(19): start conditions, track paths from the capture writers, STOPPING, discard, queue hand-off, refused finish, fallback, shutdown, and recordings that won't be transcribed.Without the
MainWindowchange, all 19 window tests fail. Also added adecode_batchlock test and an offscreen Settings check (default off, enabled only for the Intel GPU device, saved). Full suite: 501 passed.🤖 Generated with Claude Code