Skip to content

feat: transcribe while recording on the Intel GPU (#139) - #140

Open
raphaelm22 wants to merge 4 commits into
ObscureAintSecure:masterfrom
raphaelm22:feat/live-transcription
Open

raphaelm22 wants to merge 4 commits into
ObscureAintSecure:masterfrom
raphaelm22:feat/live-transcription

Conversation

@raphaelm22

Copy link
Copy Markdown

Closes #139.

Stacked on #132 (Intel GPU transcription). The branch contains it, so the diff against master includes it. This PR's own changes are the last two commits: d92fdae and a4d0c29. Merge after #132.

Summary

An opt-in setting, Settings → Transcription → Transcribe while recording (Intel GPU), transcribes each recording as it is recorded. At Stop only the last stretch of speech is left, so the transcript is ready in seconds instead of minutes.

No capture-path changes, which was the main risk raised in #78. The worker reads the track files ChunkWriter already streams to disk; audio callbacks and writers are untouched.

app/transcription/live_transcriber.py (new)

  • TrackFollower reads the frames appended to a WAV that is still being written, up to the frame count in its last flushed header.
  • LiveMixer mixes mic(s) and system the way DualAudioCapture.stop builds combined_audio.wav: by sample index, at 0.5/0.5. It reads only as far as every track has data, because a track that is behind may still catch up.
  • LiveSession, plain logic with no Qt:
    • transcribes speech regions that ended at least 3 s before the write head, so a sentence is never cut;
    • waits until a batch holds 30 s of speech, for context, and skips silence without re-scanning it;
    • pins the language after the first batch;
    • at Stop, transcribes the rest from the final combined file (WAV or MP3), as is.
  • LiveTranscriptionWorker (QThread) has the same signals as TranscriptionWorker, plus fallback.

openvino_whisper

decode_batch is factored out of transcribe(). _GENERATE_LOCK serializes generate() on the shared pipeline, because a live recording and the queue's job can both be transcribing.

MainWindow

  • One worker per recording directory, started when recording starts. It needs live on, intel_gpu selected and auto-transcribe on.
  • Stop: RecordingState.STOPPING makes the workers stop reading before the writers close and the mix runs.
  • Discarded or not auto-transcribed recording: the worker is cancelled.
  • The job, not the recording, takes the worker. The queue rules are unchanged. When the recording's job comes up, _take_live_worker and finish() hand it the job slot as _transcription_worker, so the busy check, cancel, diarization and the session binding all work as before.
  • Fallback: a worker that failed is dropped and the recording is transcribed normally. finish() and a worker failure share a lock, so a failure after finish() was accepted emits fallback, and a normal TranscriptionWorker starts in the same slot.
  • Exit: _shutdown_workers cancels and waits for live workers.

Settings and config

A checkbox, enabled only for the Intel GPU device. transcription.live defaults to False.

Measured (Core Ultra 7 265U, large-v3-turbo)

Through the real MainWindow, headless. The recorder's capture was replaced by real ChunkWriters fed from a 16-minute meeting at 3x, and Stop was driven as Recorder.stop_recording does:

After Stop Before (CPU) With this
Transcript shown 39.6 min 4.5 s
Speakers identified and voice profiles matched (with #135) 39.6 min 1.6 min

There were no dialogs and the app closed cleanly. The voice-profile scores were the same as in the original CPU run.

Replay of a 7.4-minute meeting:

Time after Stop
Live: 386 s of 446 s already transcribed at Stop 7.8 s
Normal post-stop transcription 60 s

Text agreement with the normal transcript was 0.886 word-sequence similarity, 1,082 vs 1,099 words. This is within Whisper's own variance on the same recording:

Same file, transcribed differently Similarity to the normal transcript
Normal path, 30 s batches instead of 60 s 0.844
Live logic fed the final file 0.971
Raw mix at exactly the final gain 0.917

The residual comes from the live level following the running peak; the final mix's global peak is only known at Stop.

Not yet tested: a real Teams call. The question is whether GPU use during the call (about 7 s of generate per ~60 s of speech) affects call video. That is why the setting is opt-in, and why this is a draft.

Tests

43 new tests:

  • test_live_transcriber.py (24): reading a growing WAV, mixing up to the shortest track, settle window, short-batch hold, silence skipping, recording-time offsets, running-peak level vs. a lone track left as is, language pinning, stopping between batches, finish from the final file, and the worker's finish/failure/fallback/cancel handshake.
  • test_live_transcription_window.py (19): start conditions, track paths from the capture writers, STOPPING, discard, queue hand-off, refused finish, fallback, shutdown, and recordings that won't be transcribed.

Without the MainWindow change, all 19 window tests fail. Also added a decode_batch lock test and an offscreen Settings check (default off, enabled only for the Intel GPU device, saved). Full suite: 501 passed.

🤖 Generated with Claude Code

…cure#131)

faster-whisper (CTranslate2) only runs on the CPU or NVIDIA GPUs. A new
"Intel GPU (OpenVINO)" compute device runs Whisper through OpenVINO
GenAI's WhisperPipeline on Intel's int8 model builds. On a Core Ultra 7
265U, large-v3-turbo transcribed a 7.4 min meeting in ~45 s instead of
7.7 min on the CPU.

- openvino_whisper.py (no Qt): model repos, pipeline cache with a
  compiled-model cache, faster-whisper's Silero VAD (WhisperPipeline has
  none and hallucinates over silence), 60 s speech batches as the only
  progress/cancel points, language detected once then pinned.
- TranscriptionWorker._run_openvino: any OpenVINO failure, at load or
  mid-run, falls back to the CPU; unreadable audio still errors.
- Settings: new device entry with inline status; openvino-genai is
  installed on demand, asked before any config write since
  Config.set() saves immediately.
- System Status: Intel GPU row when selected, and the Whisper model
  check looks for the OpenVINO build.
- openvino extra in pyproject (uv.lock: only the four new packages),
  commented line in requirements.txt, module in the CI import smoke.
)

Opt-in (transcription.live, intel_gpu only). A LiveTranscriptionWorker reads the track files ChunkWriter streams to disk, mixes them as DualAudioCapture.stop does, and transcribes speech once it has settled. At Stop it finishes from the final combined file and takes the recording's job slot with TranscriptionWorker's signals; any failure falls back to the normal transcription.

A replayed 7.4-minute meeting had its transcript 7.8 s after Stop instead of 60 s. No capture-path changes.
@raphaelm22

Copy link
Copy Markdown
Author

Ready for review (stacked on #132). Used daily since 15/10 on a Core Ultra 7 265U (Intel GPU selected, live transcription on): 7 real Teams meetings, 11 to 115 min, pt-BR, 4 to 6 speakers. No fallbacks or errors in the log. The note in the description about a real call is now answered: no audio or video stutter in Teams, and the text keeps up during the meeting (batches every ~55 s). After Stop the transcript appears in ~5 s on a 40 min meeting.

@raphaelm22
raphaelm22 marked this pull request as ready for review October 22, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Transcribe while recording (opt-in, Intel GPU): transcript ready seconds after Stop

1 participant