[raphaelm22]

Closes #135.

Stacked on #132 (Intel GPU transcription) and #136 (per-chunk embeddings). This branch contains both, so the diff against master includes them. This PR's own changes are the last two commits: f1e429a and 7eddc9d. Review those, and merge after the two base PRs.

Summary

With the Intel GPU device selected, transcription runs on the GPU (#132) but pyannote still ran entirely on the CPU, and was nearly all of the wait after Stop. This moves the expensive part of diarization, the WeSpeaker ResNet34 trunk, to the GPU through OpenVINO.

• app/transcription/openvino_embeddings.py (new):
  • get_trunk converts the trunk from the loaded torch model once. pyannote's weights are gated, so no prebuilt model can ship.
  • The converted model is saved to ~/.talktrack/openvino_cache/wespeaker-trunk-<weights hash>.xml, so a pyannote model update converts again rather than reusing a stale file.
  • It is compiled for the GPU with a static shape and cached between recordings, like the Whisper pipeline.
  • GpuTrunk.embed keeps fbank and the per-speaker statistics pooling in torch on the CPU and runs only the trunk on the GPU.
• chunk_embeddings.set_accelerator. The per-chunk pass from perf: one speaker-embedding pass per chunk, not per speaker (#133) #136 can hand its network call to an accelerator. If that raises mid-run, the error is kept on accelerator.error and the remaining batches run on the CPU model.
• DiarizationWorker sets the accelerator on every run: the trunk for intel_gpu, None for everything else. The cpu and intel_gpu settings share the cached (token, "cpu") pipeline, so a stale accelerator would otherwise keep using the GPU after switching back.
  • If the trunk can't be built (no OpenVINO, no GPU, conversion error), the run stays on the CPU and the progress line says why.
  • After a mid-run failure, the worker evicts the trunk and reports that it finished on the CPU.

Measurements (Core Ultra 7 265U)

Full 31-minute meeting through DiarizationWorker:

	Diarization
CPU, as shipped	34 min
GPU, first use (pipeline load + about 70 s of conversion)	4.0 min
GPU, afterwards	2.2 min

All 709 transcript segments got the same speaker label as the original CPU run, under the same 8 labels.

5-minute excerpt: 267 s (pyannote as is) → 18.5 s, DER 0.0002 against the original, speaker centroids at cosine ≥ 0.99999. Saved voice profiles (#128) stay comparable.

Trunk alone per batch of 32 chunks:

Device	Time	Embedding cosine vs CPU
CPU, torch	8.4–11.3 s	—
GPU	1.7 s	0.999906
NPU	1.1 s	0.999991

The NPU is not offered. It was no faster end to end, compiles in 30 s instead of 5, and is the device where large-v3-turbo hung in #132.

Tests

tests/test_openvino_embeddings.py, 17 new tests, with torch and OpenVINO stubbed. They cover:

• short batches padded to the compiled shape;
• a wrong shape raising rather than being guessed;
• embed equal to the CPU model;
• first use converting and then compiling for GPU with CACHE_DIR;
• a saved model reused without converting;
• the cache kept until release_trunks;
• the weights hash following the weights;
• a missing OpenVINO raising;
• the accelerator doing the pass, falling back to the CPU on failure, and clearing;
• _intel_gpu_trunk building the trunk, explaining an unavailable GPU, and skipping unsupported models;
• run() attaching, clearing and evicting the trunk.

Without the integration code, 9 fail. Full suite on this branch: 487 passed.

🤖 Generated with Claude Code

[raphaelm22]

Ready for review (stacked on #132 and #136). Used daily since 15/10 on a Core Ultra 7 265U (Intel GPU selected, live transcription on): 7 real Teams meetings, 11 to 115 min, pt-BR, 4 to 6 speakers. No fallbacks or errors in the log. Speaker names after Stop: 11 min meeting 57 s, 40 min 2.1 min, 50 min 2.5 min, 108 min 7.4 min. Before #136/#138 a 31 min meeting took 39 min.