Fused NeMo transcribe diarize actor#43
Conversation
Performance ResultsThese are the measurements captured while developing this PR. The smoke result is from the current pushed commit ( Current fused actor smokeCommand: MODAL_PROFILE=everett-38139 uv run --extra models modal run pipelines/transcribe_diarize/modal_app.py::measure \
--source '/audio/flac/Build_Scalable_Batch_Inference_Pipelines_in_3_Lines_Daft_GPT_vLLM_0.00_2.38.flac' \
--config parakeet+sortformerResult:
This smoke is mainly a runtime/API validation because the clip is too small for stable throughput conclusions. 2-hour L40S scale runArtifact:
The old linear path did not produce a comparable throughput number:
Artifact: Real-file processing runsIMG_3800:
Artifacts:
Zoom batch:
Artifact: MarbleNet VAD threshold benchArtifact: On the 155.64s sample, threshold
Threshold Profiler smokeArtifact:
|
everettVT
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 426ea33dc1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if not active: | ||
| return [] | ||
| try: | ||
| raw = self.sortformer.diarize(audio=[path for path, _window in active]) |
There was a problem hiding this comment.
When MarbleNet produces more than one window for a recording, paths contains separate VAD chunks from that same file, but this submits them to Sortformer as independent audio inputs. Sortformer speaker labels are local to each input, so offsetting timestamps later does not reconcile that speaker_0 in chunk 1 may be a different person than speaker_0 in chunk 2; the default parakeet+sortformer path now uses MarbleNet VAD, so multi-turn/multi-speaker files can get corrupted speaker assignments. Run diarization on the original recording or add a reliable cross-window speaker mapping before merging.
Useful? React with 👍 / 👎.
| asr_kwargs=kwargs, | ||
| include_vad_stats=True, | ||
| ) | ||
| return df.collect().to_pydict() |
There was a problem hiding this comment.
When bench_vad --limit is used, this collect() still materializes the entire source, and each threshold does the same before filter_paths narrows the result. For large folders the advertised smoke-test limit still transcribes all audio for the baseline and every threshold, which can make the experiment much slower and more expensive than requested; apply df.limit(limit) inside collect() before calling collect().
Useful? React with 👍 / 👎.
everettVT
force-pushed
the
everettVT/clean-up-dakar-models
branch
from
426ea33 to
ace200c
Compare
everettVT
force-pushed
the
everettVT/clean-up-dakar-models
branch
from
ace200c to
ca8f335
Compare
1 participant
Summary
TranscribeDiarizeVadDaft class that loads Parakeet, MarbleNet VAD, and Sortformer directly in__init__and runs decode, VAD, ASR, diarization, speaker merge, and VAD stats in oneprocessmethod.batch_size=1calls from the NeMo ASR/diarization path.Why
The previous implementation kept model weights resident but still crossed Daft graph/host boundaries between VAD, ASR, and diarization. This PR makes the Daft actor itself the residency boundary and keeps model inference orchestration inside one device-resident class.
Validation
uv run --extra models python -m py_compile models/common/speech.py models/common/vad.py models/parakeet/model.py models/sortformer/model.py pipelines/transcribe_diarize/pipeline.py pipelines/transcribe_diarize/modal_app.py pipelines/transcribe_diarize/benchmark.pyuv run --extra lint ruff check models/common/speech.py models/common/vad.py models/parakeet/model.py models/sortformer/model.py pipelines/transcribe_diarize/pipeline.py pipelines/transcribe_diarize/modal_app.py pipelines/transcribe_diarize/benchmark.pyMODAL_PROFILE=everett-38139 uv run --extra models modal run pipelines/transcribe_diarize/modal_app.py::measure --source '/audio/flac/Build_Scalable_Batch_Inference_Pipelines_in_3_Lines_Daft_GPT_vLLM_0.00_2.38.flac' --config parakeet+sortformer