Improve core audio resiliency

[grohith327]

Description

So this is a huge PR, it touches upon a few things but the core problem this PR attempts to solve is to improve the resiliency of the core audio transcription pipeline. AVAudioEngine has been very problematic and CoreAudio is currently experimental, not well tested out but much faster. The changes part of this PR are highlighted below including the reasoning

• Enables the core audio ASR processing flow as the default one - AVAudioEngine has been causing a lot of headaches - [BUG] FluidVoice 1.6.5 hangs when switching between a preferred microphone and AirPods #709, [BUG] recent update stopped it working #680 etc. CoreAudio is also the one we want to move to eventually
• Remove the duration mismatch fallback - this logic itself was flawed in a sense that it starts the timer when the hotkey is pressed, there has been cases where the MacOS is binding the new bluetooth input device (300-400 ms) and such cases we have no control. So the mismatch logic incorrectly assumes that the audio speed up happened and toggles it back to AVAudioEngine
• Now, the fallback itself was added because the audio was sped up certain times. I could not reproduce this but it should be fixed, what was likely happening was that the when CoreAudio was sharing the channels and sample rate, when we resampled to 16 kHZ we did not respect those values. E.g. Audio starts at 48 kHz and switches to 24 for some reason, this will come through from CA but when we downsample - we didn't respect it. Resulting in the audio "sped up"
• AVAudioEngine fallback is exercised only if the core audio setup fails, so the toggle is the source of truth
• Core audio lifecycle is managed off main thread, therefore during binding/draining you don't see a beach ball on the app
• Hardened the Core audio integration to be more resilient to faster device switches, or drop in device etc.

Type of Change

• 🐞 Bug fix
• ✨ New feature
• 💥 Breaking change
• 🧹 Chore
• 📝 Documentation update

Related Issue or Discussion

#709, #680

Testing

• Tested on Apple Silicon Mac
• Tested on macOS version: 27
• Ran linter locally: swiftlint --strict --config .swiftlint.yml Sources

Test cases

Tested out a few different use-cases across different audio input devices, constant changes, removal/additions of devices (mainly bluetooth) in order ensure that the transcription works

Screenshots / Video

Notes

Add reviewer context, rollout notes, or known tradeoffs here.

[greptile-apps]

Greptile Summary

This PR makes Direct Core Audio the default capture path and hardens its startup, readiness, device-change, and teardown lifecycle.

• Adds first-PCM readiness gating and bounded same-backend startup retries.
• Detects Core Audio format changes and serializes route recovery off the main thread.
• Updates packet publication, resampling, device observation, shutdown, and diagnostic logging.
• Preserves AVAudioEngine as the explicitly selected path when faster recording is disabled.
• Adds integration coverage for capture readiness, lifecycle serialization, teardown failures, and persisted preferences.

Confidence Score: 5/5

The PR appears safe to merge with no eligible or outstanding failure identified.

No blocking failure remains; the previously reported absence of an automatic AVAudioEngine fallback matches the maintainer-confirmed intent to retry the primary Core Audio pipeline.

_{Reviews (2): Last reviewed commit: "big update" | Re-trigger Greptile}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 250f8bfbbe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep accepting timestamped packets until the IOProc stops

When stop races the next hardware callback, closing packetGateOpen before AudioDeviceStop causes that callback to return without publishing even if its timestamped buffer contains frames acquired before the stop boundary. ASRService.stop() explicitly marks a host-time boundary and expects the synchronously drained final packet to be trimmed to it, so this can cut the last hardware buffer—and potentially the final phoneme—from recordings. Keep publication open until AudioDeviceStop has synchronized with the IOProc, then let the pipeline discard frames after the recorded boundary.

Useful? React with 👍 / 👎.

[github-actions]

FluidVoice PR build ready

Download FluidVoice-PR-732-250f8bfbbecc

The artifact contains the ad-hoc-signed app ZIP, Xcode archive, build manifest, and installation instructions. It expires 5 days after the build.

Install the app

1. Extract the downloaded artifact, then extract FluidVoice-PR-732.app.zip.

2. Move FluidVoice Improve core audio resiliency #732.app into the /Applications folder.

3. Open Terminal and remove the download quarantine marker:

   xattr -dr com.apple.quarantine "/Applications/FluidVoice #732.app"
   

4. In Applications, Control-click FluidVoice Improve core audio resiliency #732.app and choose Open.

5. If macOS still blocks it, open System Settings → Privacy & Security, click Open Anyway, and confirm.

This build has its own app identity, so its permissions are separate from the release version of FluidVoice.

View workflow run

[grohith327]

@greptileai re-run review with my updated responses