[vagxrth]

MP4 exports encode their audio track with FFmpeg's built-in AAC encoder (aac, two-loop search). On macOS that encoder is most of the CPU work an export does: in a 60 s export it retires ~56G of the export process's ~76G instructions, on a thread (export-audio) that sampling shows inside avcodec_send_frame ~99% of the time.

This PR encodes export audio with AudioToolbox's AAC encoder (aac_at, FFmpeg's wrapper, already in the bundled native-deps) in average-bitrate mode. If AudioToolbox fails to open, the export falls back to FFmpeg's encoder exactly as before. Other platforms are unchanged. Studio recordings moved to AudioToolbox separately in #2626; this PR reuses that encoder setup for exports.

Results

Release build of the GPUI workspace (codegen-units = 1, thin LTO), Apple M2 (8 cores, 8 GB), macOS 27.0. "Before" is main as of Oct 15, "after" is this branch. The export-benchmark example added here calls ExporterBase::build_for_mp4 + Mp4ExportSettings::export_prepared the way the GPUI export page does. Before and after binaries ran alternately; instructions and cycles come from /usr/bin/time -l. Medians of 7.

Cap's reference recording (reference-recording.cap, 30.8 s, two segments, mic + system audio):

Export	Instructions	CPU cycles	Wall, idle	Wall, 4 busy background processes
1080p30 (GPUI default fps)	37.0G → 15.1G (−59%)	11.8G → 6.2G (−47%)	4.48 s → 4.39 s	5.66 s → 5.16 s (−9%)
720p30	36.5G → 14.5G (−60%)	11.0G → 6.8G (−38%)	2.88 s → 2.95 s	5.52 s → 4.31 s (−22%)

60 s Studio recording (3024×1964 @ 60 fps display from VideoToolbox at Cap's studio bitrate, 720p camera, 48 kHz mono mic with a minute of speech, 60 Hz cursor, default editor config):

Export	Instructions	Wall, 4 busy background processes
1080p30	76.2G → 23.5G (−69%)	9.89 s → 8.52 s (−14%)
720p30	75.1G → 22.2G (−70%)	11.25 s → 7.94 s (−29%)

The same project exported without a mic track retires 19.3G instructions at 720p30, so the audio path itself goes from ~56G to ~3.3G instructions (~17× less).

On an idle machine wall time doesn't move: MP4 export on an M2 is bound by the hardware H.264 encoder (VideoToolbox runs at ~190 fps at 1080p, two or three sessions in parallel share that rate, and exporting the project without any audio is no faster). The gain shows up when the Mac is doing anything else, because the export no longer needs a core to itself for its audio. The "busy" columns run four yes processes for the whole benchmark.

Encoder alone (FFmpeg CLI, 60 s of 48 kHz stereo at 320 kbps): FFmpeg's aac takes 3.35 s on speech, 3.00 s on music-like content; AudioToolbox ABR takes 0.43 s and 0.42 s (7–8×).

Re-checked after merging current main (merge bca5f2a, reference recording, medians of 7, alternating). "Before" is this branch with the exporter calling AACEncoder::init again, which is main's export path:

Export	Instructions	Wall, other desktop apps busy	Wall, + 4 busy background processes
1080p30	35.1G → 13.0G (−63%)	5.03 s → 4.29 s (−15%)	6.87 s → 5.31 s (−23%)
720p30	34.3G → 12.4G (−64%)	4.17 s → 3.05 s (−27%)	6.26 s → 4.71 s (−25%)

Other desktop apps were using about 1.5 cores when this run started, so its first wall column is a light-load number, not an idle one (the clean idle numbers are in the first table, where wall time didn't move).

Output

• A/V sync is unchanged. A project whose video flashes white on exactly one frame at 2.000 s and whose mic clicks at 2.000 s exports with the flash on frame 120 (60 fps) / 60 (30 fps) and the click on sample 96000, before and after: 0.00 ms offset in both.
• Re-checked after merging current main: the sync project still exports with a 0.00 ms offset at 60 and 30 fps, before and after, with both streams exactly 5.000 s long. The export's first audio packet is at −2112 (AudioToolbox priming, hidden by the edit list, no leading trim) and the track ends on sample 240,000 in both builds.
• Durations are unchanged. Container and video durations are identical, and the audio track ends on the same sample before and after (1478656 + 624 vs 1478592 + 688 = 1479280 on the reference recording; exactly 5.000 s on the sync project). The audio edit list's millisecond field can differ by 1 ms (30.818 vs 30.819 s) because the two encoders' priming lengths round differently.
• Audio fidelity improves. Signal-to-distortion against the source at the same 320 kbps setting: speech 46.7 → 56.5 dB, music-like content 38.3 → 46.1 dB.
• Size. Silence still costs ~3 kbps. Voice and music now get the 320 kbps the encoder has always been configured for (FFmpeg's encoder undershoots to ~160–245 kbps): +78 kbps on the reference recording, ~0.6 MB per minute, +1.2% on its 1080p30 export. Average-bitrate mode matters here: AudioToolbox's default constant bitrate would also spend 320 kbps on silence, and a new test fails if it ever does.
• Video is untouched. VideoToolbox does not produce byte-identical video run to run, on main either; the audio track is deterministic in both builds.

Priming: why exports don't trim like recordings

AudioToolbox primes 2112 samples, FFmpeg's encoder 1024. A plain MP4 hides priming with its edit list; a fragmented MP4 keeps it in the timeline. Decoding a click encoded at exactly 1.000 s:

Container	FFmpeg aac	AudioToolbox aac_at
Plain MP4 (export)	+0 samples	+0 samples
Fragmented MP4 (Studio mic / system audio)	+1024 (21.3 ms)	+2112 (44.0 ms)

#2626 handles the fragmented row by dropping the first 1088 input samples of each recording track, so content lands where FFmpeg's encoder put it. An export is a plain MP4: its edit list already skips all 2112 priming samples, so it needs no trim and keeps every input sample. init_with_codec therefore takes a Priming argument: recordings pass TrimToFfmpegAac (their behaviour is unchanged), exports and AACEncoder::init pass Keep. With the recording trim an export would lose its first 22.7 ms of audio; the export test has a click at sample 240 that fails in that case.

Changes

• AACEncoder::init_for_export: on macOS opens aac_at with aac_at_mode=abr at the exporter's existing 320 kbps through init_with_codec, and falls back to aac; elsewhere it is init.
• init_with_codec takes a Priming argument (above).
• crates/export/src/mp4.rs: the MP4 exporter calls init_for_export.
• apps/desktop-gpui/examples/export-benchmark.rs: exports a project like the GPUI export page and prints wall time, frames per second and process CPU time.
• The end-of-track trim this PR first added (EndTrimmedSink: AudioToolbox only takes full frames, and libavcodec's padding of the last one made the track up to a frame longer than its input) and the codec-driven sample format reached main through perf(recording): encode macOS studio audio with AudioToolbox AAC #2626 under the same names, so they are no longer part of this diff.

Checks

• New tests: the export encoder keeps clicks at samples 240 and 24,000 on their input samples and ends the track on the last input sample; 10 s of silence stays under 40 KB (401,851 bytes in constant-bitrate mode); on macOS the export encoder is aac_at; benchmark_export_aac_encoder (ignored) compares the two encoders.
• cargo test -p cap-enc-ffmpeg: 185 passed (macOS), including perf(recording): encode macOS studio audio with AudioToolbox AAC #2626's recording tests.
• cargo clippy -p cap-enc-ffmpeg -p cap-export --all-targets -- -D warnings and cargo clippy --example export-benchmark -- -D warnings (GPUI workspace) are clean; cargo fmt --all is clean.

Reproduce

cd apps/desktop-gpui
# use a disposable copy of a project, e.g. reference-recording.cap from cap-performance-fixtures
cargo run --release --example export-benchmark -- /path/to/copy.cap 1920 1080 30 5
# wrap it in /usr/bin/time -l for instructions and cycles; run `yes > /dev/null &` x4 alongside for the busy columns

# encoder throughput, from the repo root
cargo test -p cap-enc-ffmpeg --lib benchmark_export_aac_encoder -- --ignored --nocapture

Confidence Score: 5/5

The PR appears safe to merge; no new actionable issues were found.

Summary

MP4 exports now try AudioToolbox AAC on macOS with average bitrate, then fall back to FFmpeg AAC if opening fails.

• MP4 exports use AudioToolbox AAC on macOS when it is available.
• The GPUI example measures full MP4 export runs.

The full PR diff was reviewed because the previous and current commits have different merge bases with the target commit. Tests were not run during this review.

_{Reviews (4) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile}

[vercel]

@vagxrth is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

[richiemcilroy]

Thanks @vagxrth! Greptile is at 4/5 here. We only merge once Greptile reaches 5/5, so please address its findings in the summary above (or reply to them if you think one is wrong), then comment @greptileai review to re-run it. Once it is 5/5 we will run the full verification suite on it.

[vagxrth]

@greptileai review

[vagxrth]

@greptileai review

[vagxrth]

@greptileai review

[vagxrth]

@richiemcilroy Greptile is at 5/5 on the latest commit now.

• Fixed both findings: each benchmark process writes to its own temp dir, and the narrating comment is gone.
• Merged main to resolve the conflict with perf(recording): encode macOS studio audio with AudioToolbox AAC #2626. Exports now reuse its AudioToolbox setup, but without the recording leading trim: a plain MP4's edit list already hides the encoder priming, so the trim would cut the first ~23 ms of every export. A new test click at sample 240 catches that.
• Re-ran everything on current main: A/V sync is still 0.00 ms at 60 and 30 fps, instructions are down 63% / 64% (1080p30 / 720p30), and wall time is down 23% / 25% with background load. The numbers are in the description.

Ready for the verification suite whenever you are. Thanks!