perf(bench): ROCm support in the benchmark harness and a published gfx1151 baseline #1810

Part of #1801. Phase 3. Depends on #1802, #1805. (Both dependencies are closed: #1802 landed in PR #1818 and #1805 in PR #1883, so this issue is unblocked.)

Context

scripts/bench_decode.sh runs on Linux, but on an AMD host it mislabels and misjudges. Every point below was re-verified on origin/main d8d34e2b on 2026-09-29 and is still true; grep -c rocm scripts/bench_decode.sh returns 0.

• detect_backend (scripts/bench_decode.sh:297-305) returns cuda only when nvidia-smi works or generate --help mentions cuda, otherwise metal, so ROCm results would be written as benchmarks/metal_<hw>_<date>.csv.
• The hardware tag falls back to the first 20 characters of the /proc/cpuinfo model name (detect_hardware_full at :261 when nvidia-smi finds no GPU at :249, then the cut -c1-20 fallback of detect_hardware_short at :292).
• The out-of-memory guard uses 85% of host RAM (detect_memory_bytes at :315 reads free -b at :319, the 85% limit is :325), which is wrong on a UMA carve-out (host sees about 31 GiB, the GPU 96 GiB); the only knob is BENCH_MEM_OVERHEAD_FACTOR.
• scripts/compare_bench_csv.py:107-108 (in newer_readings_elsewhere) hardcodes hosts (m5max, m1ultra) and runtimes (pylm, metal), and :109-110 returns {} when the host is neither, so its superseded-baseline warning silently turns off for other hosts.
• The CSV mlx_commit column stores 8 characters (scripts/bench_decode.sh:208) and cannot tell upstream MLX from a build that carries the mlxcelverse ROCm overlay.

Spike numbers for orientation (mlx-lm on a build of the same mlxcelverse ROCm tree, pp512/tg128): Qwen3-0.6B-4bit 3,899/224, Llama-3.1-8B-Instruct-4bit 925/32.3, Qwen3-30B-A3B-4bit 295/58.7 tok/s. Decode-shaped q4 GEMV reached about 150 GB/s.

What #1805 now provides to the harness

PR #1883 added GpuVendor, GpuBackendKind and gpu_backend_kind() to src/lib/mlxcel-core/src/hardware.rs (gpu_backend_kind() at :206), plus device_name, device_architecture and device_memory_bytes on the hardware report. That is a reliable source for the backend, device and memory, in place of probing nvidia-smi. The harness is a shell script, so it can only reach them through CLI output. What mlxcel generate actually prints today:

• stdout, GPU: <device name> (Amd), <N> GiB device memory. (src/commands/generate.rs:163, printed when the model loads).
• stdout, HIP architecture gfx1151; compiled for [...] (src/commands/generate.rs:147-149, from rocm_arch_startup_summary), silent on Metal, CUDA and CPU-only builds.
• stdout, Runtime device: GPU (src/commands/generate.rs:95).
• stderr, only with MLXCEL_DEBUG_KERNEL_BACKEND=1: [mlxcel] custom kernel backend: rocm (src/lib/mlx-cpp/turbo/gpu_backend.cpp:73-79). This is the same resolution gpu_backend_kind() maps to an enum (the bridge's gpu_backend_kind at src/lib/mlxcel-core/cpp/mlx_cxx_bridge.cpp:5049 returns gpu_kernel_backend()).

No CLI subcommand prints gpu_backend_kind() on its own, and these lines appear only during a real generate run, not on --help. scripts/ci/rocm_smoke.sh:105-110 already parses the first, second and fourth of them and is the model to copy.

Scope

Make the harness produce correctly labeled, comparable ROCm results, and publish a first gfx1151 baseline.

Implementation plan

1. detect_backend: return rocm when the binary was built with the rocm feature (query generate --help or a version/diagnostics flag from feat(rocm): memory estimation, device info and hardware detection on AMD UMA hosts #1805) or rocminfo finds a GPU agent. Update: feat(rocm): memory estimation, device info and hardware detection on AMD UMA hosts #1805 did not add a standalone diagnostics flag. Use the [mlxcel] custom kernel backend: <kind> line from a short probe generate run with MLXCEL_DEBUG_KERNEL_BACKEND=1 (or from the first model's run) as the primary source, since it names exactly what gpu_backend_kind() reports for metal, cuda and rocm alike; rocminfo stays a fallback only. If adding a flag that prints gpu_backend_kind() without loading a model turns out simpler, that is an acceptable alternative, decided by whichever keeps the probe under a few seconds.
2. Hardware tag: recognize AMD GPUs (for example strixhalo-gfx1151) from the device info exposed in feat(rocm): memory estimation, device info and hardware detection on AMD UMA hosts #1805, now the GPU: ... (Amd) and HIP architecture gfx... lines above.
3. Memory guard: use device memory from feat(rocm): memory estimation, device info and hardware detection on AMD UMA hosts #1805 when the backend is ROCm, now the <N> GiB device memory figure on the GPU: line. Note that mlxcel generate also runs its own pre-load memory preflight, which since PR feat(rocm): add a GPU vendor concept and AMD device reporting #1883 reads the ROCm allocator's memory_limit() rather than host /proc/meminfo (src/execution/memory_estimate.rs:751-757).
4. Add a column or metadata line for the ROCm overlay's source fork commit (src/lib/mlx-cpp/patches-rocm/UPSTREAM, currently 75915908) and the ROCm/HIP version, and keep mlx_commit meaning the upstream pin.
5. Generalize host and runtime detection in compare_bench_csv.py.
6. Run ./scripts/bench_decode.sh all --cooldown 30 --big-cooldown 30 on the spike host for the models in test(rocm): correctness matrix against a Metal baseline and a verify-test-rocm gate #1809, plus the mlx-lm Python baseline built from the same mlxcelverse ROCm tree (scripts/bench_mlxlm.py), same day and same mlxcel commit. Publish the CSV in benchmarks/ and a page in docs/benchmark_results/ with hardware, OS, ROCm/HIP version, gfx target, mlxcel commit, MLX pin, mlxcelverse commit, checkpoint and quantization format, prompt/decode lengths, batch, warmup and flags.
7. Note in the page whether other GPU tenants were stopped during the run (the spike host also runs GPU services).

Acceptance criteria

• ROCm runs are written as benchmarks/rocm_<hw>_<date>.csv with correct backend and hardware tags.
• The memory guard does not skip models that fit in device memory on the UMA host.
• compare_bench_csv.py handles the new host and runtime without disabling its warnings.
• A gfx1151 baseline page with mlxcel versus mlx-lm on the same build is published. (None exists yet: docs/benchmark_results/ has only the correctness matrix from test(rocm): correctness matrix against a Metal baseline and a verify-test-rocm gate #1809, and benchmarks/ has no ROCm CSV.)

References

• scripts/bench_decode.sh (backend detection 297-305, GPU name 249, memory guard 315-325), scripts/compare_bench_csv.py:107-110, scripts/bench_mlxlm.py
• src/lib/mlxcel-core/src/hardware.rs:206 (gpu_backend_kind), src/commands/generate.rs:95,147-149,163, scripts/ci/rocm_smoke.sh:105-110
• Existing result pages: docs/benchmark_results/

Refresh log

2026-09-29

• Re-verified every harness gap on d8d34e2b: all still open. Corrected detect_backend to scripts/bench_decode.sh:297-305 (was 297-309) and added exact lines for the hardware fallback, memory guard and mlx_commit; compare_bench_csv.py hardcoding is still at :107-108, with the return {} at :109-110.
• Recorded that both dependencies are closed (PR feat(rocm): vendor the ROCm backend into mlxcelverse, add rocm feature #1818, PR feat(rocm): add a GPU vendor concept and AMD device reporting #1883).
• Added what PR feat(rocm): add a GPU vendor concept and AMD device reporting #1883 gives the harness (gpu_backend_kind() and the device fields) and the CLI lines it can actually parse, checked against src/commands/generate.rs and gpu_backend.cpp, since no diagnostics flag was added; plan items 1 to 3 are updated to use them.
• Noted that no ROCm benchmark page or CSV exists yet.