Skip to content

fix(rocm/gemma4): #837 GetBlas dual-slot TLS + host lifetime seam - #1045

Merged
localai-bot merged 7 commits into
mudler:mainfrom
bakon11:fix/gemma4-837-getblas-dual-slot
Aug 17, 2026
Merged

fix(rocm/gemma4): #837 GetBlas dual-slot TLS + host lifetime seam#1045
localai-bot merged 7 commits into
mudler:mainfrom
bakon11:fix/gemma4-837-getblas-dual-slot

Conversation

@bakon11

@bakon11 bakon11 commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Fixes #837.

Problem

On a peer-MoE device hop, the single per-thread (TLS) hipBLAS handle is destroyed
on the wrong device, corrupting the GetBlas accessor during multi-GPU Gemma-4
MoE serving.

What changed

  • GetBlas dual-slot TLS keyed by device, plus a host-lifetime seam, so the
    handle is no longer destroyed across the device hop. The production GetBlas
    in src/vt/rocm/rocm_matmul_hipblaslt.hip is edited in place and routes
    through the new engine, so the change is reached rather than merely present.
  • Product-call seam and an exit-77 HIP probe; StreamIsCapturing is made
    load-bearing on the product path.
  • Spec and identity probe (0→1→0→1) land in the same PR, spec first.

Verification

Built and smoked on gfx1201, shipping ROCm 7.2.4 (clang f58b06d), as the
combined #837+#838+#839 stack on d1b0ea3a: gfx1201 build BUILD_RC=0, all 30
layers resident and bound, ready in ~20 s; batched-MoE prefill deadlock-free
(T=2012 done, PEER_ACT active, no hang, no HIP error); Paris/63 quality PASS.

Stated plainly, because it bounds what this PR alone demonstrates: that smoke
covers the three-PR stack, not this branch in isolation. What is isolated here
is the host-side test evidence — 12 cases with real RED mutants (swapped
selector, destroy-on-hop, missing SetStream, capture-time SetDevice,
forwarded device 0, forwarded null stream), plus a source-invariant case that
reds if the production call site is deleted.

Maintainer changes on top

Two repairs were applied while landing, neither touching behavior:

The branch was rebuilt by rebase rather than merge so it carries no untrailered
merge commit; all six original commits are preserved with their authorship.

Known-unrelated CI

windows-msvc-cpu and windows-msvc-vulkan were red on every open PR from a
break predating this branch (#503). That queue has since been cleared — #968
landed as #983 and #1068 as #1069 — and only #584's runtime crash remains.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: ClaudeCode:claude-opus-5 [ClaudeCode]

Don Mirror and others added 7 commits August 17, 2026 10:49
Independent row/ROCM-GEMMA4-GETBLAS-DUALSLOT from origin/main 3ce5a1d.
Pins donor GetBlas bytes (SHA256 9df2b163…) — dirty 2bb4bd8 is not a clean donor.
Hypothesis B only; cause of T=2029 wedge unconfirmed.
Host lifetime seam + 0→1→0 / 1→0→1 mutations required (64cb mudler#5).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Hermes:grok-4.6 [Hermes]
Replace single-TLS destroy-on-hop with tls_slots[2] (device 1 → slot 1).
Host recorder table covers first-use / 0→1→0 / stream rebind / capture.
RED mutants: swapped selector, destroy-on-hop, missing SetStream, capture SetDevice.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Hermes:grok-4.6 [Hermes]
Fold research c24b: GetBlas and host tests both execute RocmProductGetBlasOn
(device+stream forwarded). RED mutants: forward device 0, forward null stream.
Host 0→1→0→1 always runs; live hipblas probe compiles under VLLM_CPP_HIP.

3c08ce35 is not a review target.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Hermes:grok-4.6 [Hermes]
Fold research 4d82/e819: ProductGetBlasHandle calls file-local GetBlas
(real HipBlasHooks + static thread_local tls_slots). Host table asserts
SetDevice revisit. HIP 0→1→0→1 is a separate CTest; missing devices exit 77.

c956ba6d is not a review target.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Hermes:grok-4.6 [Hermes]
Fold research 6195: ProductGetBlasStreamIsCapturing calls the exact
HipBlasHooks hook. HIP product probe begins capture and asserts true;
always-false hook mutation is RED. Host fake-capture case unchanged.

1a1153d6 is not a review target. Adjacent mudler#785/mudler#523/mudler#509/mudler#834 noted
in spec; no pickup.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Hermes:grok-4.6 [Hermes]
Host CXX missed __HIP_PLATFORM_AMD__ on hip_runtime.h. LANGUAGE HIP
is the branch contract, not a recipe-only define.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Hermes:grok-4.6 [Hermes]
… cell whole

Two repairs on top of the mudler#837 dual-slot work. Neither touches its behavior.

`.agents/evidence/` is a path `scripts/check-pr-size.py` cannot classify, so the
gate refused the change outright rather than sizing it: `ERROR: PR size check
could not classify the change: unclassified repository path`. The donor manifest
and the extracted `GetBlas` body move to `.agents/specs/` as
`rocm-gemma4-getblas-dualslot-donor.md` and `.log`, matching `SPEC` and
`SPEC_EVIDENCE` at `check-pr-size.py:188-189`. Teaching the checker a new
`.agents/evidence/` pattern would be a governance-checker change owing its own
spec and mutation evidence, which this is not.

`docs/FEATURES.md` goes back to what main has. The branch had rewritten the
Gemma4 row, which dropped the `VT_GEMMA4_*`/`VT_ATTN_*` env pointer, the
`test_gemma4_rocm_fp8_seams` seam name and the spec link. Restoring those AND
adding the mudler#837 sentence does not fit: `check-public-doc-tables` caps a table
cell at 220 characters and main's cell is already 219, which is presumably why
the row was shortened in the first place. Since `check-doc-checkpoint` does not
ask for a FEATURES edit here -- `src/vt/rocm/` is not a feature surface and no
`REGISTER_VLLM_MODEL` set changed (mudler#595, mudler#1086) -- the row keeps its existing
content and the mudler#837 detail stays where the branch already put it in full, in
`docs/USAGE.md` and the spec.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: ClaudeCode:claude-opus-5 [ClaudeCode]
@localai-bot
localai-bot force-pushed the fix/gemma4-837-getblas-dual-slot branch from 18ebbd4 to 090f1c6 Compare August 17, 2026 10:49
@localai-bot
localai-bot merged commit 559973c into mudler:main Aug 17, 2026
18 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ROCm Gemma-4 hyp B: GetBlas single TLS destroys hipBLAS handle on peer-MoE device hop

3 participants