fix(rocm/gemma4): #837 GetBlas dual-slot TLS + host lifetime seam - #1045
Merged
localai-bot merged 7 commits intoAug 17, 2026
Conversation
localai-bot
force-pushed
the
fix/gemma4-837-getblas-dual-slot
branch
from
August 17, 2026 08:55
8d19b60 to
18ebbd4
Compare
Independent row/ROCM-GEMMA4-GETBLAS-DUALSLOT from origin/main 3ce5a1d. Pins donor GetBlas bytes (SHA256 9df2b163…) — dirty 2bb4bd8 is not a clean donor. Hypothesis B only; cause of T=2029 wedge unconfirmed. Host lifetime seam + 0→1→0 / 1→0→1 mutations required (64cb mudler#5). FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Replace single-TLS destroy-on-hop with tls_slots[2] (device 1 → slot 1). Host recorder table covers first-use / 0→1→0 / stream rebind / capture. RED mutants: swapped selector, destroy-on-hop, missing SetStream, capture SetDevice. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Fold research c24b: GetBlas and host tests both execute RocmProductGetBlasOn (device+stream forwarded). RED mutants: forward device 0, forward null stream. Host 0→1→0→1 always runs; live hipblas probe compiles under VLLM_CPP_HIP. 3c08ce35 is not a review target. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Fold research 4d82/e819: ProductGetBlasHandle calls file-local GetBlas (real HipBlasHooks + static thread_local tls_slots). Host table asserts SetDevice revisit. HIP 0→1→0→1 is a separate CTest; missing devices exit 77. c956ba6d is not a review target. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Fold research 6195: ProductGetBlasStreamIsCapturing calls the exact HipBlasHooks hook. HIP product probe begins capture and asserts true; always-false hook mutation is RED. Host fake-capture case unchanged. 1a1153d6 is not a review target. Adjacent mudler#785/mudler#523/mudler#509/mudler#834 noted in spec; no pickup. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Host CXX missed __HIP_PLATFORM_AMD__ on hip_runtime.h. LANGUAGE HIP is the branch contract, not a recipe-only define. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
… cell whole Two repairs on top of the mudler#837 dual-slot work. Neither touches its behavior. `.agents/evidence/` is a path `scripts/check-pr-size.py` cannot classify, so the gate refused the change outright rather than sizing it: `ERROR: PR size check could not classify the change: unclassified repository path`. The donor manifest and the extracted `GetBlas` body move to `.agents/specs/` as `rocm-gemma4-getblas-dualslot-donor.md` and `.log`, matching `SPEC` and `SPEC_EVIDENCE` at `check-pr-size.py:188-189`. Teaching the checker a new `.agents/evidence/` pattern would be a governance-checker change owing its own spec and mutation evidence, which this is not. `docs/FEATURES.md` goes back to what main has. The branch had rewritten the Gemma4 row, which dropped the `VT_GEMMA4_*`/`VT_ATTN_*` env pointer, the `test_gemma4_rocm_fp8_seams` seam name and the spec link. Restoring those AND adding the mudler#837 sentence does not fit: `check-public-doc-tables` caps a table cell at 220 characters and main's cell is already 219, which is presumably why the row was shortened in the first place. Since `check-doc-checkpoint` does not ask for a FEATURES edit here -- `src/vt/rocm/` is not a feature surface and no `REGISTER_VLLM_MODEL` set changed (mudler#595, mudler#1086) -- the row keeps its existing content and the mudler#837 detail stays where the branch already put it in full, in `docs/USAGE.md` and the spec. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: ClaudeCode:claude-opus-5 [ClaudeCode]
localai-bot
force-pushed
the
fix/gemma4-837-getblas-dual-slot
branch
from
August 17, 2026 10:49
18ebbd4 to
090f1c6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #837.
Problem
On a peer-MoE device hop, the single per-thread (TLS) hipBLAS handle is destroyed
on the wrong device, corrupting the
GetBlasaccessor during multi-GPU Gemma-4MoE serving.
What changed
GetBlasdual-slot TLS keyed by device, plus a host-lifetime seam, so thehandle is no longer destroyed across the device hop. The production
GetBlasin
src/vt/rocm/rocm_matmul_hipblaslt.hipis edited in place and routesthrough the new engine, so the change is reached rather than merely present.
StreamIsCapturingis madeload-bearing on the product path.
Verification
Built and smoked on gfx1201, shipping ROCm 7.2.4 (clang
f58b06d), as thecombined #837+#838+#839 stack on
d1b0ea3a: gfx1201 buildBUILD_RC=0, all 30layers resident and bound, ready in ~20 s; batched-MoE prefill deadlock-free
(T=2012 done,
PEER_ACTactive, no hang, no HIP error); Paris/63 quality PASS.Stated plainly, because it bounds what this PR alone demonstrates: that smoke
covers the three-PR stack, not this branch in isolation. What is isolated here
is the host-side test evidence — 12 cases with real RED mutants (swapped
selector, destroy-on-hop, missing
SetStream, capture-timeSetDevice,forwarded device 0, forwarded null stream), plus a source-invariant case that
reds if the production call site is deleted.
Maintainer changes on top
Two repairs were applied while landing, neither touching behavior:
.agents/evidence/— a pathscripts/check-pr-size.pycannot classify, so the gate refused the changeoutright — to
.agents/specs/rocm-gemma4-getblas-dualslot-donor.{md,log},which match
SPECandSPEC_EVIDENCEatcheck-pr-size.py:188-189.docs/FEATURES.mdis left asmainhas it. The branch had rewritten theGemma4 row, dropping the
VT_GEMMA4_*/VT_ATTN_*env pointer, thetest_gemma4_rocm_fp8_seamsseam name and the spec link. Restoring those andadding the ROCm Gemma-4 hyp B: GetBlas single TLS destroys hipBLAS handle on peer-MoE device hop #837 sentence does not fit —
check-public-doc-tablescaps a cellat 220 characters and main's is already 219 — and
check-doc-checkpointdoesnot ask for a FEATURES edit here, since
src/vt/rocm/is not a featuresurface and no
REGISTER_VLLM_MODELset changed (doc-checkpoint makes docs/FEATURES.md a LOCK: 2 of 5 concurrent LTX-2.5 PRs conflict there and nowhere else #595, fix(#595): doc-checkpoint asks whether the registry moved, not whether a file did #1086). The ROCm Gemma-4 hyp B: GetBlas single TLS destroys hipBLAS handle on peer-MoE device hop #837detail stays where this branch already put it in full:
docs/USAGE.mdand thespec.
The branch was rebuilt by rebase rather than merge so it carries no untrailered
merge commit; all six original commits are preserved with their authorship.
Known-unrelated CI
windows-msvc-cpuandwindows-msvc-vulkanwere red on every open PR from abreak predating this branch (#503). That queue has since been cleared — #968
landed as #983 and #1068 as #1069 — and only #584's runtime crash remains.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: ClaudeCode:claude-opus-5 [ClaudeCode]