DeepSeek-V4-Flash-0731-ROCmFPX-MIX-STRIX.gguf fails to load: tensor type 105 (unknown).

[Arkovski]

DeepSeek-V4-Flash-0731-ROCmFPX-MIX-STRIX.gguf fails to load: tensor type 105 (unknown).

Type 105/106 STRIX tensors not registered in ggml.h on main or any public branch (checked q5-verifier, overlapped-moe-balance, dspark-runtime).

Docker :rocm image @3f3d60bb same failure.

[davide221]

Thanks for reporting, we are stil working on the code integration for the new quantization of the model, much higher quality than the previous one

[Arkovski]

If you need some testing, ping me and give instruction, I can do it to help you :) We'll both benefit.

---

And here is the analysis by my local AI agent about this issue, maybe it will help you in any way, but I am not data-scientist.

DeepSeek-V4-Flash-0731-ROCmFPX-MIX-STRIX.gguf fails to load: tensor type 105 (unknown) — full investigation report

Status: still broken. The 43 blk.*.ffn_down_exps.weight tensors are type 105 (Q3_1_ROCMFP3_MIX per unmerged PR #556), and the data-level mode census proves every one of the 11,008
experts uses the adaptive per-expert codebook path (mode-1) — which requires per-expert codebooks, modes, and rotations that are not present in the GGUF and not published anywhere (no
KV keys, no extra tensors, no sidecar file on the HF repo). The file cannot be decoded with type-104 semantics (the 105→104 alias produces garbage), and there is no code path in any
public branch that can supply the missing side data.

1. Environment

• Hardware: AMD Ryzen AI Max+ 395, gfx1151, 128 GB unified memory
• Stack: ROCm/HIP, custom dflash_server (no Docker), llama.cpp fork at Luce-Org/lucebox
• File: DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf (102,321,006,592 B)
• Failure mode: gguf.cpp load rejects type 105 ("not a multiple of block size (0)") because slots 105/106 in the traits table are empty on every public branch

2. Integrity (before any analysis)

• sha256 of the file: a431f955c639107d3f34ab4cc65edc806038588a0c258abd0bc3e9074b908096 — exact match with the HF LFS oid (fresh download)
• GGUF v3, 1328 tensors, 57 KV entries, arch deepseek4, 43 blocks, 256 experts, embed 4096, vocab 129280, ctx 1M

3. What was done and how

3.1 Structural map of the type-105 tensors

• 43 tensors named blk.%d.ffn_down_exps.weight, dtype 105, dims 2048x4096x256 (ne0=2048 in, ne1=4096 out, ne2=256 experts), 939,524,096 B each
• Block layout cross-validated against a known-good DSpark draft model: 14-byte blocks, 32 elements per block, 12 bytes of 3-bit codes + 2 meta bytes at block offsets 12/13
• Expert stride 3,670,016 B; 262,144 blocks/expert

3.2 KV and tensor-list scan for embedded side data

• Grepped all 57 KV keys and the 1328-tensor list for codebook/mix/book/mode/p4/rocmfp patterns
• Result: nothing. No codebook keys, no codebook tensors → the "codebook in GGUF KV" theory from the PR comment is dead for this file

3.3 Per-expert mode census (the decisive experiment)

Method: for every one of the 43 layers × 256 experts = 11,008 experts, count the top-bit-set rate of the two meta bytes (offsets 12/13) across all 262,144 blocks, using the decode
semantics from PR #556's rocmfp3_mix.cu:

• mode-0 (legacy, identical to type 104): value = ue4m3(meta) × fixed_codebook{0,±1,±2,±4} — the FULL meta byte is the scale; a top-bit-set byte would mean scale ≥ 256
• mode-1 (adaptive): value = ue4m3(meta & 0x7F) × book[(meta>>7)*8 + code] — top bit is a per-half-block book-bank selector, low 7 bits are the scale

Results (full file, verified bytes):

• Per-meta-byte top-bit rate: 49–51% across essentially all experts (min 37.8%, max 50.6% across the whole file)
• Zero experts near 0% → no expert uses the mode-0/type-104 layout
• Bank selection is an independent ~50/50 coin between the two halves of each block (joint 00/01/10/11 ≈ 25/25/25/25)
• Bank-stripped low-7-bit bytes decode as a genuine ue4m3 scale distribution (median 0.94, p95 112, max 240) — consistent with real NN-weight scales
• Mode-0 interpretation of the same bytes: 97.96% of scales > 4, median 229,376 → mathematically impossible

Verdict: 11,008/11,008 experts are mode-1 (adaptive per-expert codebook). Decoding requires per-expert bf16 codebooks (E×2×8), modes (E), and rotations (E) as described by the PR's
sidecar format (P4MIXv1\0 magic; per-layer header {layer,E,odim,idim,C,K} followed by modes[E], rots[E], books[E*16] u16).

3.4 Sidecar availability check

• https://huggingface.co/DeepSeek/.../resolve/main/DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf.p4mix.bin → 404
• HF model API now requires auth (401) — cannot enumerate repo files
• No sidecar data exists locally either

3.5 What was attempted and did NOT work

1. 105→104 alias at the gguf.cpp load boundary — Rejected by the census: would decode ~50% of half-blocks with scales ≥ 256 → guaranteed garbage. Structurally safe (identical 14-B layout) but semantically wrong
2. Embedded-codebook theory (KV/tensors in GGUF) Dead — no such keys/tensors
3. Fetch the sidecar from HF — 404
4. Use ROCMFP2 build as reference/fallback — Invalid — local file hash 313ff082… ≠ official oid 8fa6c30d…; the HF pointer never changed since Jul 8, so the local copy was never valid
5. Full port of PR Codex/deepseek4 qtype105 mix #556 (CUDA kernels + host staging registry) — Impossible without the sidecar data itself

4. What probably should be done (recommendations)

For the maintainers to make this file loadable, one of these is required:

1. Make the side data available. Preferred: embed the per-expert codebooks/modes/rotations directly in the GGUF — either as KV entries (*.p4mix.codebooks, etc.) or as extra tensors
   with a clearly defined layout — so the loader needs no external file. If the sidecar format is retained, publish .p4mix.bin alongside the GGUF and update the loader to locate
   it next to the model file.
2. Publish the format spec. The release decoder must confirm:
   • bank = top bit of each meta byte per half-block (as in the PR kernel), or a different/rotated layout — the rotations[E] array suggests a per-expert rotation is applied on top;
     its exact semantics and byte order must be specified
   • how modes[E] selects mode-0 vs mode-1 per expert, and whether any expert in this file is mode-0 (our census says none are)
3. Register type 105 in the traits table with blck_size = 32, type_size = 14, is_quantized = true (the PR Codex/deepseek4 qtype105 mix #556 diff to ggml.c/ggml.h shows exactly this; slots 105/106 are empty on all
   public branches including the checked branches q5-verifier, overlapped-moe-balance, dspark-runtime).
4. Quantizer tooling. If the quantizer is internal, a short descriptor (block layout, side-data layout, byte order, per-expert mode assignment) in the repo would let third-party
   runtimes implement decode without guessing.

5. Testing offer

I can validate any fix on gfx1151 (ROCm/HIP, no Docker) against the official oid: 43-tensor load, weight-level decode equivalence, greedy coherence, and tok/s. The census tooling
(numpy + memmap mode scan) is available on request.

Appendix: key numbers for verification

• File: 102,321,006,592 B; sha256 a431f955…908096 (LFS oid match)
• 43 × blk.%d.ffn_down_exps.weight, type 105, dims 2048×4096×256, 14-B blocks
• meta[12] top-bit: 50.04% (blk.0), meta[13]: 49.58%; all 11,008 experts 37.8–50.6%
• bank joint 00:25.1% / 01:24.8% / 10:25.3% / 11:24.7%
• mode-1 scale (low 7 bits): median 0.94, p95 112, max 240
• mode-0 scale (full byte): 97.96% > 4 → impossible
• PR Codex/deepseek4 qtype105 mix #556 reference commit bf79ba5

[Arkovski]

The fixed MIX-STRIX file now loads fine (built from feat/qtype106-down-surface, commit 90f85fa4), but the server segfaults on the first chat request.

Startup is clean: 93.3 GiB GPU buffer, all experts hot, KV 877.8 MB for ctx=131072, server listens on 8080. The first request then crashes instantly, right after the prefix-cache line, before any token:

[server] chat ... msgs=2 tools=39 prompt_tokens=19842 max_tokens=106668 max_ctx=131072
[server] chat CACHE ... restore=false slot=-1 prefix_len=0 effective_prompt=19842 pflash=false
Segmentation fault (core dumped)

Launched in quality mode: --target-device hip:0 --max-ctx 131072, no extra flags, no draft. Same client and request shape works on llama.cpp (ROCm, 131k ctx) on this machine, so it's not a hardware/memory limit.

I wanted to attach the error logs, BUT 2 issues:

• why does it use 80% of my GPU and 35% of CPU instead of full GPU offload?

• sometimes (or I should say - it happens very often) it uses 100% of my NVMe drive, why? It should be fully offloaded to the vRAM.

[Arkovski]

PS. 131072 context should be no issue when it comes to DeepSeek architecture, I can run it freely with no problems:

cd ~/ai-stack/llama.cpp && ./build-rocm/bin/llama-server --device ROCm0 --model "/run/media/arkovski/7766b08a-3b4c-4976-bccd-08d2e86e5660/AI/models/DeepSeek_V4_Flash_UD-IQ3_XXS/DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf" --n-gpu-layers all --fit off --no-repack --load-mode dio --flash-attn on --jinja --reasoning on --reasoning-format deepseek --reasoning-preserve --chat-template-kwargs '{"reasoning_effort":"max"}' --ctx-size 131072 --cache-type-k f16 --cache-type-v f16 --batch-size 8192 --ubatch-size 4096 --threads 16 --threads-batch 16 --parallel 1 --cache-prompt --cache-reuse 0 --cache-ram 1024 --ctx-checkpoints 2 --checkpoint-min-step 16384 --no-warmup --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0.0 --presence-penalty 0.0 --frequency-penalty 0.0 --repeat-penalty 1.0 --host 127.0.0.1 --port 8080 -lv 4

And it uses ~110GB of RAM, 100% is GPU offloaded, token generation speed is ~11 /s