Skip to content

backend: Add MX quant/dequant on-the-fly compute - #170

Merged
DanielKellerM merged 5 commits into
develfrom
backend/mx-compute
Aug 11, 2026
Merged

backend: Add MX quant/dequant on-the-fly compute#170
DanielKellerM merged 5 commits into
develfrom
backend/mx-compute

Conversation

@DanielKellerM

@DanielKellerM DanielKellerM commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds size-changing on-the-fly compute (write byte count != read byte count) to the otf framework, with MX (OCP microscaling) quantization and dequantization as the first ops. One commit on devel.

Ops (selected via compute_cfg.compute_op, same programming model as transpose):

Op Direction Ratio (per 32-elem block) Width
COMPUTE_MXQUANT FP32 -> MXFP8 128B -> 33B up to 1024b
COMPUTE_MXQUANT_FP16 FP16 -> MXFP8 64B -> 33B up to 512b
COMPUTE_MXDEQUANT MXFP8 -> FP32 33B -> 128B up to 1024b
COMPUTE_MXDEQUANT_FP16 MXFP8 -> FP16 33B -> 64B up to 512b

MX block layout is inline [1B E8M0 scale][32B E5M2]; the 512b configuration roundtrips FP16 -> MXFP8 -> FP16 natively. Block geometry is single-homed in idma_pkg and the per-op byte ratios derive from it.

Mechanism. The per-op ratios program the legalizer write length, so the write burst's beat count matches what the engine emits. The compute seam presents per-lane pack occupancy with full strobes; the write manager's own beat mask gates acceptance, and the engine retires lane-exactly against the byte lanes the write port consumed (buffer_out_ready) — no drain FSM, no transport-layer state machine, and back-to-back transfers with an identical compute config pipeline with zero inter-transfer bubbles at 100% bus utilization. A config change is serialized in hardware at the request input (the request is held until the datapath drained, so its reads can never race a draining engine); software needs no ordering rules. Transpose keeps its per-write-transaction retire and single-beat burst contract, now asserted.

Parameters. ComputeOps (per-op elaboration mask; mxfp16 gates the FP16 source/destination format paths — area opt-out) and ComputeTuning (engine tuning knobs). Element formats are typed (mx_fmt_e), derived from the opcode via a single-source function, like op support and byte ratios.

Numerics. The E8M0 block scale saturates at [-128,127] and excludes Inf/NaN lanes from its scan, so extreme blocks don't flush and one non-finite lane doesn't poison its block's finite values. The E5M2 quantizer is RNE with full subnormal, saturation and Inf/NaN handling. The FP32 -> FP16 narrowing is IEEE (RNE, overflow to Inf), cross-validated against numpy's float16 cast over 4.2M vectors: zero mismatches.

Validation

Questa 2026.1, byte-exact against pure-integer DPI-C goldens (test/golden/) at every legal StrbWidth (DataWidth 32-1024; 32-512 in the make targets). Testbenches in test/backend/ with a shared skeleton in test/include/:

  • tb_idma_mxquant: FP16/FP32 sources incl. an all-tiny block, an Inf/NaN-poisoned block, FP specials, a genuine partial 33B tail beat and a 4K-crossing write.
  • tb_idma_mxroundtrip: quant -> dequant through one backend, byte-exact at both stages; FP16 legs at 512b, FP32 at 1024b.
  • tb_idma_mxrand: constrained-random campaign (random ops/blocks/addresses, 4K-biased writes) behind an AXI shim injecting 40% per-channel stalls, canary-fenced destinations, plus a pipelined same-config stream and a pipelined config-change transfer exercising the interlock.
  • tb_idma_mxneg: all 13 guard fences proven to fire (alignment, widths, protocols, length overflow, transpose single-beat, un-elaborated op fatal). This PR also adds +define+INC_ASSERT to the sim flow — without it ASSERT_NEVER compiles to nothing.
  • tb_idma_mxperf: bottleneck-channel utilization vs a plain-copy baseline — 100% at every width, single transfers and pipelined b2b streams; the other channel sits at the compression ratio.
  • tb_idma_transpose_b2b/_nd unchanged and green.

Synthesized standalone in a 12 nm technology at 1 GHz, timing met: quant 39.5k/71.5k cells (512b/1024b), dequant 37.4k/52.8k.

Known scope

  • Compute is functionally validated on idma_backend_rw_axi; size-changing compute on non-AXI src/dst is fenced (ComputeMxSrcProtocol/ComputeMxDstProtocol) until validated. The stated path to lift this: a compute-aware job format in the generated tb_idma_backend template, giving every topology MX coverage through the existing harness.
  • Identical-config transfers pipeline freely; config changes drain first (hardware-enforced, verified byte-exact).
  • 1D transfers; ND rows work per the per-row fences (dst strides must keep rows beat-aligned). Whole-tile "compact" transpose remains future work.
  • No in-system PnR data yet.

@DanielKellerM
DanielKellerM requested a review from micprog as a code owner August 11, 2026 08:12
Copilot AI lite review requested due to automatic review settings August 11, 2026 08:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Size-changing on-the-fly compute on the otf framework: COMPUTE_MXQUANT
(FP32->MXFP8, 128B->33B), COMPUTE_MXQUANT_FP16 (64B->33B, <=512b),
COMPUTE_MXDEQUANT (33B->128B) and COMPUTE_MXDEQUANT_FP16 (33B->64B,
<=512b; the 512b roundtrip is FP16->MXFP8->FP16). Per-op byte ratios,
derived from the MX block geometry homed in idma_pkg, program the
legalizer write length; the seam presents per-lane pack occupancy and
retires lane-exactly against buffer_out_ready, so tails stay sound and
identical-config transfers pipeline back-to-back at 100% bus
utilization. Config changes are serialized in hardware at the request
input. ComputeOps selects ops and the FP16 format paths per instance
(mx_fmt_e element formats, compute_tuning_t engine knobs). The E8M0
block scale saturates and excludes Inf/NaN lanes from its scan; the
FP32->FP16 narrowing is IEEE RNE, validated against numpy over 4.2M
vectors. Legalizer guard asserts fence every unsupported configuration,
each covered by a firing negative test; byte-exact DPI-golden
testbenches (test/backend, goldens in test/golden, shared skeleton in
test/include) cover directed corners, constrained-random stall
campaigns, roundtrips and throughput gates at DataWidth 32-1024.
@DanielKellerM DanielKellerM changed the title backend: Add MX quant/dequant on-the-fly compute (size-changing datapath) backend: Add MX quant/dequant on-the-fly compute Aug 11, 2026

@DanielKellerM DanielKellerM left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix comments

Comment thread src/backend/idma_otf_mxquant.sv Outdated
Comment thread src/backend/idma_axi_write.sv Outdated
Comment thread src/backend/idma_otf_compute.sv
Comment thread src/backend/idma_otf_compute.sv Outdated
Comment thread src/backend/idma_otf_mxquant.sv Outdated
Comment thread src/backend/idma_otf_mxdequant.sv Outdated
Comment thread src/idma_float_pkg.sv
@DanielKellerM
DanielKellerM merged commit 922ca04 into devel Aug 11, 2026
12 of 13 checks passed
@DanielKellerM
DanielKellerM deleted the backend/mx-compute branch August 11, 2026 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants