backend: Add MX quant/dequant on-the-fly compute - #170
Merged
Conversation
Size-changing on-the-fly compute on the otf framework: COMPUTE_MXQUANT (FP32->MXFP8, 128B->33B), COMPUTE_MXQUANT_FP16 (64B->33B, <=512b), COMPUTE_MXDEQUANT (33B->128B) and COMPUTE_MXDEQUANT_FP16 (33B->64B, <=512b; the 512b roundtrip is FP16->MXFP8->FP16). Per-op byte ratios, derived from the MX block geometry homed in idma_pkg, program the legalizer write length; the seam presents per-lane pack occupancy and retires lane-exactly against buffer_out_ready, so tails stay sound and identical-config transfers pipeline back-to-back at 100% bus utilization. Config changes are serialized in hardware at the request input. ComputeOps selects ops and the FP16 format paths per instance (mx_fmt_e element formats, compute_tuning_t engine knobs). The E8M0 block scale saturates and excludes Inf/NaN lanes from its scan; the FP32->FP16 narrowing is IEEE RNE, validated against numpy over 4.2M vectors. Legalizer guard asserts fence every unsupported configuration, each covered by a firing negative test; byte-exact DPI-golden testbenches (test/backend, goldens in test/golden, shared skeleton in test/include) cover directed corners, constrained-random stall campaigns, roundtrips and throughput gates at DataWidth 32-1024.
DanielKellerM
force-pushed
the
backend/mx-compute
branch
from
August 11, 2026 17:14
8be35a7 to
67ad5a4
Compare
DanielKellerM
commented
Aug 11, 2026
DanielKellerM
left a comment
Collaborator
Author
There was a problem hiding this comment.
Fix comments
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds size-changing on-the-fly compute (write byte count != read byte count) to the otf framework, with MX (OCP microscaling) quantization and dequantization as the first ops. One commit on
devel.Ops (selected via
compute_cfg.compute_op, same programming model as transpose):COMPUTE_MXQUANTCOMPUTE_MXQUANT_FP16COMPUTE_MXDEQUANTCOMPUTE_MXDEQUANT_FP16MX block layout is inline
[1B E8M0 scale][32B E5M2]; the 512b configuration roundtrips FP16 -> MXFP8 -> FP16 natively. Block geometry is single-homed inidma_pkgand the per-op byte ratios derive from it.Mechanism. The per-op ratios program the legalizer write length, so the write burst's beat count matches what the engine emits. The compute seam presents per-lane pack occupancy with full strobes; the write manager's own beat mask gates acceptance, and the engine retires lane-exactly against the byte lanes the write port consumed (
buffer_out_ready) — no drain FSM, no transport-layer state machine, and back-to-back transfers with an identical compute config pipeline with zero inter-transfer bubbles at 100% bus utilization. A config change is serialized in hardware at the request input (the request is held until the datapath drained, so its reads can never race a draining engine); software needs no ordering rules. Transpose keeps its per-write-transaction retire and single-beat burst contract, now asserted.Parameters.
ComputeOps(per-op elaboration mask;mxfp16gates the FP16 source/destination format paths — area opt-out) andComputeTuning(engine tuning knobs). Element formats are typed (mx_fmt_e), derived from the opcode via a single-source function, like op support and byte ratios.Numerics. The E8M0 block scale saturates at [-128,127] and excludes Inf/NaN lanes from its scan, so extreme blocks don't flush and one non-finite lane doesn't poison its block's finite values. The E5M2 quantizer is RNE with full subnormal, saturation and Inf/NaN handling. The FP32 -> FP16 narrowing is IEEE (RNE, overflow to Inf), cross-validated against numpy's float16 cast over 4.2M vectors: zero mismatches.
Validation
Questa 2026.1, byte-exact against pure-integer DPI-C goldens (
test/golden/) at every legal StrbWidth (DataWidth 32-1024; 32-512 in the make targets). Testbenches intest/backend/with a shared skeleton intest/include/:tb_idma_mxquant: FP16/FP32 sources incl. an all-tiny block, an Inf/NaN-poisoned block, FP specials, a genuine partial 33B tail beat and a 4K-crossing write.tb_idma_mxroundtrip: quant -> dequant through one backend, byte-exact at both stages; FP16 legs at 512b, FP32 at 1024b.tb_idma_mxrand: constrained-random campaign (random ops/blocks/addresses, 4K-biased writes) behind an AXI shim injecting 40% per-channel stalls, canary-fenced destinations, plus a pipelined same-config stream and a pipelined config-change transfer exercising the interlock.tb_idma_mxneg: all 13 guard fences proven to fire (alignment, widths, protocols, length overflow, transpose single-beat, un-elaborated op fatal). This PR also adds+define+INC_ASSERTto the sim flow — without itASSERT_NEVERcompiles to nothing.tb_idma_mxperf: bottleneck-channel utilization vs a plain-copy baseline — 100% at every width, single transfers and pipelined b2b streams; the other channel sits at the compression ratio.tb_idma_transpose_b2b/_ndunchanged and green.Synthesized standalone in a 12 nm technology at 1 GHz, timing met: quant 39.5k/71.5k cells (512b/1024b), dequant 37.4k/52.8k.
Known scope
idma_backend_rw_axi; size-changing compute on non-AXI src/dst is fenced (ComputeMxSrcProtocol/ComputeMxDstProtocol) until validated. The stated path to lift this: a compute-aware job format in the generatedtb_idma_backendtemplate, giving every topology MX coverage through the existing harness.