ECE/CS 755, VLSI Systems Design, Spring 2026, Group 5 Ashwin K. Avula, Shao-Kai Chang, Samuel Cooper, Advait Paithankar, Rohan Rao
A standalone, FP4-quantized General Matrix Multiply (GEMM) core, taken from RTL through synthesis and full place-and-route in the OpenLane 2 flow on the SkyWater SKY130 process.
The goal of this project is to design a standalone GEMM-Core that operates as the matrix-multiply engine inside a larger TPU subsystem. The core is responsible for the dense Y = Activation x Weight workload at the heart of neural-network inference, while the surrounding subsystem handles data movement, tiling, and host interfacing. By exposing a clean tiled interface with ready/valid handshakes, the core can be dropped into the subsystem as a self-contained, independently verified block.
The design target is edge inference, so the core prioritizes area and energy efficiency: it uses FP4 (4-bit float) activations and weights, FP16 accumulation, and in-hardware quantization so that quantized results never leave the accelerator at full precision.
The core is built around an 8x8 output-stationary systolic array of 64 MAC processing elements (PEs), fed by activation and weight FIFO banks, and followed by a 1x8 vector unit that performs scaling, FP4 quantization, and ReLU activation entirely in hardware.
| Parameter | Value |
|---|---|
| Systolic array | 8x8 output-stationary MAC PEs (64 total) |
| Vector unit | 1x8 lanes, hardware FP4 quantization + ReLU |
| Activation format | FP4, E2M1 (1 sign, 2 exponent, 1 mantissa) |
| Weight format | FP4, E2M1 |
| Accumulation format | FP16, IEEE 754 half precision |
| Output format | FP4, E2M1 (quantized) |
| Max tile size | A = 8, W = 8, K = 8 (an 8x8x8 tile) |
| Supported tile sizes | Any rectangular or square tile with A, W, K each in [1, 8] |
| FIFO depth | 8 entries per array row and column |
| Target clock | 14 ns (71.4 MHz constraint) |
| Process | SkyWater SKY130 (sky130A, sky130_fd_sc_hd) |
Tile semantics. A tile computes Y[A x W] = Activation[A x K] x Weight[K x W], where A is the number of activation rows, W the number of weight columns, and K the contraction (shared) dimension. Tile dimensions are passed in as 4-bit metadata fields (A_LEN, W_LEN, K_LEN).
Sub-maximal tiles. The control unit derives A_MASK and W_MASK row/column masks from the tile dimensions and gates every enable vector with them. Any tile smaller than 8x8x8 simply activates a sub-rectangle of PEs and FIFOs, so the same hardware natively supports all A, W, K combinations in [1, 8] without reconfiguration.
Cross-tile accumulation. A K dimension larger than 8 is handled by the surrounding subsystem as a sequence of tiles. The BIAS_NEW flag selects whether a tile starts a fresh accumulation (load bias) or continues accumulating onto the in-place PE accumulators; TILE_LAST marks the tile whose result should be flushed out through the vector unit.
All external channels use an independent ready/valid handshake: a data beat transfers on a rising clock edge only when both the producer's *_VLD and the consumer's *_RDY are asserted. Channels handshake independently of one another, which lets the subsystem stream metadata, data, and bias on their own schedules.
| Channel | Valid / Ready | Dir | Payload |
|---|---|---|---|
| Metadata | METADATA_VLD / METADATA_RDY |
In | TILE_START, TILE_LAST, BIAS_NEW, RELU_EN, A_LEN, W_LEN, K_LEN |
| Data | DATA_VLD / DATA_RDY |
In | A_DATA[8][4] activations, W_DATA[8][4] weights |
| Bias | BIAS_VLD / BIAS_RDY |
In | BIAS[16] (FP16) |
| Scale | SCALE_VLD / SCALE_RDY |
In | SCALE[16] (FP16) |
| Output | Y_VLD[8] / Y_RDY |
Out | Y_OUT[8][4] quantized FP4 results |
| Status | TILE_DONE (pulse) |
Out | Single-cycle tile-completion strobe |
Per-channel ready behavior. Each channel only accepts or produces beats in the FSM state where it is meaningful, so the ready signals double as natural backpressure:
- Metadata is accepted only in
IDLE.METADATA_RDYis asserted inIDLEand a tile launches whenTILE_START && METADATA_VLD. - Data is accepted during
LOAD_FIFO.DATA_RDYstays high until theK-th (final) data beat is captured, then deasserts to block further writes. - Bias is accepted during
LOAD_FIFOonly when the capturedBIAS_NEWis set.BIAS_RDYstays high until theW-th bias beat, loading one column accumulator per beat. - Scale is consumed during
GEMM_FLUSH.SCALE_RDYfollowsY_RDY, so a scale beat is pulled in only when the downstream consumer is ready to take the corresponding output. - Output is produced during
GEMM_FLUSH.Y_VLD[8]is a per-lane valid, asserted through a 2-cycle pipeline so it lands aligned withY_OUT; the consumer drivesY_RDY. TILE_DONEis a status strobe, not a handshake. It pulses for one cycle when a tile finishes.
Control is centralized, not a global pipeline. A single shared FSM (gemm_fsm) plus its datapath wrapper (gemm_control_unit) sequences the entire core through three discrete phases per tile. Load, compute, and flush for a tile each run to completion, in order, before the next tile begins; the phases of consecutive tiles do not overlap. (The systolic array is internally pipelined as data ripples through the PE mesh, but tile-level scheduling is sequential.) The control unit expands the FSM's 1-bit enables into 8-bit, mask-gated vectors and uses an 8-bit pe_wave shift register to produce the diagonal K-wide enable window that drives the array.
IDLE asserts METADATA_RDY and waits. On TILE_START && METADATA_VLD it latches the metadata (dimensions, derived masks, and the compute limit), pulses capture_metadata, and moves to LOAD_FIFO.
LOAD_FIFO writes the activation and weight FIFO banks, one beat per data handshake, for K beats total. If BIAS_NEW was set, it loads W bias values, one column accumulator per bias handshake, in parallel. It advances to GEMM_COMPUTE on the cycle that the data channel is done and the bias channel is either not required or also done.
GEMM_COMPUTE streams the FIFO contents into the 8x8 array. The pe_wave register supplies a moving K-wide enable window so each PE is active for exactly K cycles. Compute runs for A + W + K - 1 cycles. On compute_done, if TILE_LAST is set it goes to GEMM_FLUSH; otherwise it returns to IDLE and pulses TILE_DONE, leaving partial sums in the PE accumulators for the next tile.
GEMM_FLUSH reads the array out column by column through the vector unit (scale, FP4 quantize, optional ReLU), for W cycles, each cycle gated by Y_RDY && SCALE_VLD. Outputs appear on Y_OUT / Y_VLD after a 2-cycle vector-unit pipeline delay. On flush_done it returns to IDLE and pulses TILE_DONE.
| Transition | Condition | Cycles in state |
|---|---|---|
IDLE -> LOAD_FIFO |
TILE_START && METADATA_VLD |
1 (handshake) |
LOAD_FIFO -> GEMM_COMPUTE |
data done and (bias not needed or bias done) | K data beats (max(K, W) if bias loads in parallel) |
GEMM_COMPUTE -> GEMM_FLUSH |
compute_done && TILE_LAST |
A + W + K - 1 |
GEMM_COMPUTE -> IDLE |
compute_done && !TILE_LAST (pulse TILE_DONE) |
A + W + K - 1 |
GEMM_FLUSH -> IDLE |
flush_done (pulse TILE_DONE) |
W (+ 2-cycle vector-unit drain) |
For a full 8x8x8 tile this is 8 load cycles, 23 compute cycles (8 + 8 + 8 - 1), and 8 flush cycles, roughly 40 cycles end to end including handshake and pipeline drain.
Each PE is a 2-stage pipelined MAC: stage 1 multiplies the FP4 activation by the FP4 weight; stage 2 adds the FP16 product into the PE's FP16 accumulator. The accumulator is output-stationary, staying resident in the PE across a tile and across non-last tiles. Three floating-point datapaths were custom-built and optimized.
The PE multiplier takes two FP4 E2M1 operands and produces an FP16 product. The 2-bit effective-mantissa multiply is handled by FixedP2x4_opt, a hand-built 2x2 unsigned multiplier implemented from its truth table with only AND and XOR gates and explicit zero-input skipping, rather than a generic array multiplier. The result is a single-level, very small multiply block, which matters because it is replicated 64 times.
The accumulator add is a dual-path FP16 adder with truncated rounding.
The original adder had a long sequential path: a bidirectional barrel shifter, a slow 11-bit ripple-carry mantissa adder, and a full rounding stage with an extra exponent shifter. The optimized design splits this into two paths:
- Near path, for effective subtractions with exponent difference of 1 or less: the large shift happens after the add, normalization uses a leading-one-detector tree, and the add uses a 14-bit Kogge-Stone (parallel-prefix) subtractor.
- Far path, for additions and large-difference subtractions: the large shift happens before the add, the add uses a 15-bit Kogge-Stone add/sub unit, and only a 1-bit normalization is needed.
Rounding is replaced by simple round-toward-zero truncation, removing the rounding adder and the second exponent shifter. The optimizations raised the adder from 69.6 MHz to 89.1 MHz and cut nominal power from 0.172 mW to 0.124 mW, for a measured mantissa error contribution of about 0.02%.
The vector unit multiplies each FP16 column result by an FP16 per-column scale and saturates and quantizes the result down to FP4. Its 11x11 mantissa multiply uses a pruned, truncated Dadda tree.
Because the quantized output keeps only the top of the product, the design builds only the partial products and reduction cells that feed product bits [21:14]: 45 reduction cells (3 half adders + 42 full adders) versus 90 in the full tree, followed by an 8-cell ripple-carry adder that resolves only the top 8 product bits. The discarded lower bits are an accepted truncation error. This pruning cut the multiplier from 7172 um2 to 2320 um2, dropped nominal power from 0.378 mW to 0.031 mW, and raised frequency from 110.4 MHz to 144.3 MHz, at a cost of roughly 0.15% to 0.21% lost carries.
The original single-cycle PE (gate depth 60, about 73 MHz) was split into the 2-stage pipeline above by inserting a register between the FP4 multiply and the FP16 add, raising cell-level fmax to about 91 MHz.
The final design is the Design 2, hierarchical APR implementation: the GEMM PE and the vector unit are each hardened independently as macros, then assembled at the top level. Its performance, against the two earlier baselines, is summarized below.
| Design | Norm. Area | Norm. Nom. Power | Nom. Max Freq. | Max Throughput | Norm. EDAP |
|---|---|---|---|---|---|
| Design 1, flattened | 1.00x | 1.00x | 59.1 MHz | 1.59 GFLOPS/s | 1.00x |
| Design 2, flattened | 0.99x | 0.84x | 86.5 MHz | 2.27 GFLOPS/s | 0.41x |
| Design 2, hierarchical (final) | 1.60x | 0.40x | 136.7 MHz | 3.59 GFLOPS/s | 0.13x |
The final design reaches a nominal maximum frequency of 136.7 MHz and a peak nominal throughput of 3.59 GFLOPS/s at full 8x8 utilization. It trades a larger die area for the lowest power and the best energy-delay-area product (EDAP) of the three layout strategies, an 0.13x EDAP relative to the first baseline.
Throughput depends strongly on PE utilization. Large, well-tiled workloads keep the array near 100% utilization (for example, batched MNIST inference sustains throughput close to the 3.5 GFLOPS/s range), whereas a single-row batch only fills 1/8 of the array and sees roughly 1/8 the effective rate. Functional correctness was confirmed by SDF back-annotated gate-level simulation at the nominal corner and demonstrated end to end on MNIST inference.
The core was implemented with the OpenLane 2 flow on the SkyWater SKY130 process (sky130A PDK, sky130_fd_sc_hd standard-cell library). The final design uses a hierarchical APR strategy: gemm_pe and vector_unit are each placed and routed as standalone macros, then the top level instantiates 64 PE macros plus 1 vector-unit macro alongside the FIFO banks and control unit. Timing was signed off at a 14 ns target clock across three corners: nom_tt_025C_1v80, min_ss_100C_1v60, and max_ff_n40C_1v95.
| Metric | Value |
|---|---|
| Die area | 1.632 mm2 (1240 x 1316 um) |
| Core area | 1.579 mm2 |
| Total cell area | 1.034 mm2 (macros 0.971 mm2, std cells 0.063 mm2) |
| Instances | 13,004 (65 macros: 64 PE + 1 vector unit; 12,939 std cells) |
| Core utilization | 65.5% |
| Routed wirelength | approx. 1.14 m |
| Setup slack, worst (nom corner) | +6.68 ns |
| Hold slack, worst (nom corner) | +0.31 ns |
| Setup / hold violations | 0 across all three corners |
| Nominal max frequency | 136.7 MHz (14 ns target minus 6.68 ns slack) |
The GEMM PE macro is 14,344 um2 (about 0.014 mm2) with 1,410 standard cells at 86.4% utilization, and closes timing with +6.73 ns of setup slack at the nominal corner (about 137.5 MHz). It is consistently the hierarchical critical path, whereas the flattened-APR critical path is routing-dominated and varies by corner. The vector-unit macro is hardened as a single 1149 x 46 um row (about 0.053 mm2), matched to the width of the systolic array so it places directly beneath it.
Timing closed cleanly (zero setup and hold violations) at all three corners; residual signoff items are limited to a small number of max-slew and antenna warnings at the slow corner.
.
├── RTL/ SystemVerilog / Verilog source
│ ├── gemm_top.sv Top-level core integration
│ ├── gemm_fsm.sv Centralized control FSM
│ ├── gemm_control_unit.sv FSM wrapper + datapath control / vector generation
│ ├── gemm_systolic_array.sv 8x8 PE mesh
│ ├── gemm_pe.sv 2-stage pipelined MAC PE
│ ├── gemm_fifo*.sv Activation / weight FIFO bank
│ ├── vector_unit.sv 1x8 quantization + ReLU unit
│ ├── FloatP4x16.v FP4 x FP4 -> FP16 multiplier
│ ├── FloatP16x4.v FP16 x FP16 -> FP4 quantization multiplier
│ ├── fp16_adder_*.v/.sv Dual-path truncated FP16 adder
│ └── dadda_*.sv Pruned / truncated Dadda reduction tree
├── OpenLane2_Configs/ Per-macro OpenLane 2 configs (gemm_pe, vector_unit, gemm_top)
├── OpenLane2_Outputs/ Netlists, GDS, SDF, and timing/area metrics
├── DesignDocumentation/ Architecture diagrams and GDS captures
└── README.md
ECE/CS 755 final project, Spring 2026, Group 5.