Finds the provably best scale for each block of quantized LLM weights, and measures how much MSE the usual heuristics leave behind.
Block quantizers such as Q4_0, Q3 and Q2 store one scale per block of 32 weights. Tools pick that scale with absmax (max|x| / 7), a sign trick (max / -8), or a small grid search around it, as llama.cpp's make_qx_quants does. Nobody publishes how far these heuristics are from the best possible scale, because finding the true optimum is assumed to need an expensive search. People tuning imatrix-weighted quants therefore have nothing exact to compare their heuristic against.
For a b-bit block quantizer with integer levels qmin..qmax (-8..7 at 4 bits), each weight is stored as q_i = clamp(round(x_i / s), qmin, qmax) and the weighted reconstruction error is
E(s) = Σ w_i (x_i - s·q_i)²
If the assignment q is held fixed, E is a quadratic in s. The assignment only changes when x_i / s crosses a half-integer, which happens at the breakpoints
s = |x_i| / (k + 0.5), k = 0, 1, ..., (number of levels on x_i's side) - 1
Once an element reaches its outermost level it is clamped, so it has no further breakpoints. E(s) is therefore piecewise quadratic. It is also continuous: at a breakpoint, both neighbouring levels are exactly s/2 away from x_i. And because clamp(round(·)) picks the nearest level, min_s E(s) is the joint optimum over the scale and every possible assignment, not just the best scale for one fixed rounding rule.
The solver sweeps over those pieces exactly:
- Generate every breakpoint of every element (at most
n · 2^(b-1)) and sort them in decreasing order. - Start at
s = +∞, where everyq_i = 0,A = Σ w·x·q = 0andB = Σ w·q² = 0. Walk down the sorted breakpoints. Crossing one moves one element one level further from zero, which addsw_i·|x_i|toAandw_i·(2k+1)toB. That is O(1) per breakpoint. - Between two consecutive breakpoints
[lo, hi]the assignment is fixed, so the unconstrained minimiser iss* = A / B. Clamp it to[lo, hi]and score it withE - Σwx² = s·(s·B - 2A). - The interval with the lowest score contains the global optimum.
Each block costs O(nL log nL) for n weights and L levels per side. The numpy version pads every element to the same number of breakpoints (padding sits at s = 0 and adds nothing to A or B), sorts row-wise, and turns step 2 into a cumsum, so a whole batch of blocks is solved at once. Tied breakpoints need no special handling: an interval of zero length evaluates E at the tie point, where every partial assignment gives the same error.
Negative scales. Q4_0 uses d = max / -8. That scale is negative whenever the largest-magnitude weight is positive, which puts the extra negative level on the side of that weight. A negative scale with levels -8..7 is the same as a positive scale with the mirrored levels -7..8. By default the solver runs the sweep on both x and -x and keeps the better result, so it also beats the sign trick. Pass allow_negative=False to search only s > 0.
Two weights x = [1.0, 0.7], 2 bits (levels -2..1), positive scales only. The breakpoints are 1.0/0.5 = 2.0 and 0.7/0.5 = 1.4.
| interval of s | assignment q | A, B | s* = A/B, clamped | E(s) |
|---|---|---|---|---|
| (2.0, ∞) | (0, 0) | 0, 0 | - | 1.490 |
| (1.4, 2.0] | (1, 0) | 1.0, 1 | 1.0 → 1.4 | 0.650 |
| (0, 1.4] | (1, 1) | 1.7, 2 | 0.85 | 0.045 |
Absmax picks s = 1.0 and gets E = 0.09, twice the optimum. With negative scales allowed, the best is s = -0.54 (levels (-2, -1)) with E = 0.032. The sign trick s = 1.0 / -2 = -0.5 gets E = 0.04. tests/test_core.py checks these numbers.
Requires Python 3.10+ and numpy (the only runtime dependency). From a clone of this repository:
cd exactscale
python -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
pytest
import numpy as np
from exactscale import block_errors, grid_refine_scales, optimal_scale, optimal_scales
blocks = np.random.default_rng(0).standard_normal((10_000, 32)) # (n_blocks, 32)
imatrix = np.random.default_rng(1).random(32) # optional, broadcasts per column
s_exact = optimal_scales(blocks, 4, imatrix) # one float64 scale per block
s_grid = grid_refine_scales(blocks, 4, imatrix)
e_exact = block_errors(blocks, s_exact, 4, imatrix)
e_grid = block_errors(blocks, s_grid, 4, imatrix)
print(f"{100 * (1 - e_exact.sum() / e_grid.sum()):.2f}% less error than grid refinement")
# 1.30% less error than grid refinement
print(optimal_scale([1.0, 0.7], 2)) # scalar, pure-Python path
# -0.54bits is an integer from 2 to 8 (levels -2^(b-1)..2^(b-1)-1), or an explicit (qmin, qmax) pair. symmetric=True uses -(2^(b-1)-1)..2^(b-1)-1. The baselines absmax_scales, signed_absmax_scales and grid_refine_scales take the same arguments and also return scales. Errors are always measured by the same block_errors. quantize returns the integer levels, and check_optimality runs the brute-force check on your own blocks.
exactscale bench prints the MSE gap table per distribution and bit width, then a throughput table. exactscale verify runs the brute-force optimality check and exits non-zero if any block fails. Both accept --bits, --dist, --seed, --weighted (log-normal importance weights) and --blocks. python -m exactscale works too.
$ exactscale bench --blocks 20000 --bits 4 2 --dist gaussian --no-speed
exactscale bench: 20,000 blocks of 32 per cell, seed 0, unweighted, 1000 bootstrap resamples
python 3.14.6, numpy 2.5.3, arm64 Darwin
relMSE = total squared error / total squared weight
vs X = % reduction in total squared error of exact over heuristic X [95% bootstrap CI]
distribution bits relMSE exact relMSE grid vs absmax vs signed-absmax vs grid
--------------------------------------------------------------------------------------------------------
gaussian 4 0.00653 0.00654 30.66% [30.45, 30.84] 11.48% [11.33, 11.62] 0.16% [0.14, 0.18]
gaussian 2 0.10981 0.11148 74.64% [74.52, 74.76] 16.36% [16.13, 16.57] 1.50% [1.41, 1.58]
The full benchmark (exactscale bench, defaults: 100,000 blocks, 4 distributions, 4/3/2 bits) takes about two minutes. The full verification (exactscale verify) takes about 20 minutes.
All numbers below come from the commands shown, run on an 8-core Apple Silicon Mac (arm64, macOS 26) with Python 3.14.6 and numpy 2.5.3 on a single thread. Every cell uses 100,000 blocks of 32 weights, seed 0. "vs X" is 100·(1 - ΣE_exact / ΣE_X), the percentage of X's total squared error that the exact scale removes. The bracketed range is a paired percentile bootstrap 95% CI over blocks, from 1,000 resamples. relMSE is total squared error divided by total squared weight. Seeds are fixed, so rerunning a command reproduces the error columns exactly. Throughput depends on the machine.
distribution bits relMSE exact relMSE grid vs absmax vs signed-absmax vs grid
--------------------------------------------------------------------------------------------------------
gaussian 4 0.00654 0.00655 30.58% [30.49, 30.67] 11.49% [11.42, 11.55] 0.17% [0.16, 0.18]
gaussian 3 0.02634 0.02640 48.66% [48.57, 48.74] 13.39% [13.31, 13.46] 0.21% [0.20, 0.22]
gaussian 2 0.10965 0.11137 74.67% [74.62, 74.72] 16.34% [16.24, 16.44] 1.54% [1.51, 1.58]
laplace 4 0.01040 0.01041 29.07% [28.98, 29.16] 8.79% [8.74, 8.85] 0.07% [0.07, 0.08]
laplace 3 0.03891 0.03895 48.01% [47.92, 48.09] 13.22% [13.16, 13.29] 0.12% [0.11, 0.12]
laplace 2 0.13504 0.13627 68.52% [68.45, 68.59] 18.53% [18.44, 18.63] 0.90% [0.87, 0.92]
student-t3 4 0.01519 0.01519 26.61% [26.44, 26.77] 6.43% [6.36, 6.50] 0.05% [0.04, 0.06]
student-t3 3 0.05015 0.05019 44.38% [44.21, 44.54] 11.80% [11.72, 11.89] 0.08% [0.08, 0.08]
student-t3 2 0.14173 0.14267 61.45% [61.32, 61.59] 17.75% [17.64, 17.87] 0.66% [0.64, 0.67]
gauss+outliers 4 0.02336 0.02337 24.09% [23.97, 24.21] 4.60% [4.56, 4.64] 0.02% [0.02, 0.03]
gauss+outliers 3 0.06355 0.06357 31.47% [31.28, 31.66] 7.86% [7.78, 7.94] 0.04% [0.04, 0.04]
gauss+outliers 2 0.12077 0.12141 49.09% [48.86, 49.32] 11.91% [11.75, 12.05] 0.53% [0.51, 0.54]
Each weight has a log-normal(0, 1) importance. All three heuristics pick their scale from the weights' magnitudes alone. Only grid refinement uses the importances, and only to rank its 19 candidates.
distribution bits relMSE exact relMSE grid vs absmax vs signed-absmax vs grid
--------------------------------------------------------------------------------------------------------
gaussian 4 0.00571 0.00587 39.42% [39.29, 39.56] 22.79% [22.65, 22.92] 2.73% [2.67, 2.79]
gaussian 3 0.02274 0.02364 55.70% [55.58, 55.81] 25.31% [25.17, 25.46] 3.82% [3.75, 3.90]
gaussian 2 0.09365 0.10180 78.37% [78.31, 78.43] 28.51% [28.37, 28.67] 8.00% [7.89, 8.13]
laplace 4 0.00926 0.00950 37.15% [37.01, 37.29] 19.06% [18.91, 19.19] 2.60% [2.53, 2.66]
laplace 3 0.03359 0.03507 55.25% [55.11, 55.38] 25.28% [25.11, 25.45] 4.22% [4.14, 4.30]
laplace 2 0.11331 0.12225 73.67% [73.57, 73.76] 31.87% [31.70, 32.06] 7.31% [7.20, 7.44]
student-t3 4 0.01374 0.01403 33.39% [33.12, 33.66] 15.12% [14.89, 15.33] 2.09% [2.01, 2.18]
student-t3 3 0.04338 0.04526 51.64% [51.42, 51.87] 23.42% [23.19, 23.66] 4.17% [4.05, 4.28]
student-t3 2 0.11793 0.12666 67.78% [67.60, 67.94] 31.22% [31.01, 31.43] 6.89% [6.77, 7.01]
gauss+outliers 4 0.02156 0.02187 29.90% [29.67, 30.13] 11.82% [11.63, 12.01] 1.39% [1.33, 1.46]
gauss+outliers 3 0.05655 0.05848 38.90% [38.63, 39.16] 17.93% [17.70, 18.15] 3.30% [3.18, 3.42]
gauss+outliers 2 0.10495 0.11161 55.65% [55.37, 55.95] 23.39% [23.11, 23.67] 5.97% [5.84, 6.10]
What the tables say:
- Absmax is far from optimal. It leaves 24-31% avoidable error at 4 bits and 49-75% at 2 bits, because it never clips and never uses the extra negative level.
- Q4_0's sign trick (
max / -8) recovers much of that, but still leaves 4.6-11.5% at 4 bits unweighted, and 12-23% once importance weights are involved. - Grid refinement is nearly optimal on unweighted blocks: within 0.25% at 3-4 bits and within 1.6% at 2 bits. That is worth knowing: for plain MSE, a 19-point refinement is already enough. With importance weights the gap grows to 1.4-2.7% at 4 bits and 6-8% at 2 bits. Every candidate it tries maps
maxto within 0.9 of the top level, and the weighted optimum can lie outside that window, for example when it pays to clip a large weight that has low importance.
throughput, single thread, 100,000 gaussian blocks, best of 3
bits exact blocks/s grid blocks/s grid/exact
-----------------------------------------------
4 48,789 479,195 9.8x
3 93,035 480,698 5.2x
2 205,152 480,978 2.3x
The exact solver is 2-10x slower than grid refinement, and the gap widens with bit width because each extra bit doubles the number of breakpoints. At about 49,000 blocks/s, the ~2·10⁸ blocks of a 7B-parameter model take roughly 70 minutes on one core at 4 bits. Blocks are independent, so this splits trivially across cores, but the package does not do that for you. That is affordable for building a reference quant or scoring a heuristic, and too slow to replace the heuristic in a quantizer's hot loop.
exactscale verify: 2,000 blocks per cell, 200,000-point grid on each sign + every breakpoint-interval midpoint
gaussian 4 bits violations 0/2000 worst (exact - best rival)/exact = -1.53e-13 [94.8s] ok
gaussian 3 bits violations 0/2000 worst (exact - best rival)/exact = -6.85e-16 [93.4s] ok
gaussian 2 bits violations 0/2000 worst (exact - best rival)/exact = -1.66e-16 [98.9s] ok
laplace 4 bits violations 0/2000 worst (exact - best rival)/exact = -1.24e-13 [100.2s] ok
laplace 3 bits violations 0/2000 worst (exact - best rival)/exact = -4.31e-14 [101.4s] ok
laplace 2 bits violations 0/2000 worst (exact - best rival)/exact = +0.00e+00 [111.3s] ok
student-t3 4 bits violations 0/2000 worst (exact - best rival)/exact = +0.00e+00 [102.2s] ok
student-t3 3 bits violations 0/2000 worst (exact - best rival)/exact = +0.00e+00 [103.5s] ok
student-t3 2 bits violations 0/2000 worst (exact - best rival)/exact = +0.00e+00 [102.4s] ok
gauss+outliers 4 bits violations 0/2000 worst (exact - best rival)/exact = +0.00e+00 [114.7s] ok
gauss+outliers 3 bits violations 0/2000 worst (exact - best rival)/exact = +0.00e+00 [107.3s] ok
gauss+outliers 2 bits violations 0/2000 worst (exact - best rival)/exact = +0.00e+00 [97.9s] ok
all blocks optimal
Across 24,000 blocks, 400,514 competing scales each, no competitor beat the exact scale by more than the 1e-9 tolerance. The worst case is at most 0: in some cells a competitor ties the optimum exactly, and none beats it. Those ties are real multiple optima. For example, student-t3 block 1427 has one weight of 43.6 while the next largest is 2.5. Every scale 43.6/q with q from 1 to 7 reconstructs the outlier exactly and rounds everything else to 0, so they all give the same error. The grid lands exactly on 43.6/q whenever 2q divides 200,000. The raw logs are in results/.
Negative scales are searched by default, which doubles the cost. A positive-only sweep is half the work, but it would lose to Q4_0's own max / -8 heuristic on many blocks, and then "exact" would be a misleading name. Mirroring the problem (x → -x) instead of writing a second sweep with a mirrored level range keeps one code path. Blocks whose best assignment never uses the extra negative level tie exactly between the two signs, and either sign may be returned. The invariance tests allow for that and compare errors, not signs.
The vectorised and scalar solvers are separate implementations that are meant to agree bit for bit. optimal_scales is numpy (padding, row-wise stable argsort, cumsum). optimal_scale is plain Python lists and a loop. They perform the same IEEE operations in the same order: same stable tie order, sequential accumulation, and an interval score s·(s·B - 2A) that leaves out the constant Σwx², which numpy and Python would otherwise sum in different orders. That makes "identical on 10k blocks" a real assert_array_equal, not a tolerance that could hide a bug. The reported error is never the sweep's running quadratic. block_errors always recomputes it from the definition, so the solver and the baselines are scored by the same code.
The brute-force checker does not trust the piecewise-quadratic argument. It evaluates Σ w (x - s·clamp(round(x/s)))² directly, on a 200,000-point grid over (0, 2·max|x|] on each sign plus the midpoint of every breakpoint interval. It is slow (about 95 s per 2,000 blocks), so the test suite runs the midpoint check on 2,000 blocks and the full grid on a subset, and the full claim lives behind exactscale verify. The checker is itself tested: it must flag grid refinement at 2 bits, and exact scales nudged by a factor of 1 + 1e-4.
- Scale only. Formats with a per-block minimum or offset (Q4_1, Q5_1, the K-quant mins) have a two-parameter problem that this sweep does not solve.
- Float scales. K-quants quantize the block scales themselves to 6 or 8 bits inside a super-block. The optimum here is for an unquantized float scale, so after scale quantization it is no longer exactly optimal.
- Nearest-level rounding. The error model rounds each weight to the nearest level. llama.cpp rounds halves away from zero, while numpy's
rintrounds halves to even. The two differ only on exact ties, where both levels give the same error. - Synthetic data. The benchmark uses generated Gaussian, Laplace, Student-t (ν=3) and outlier-injected blocks, and log-normal importance weights. No real model weights are included, and the gap on a real checkpoint with a real imatrix will differ.
- Speed. The solver is single-threaded numpy. It is slower than grid refinement (see the throughput table), and the gap grows with bit width because the breakpoint count doubles with each extra bit. A merge of the per-level sorted lists, or a compiled loop, would be faster. Neither is implemented.
- "Provably" means the argument above plus brute force, in float64. Optimality is checked to a relative tolerance of 1e-9 (plus
1e-12·Σwx²absolute, for error-free blocks), and dominance to 1e-12 relative. There is no machine-checked proof.
MIT. See LICENSE.