[zzylol]

Stacked on #221 (fix/univmon-certifiable-estimators) and targets that branch: terminal-mode calc_l2 uses L2HH::get_l2_sqr(), which #221 adds. It can be retargeted to main once #221 merges.

Why

UnivMonPyramid (src/sketch_framework/univmon_optimized.rs) has the same two estimator problems #221 fixes in UnivMon, so its distinct count and L2 norm carried no error bound that held:

• calc_card dropped heap keys by their Count Sketch estimate, which biases the distinct count.
• calc_l2 went through the heap recursion, with more variance than the Count Sketch's own F2 estimate and no stated bound.

What

• calc_card counts every heap key, in standard and terminal mode. Entropy and other g keep their filters.
• calc_l2 returns layer 0's Count Sketch L2 in standard mode, and sqrt of the summed per-stratum squared estimates in terminal mode.
• New UnivMonPyramid::top_layer_complete(), the precondition of the distinct-count bound.
• Doc comments on calc_l2 and calc_card state the (eps, delta) bounds for the pyramid and refer to UnivMon's derivations from fix(univmon)!: every query is the paper's Algorithm 2 G-sum #221. docs/api/api_univmon_optimized.md and docs/tests.md are updated. No CHANGELOG entry.

How

• calc_g_sum (and the terminal-mode calc_terminal_g_sum) take is_card and, when it is set, skip neither the count > 0 filter nor the heavy_threshold filter. This is the same change fix(univmon)!: every query is the paper's Algorithm 2 G-sum #221 makes in UnivMon.
• In standard mode layer 0 receives every key, so its Count Sketch estimates F2 directly (each row's sum of squared counters, median across rows). In terminal mode the layers hold disjoint strata, so their F2 estimates add.
• The bounds in the pyramid's terms:
  • Standard-mode L2 uses layer 0's dimensions (the elephant ones whenever elephant_layers >= 1): delta = P(Bin(r, p) >= ceil(r/2)) with p = 2q/eps^2. Terminal mode holds each stratum to its own layer's dimensions and union-bounds over the non-empty strata.
  • The distinct-count bound does not depend on the sketch dimensions, so the elephant/mouse split does not enter it. Given a complete top layer the estimate is unbiased with variance at most 2 F0^2/k, so P(|Y_0 - F0| > eps F0) <= 2/(k eps^2) for k = heap_size >= 8.

Before this PR

• A heap key whose Count Sketch estimate was not positive was left out of the distinct count. So was a key below L2/sqrt(k) on an incomplete layer. For example, a one-layer 1x1 pyramid holding a weight-5 key and a weight-1 key of opposite sign returned calc_card() == 1.
• With a 5x64 layer 0, calc_l2()^2 missed the true F2 by more than 50% in 39 of 150 Zipf trials, where the Count Sketch bound allows delta ~ 0.016.

After this PR

• The same pyramid returns calc_card() == 2. The distinct count is unbiased whenever top_layer_complete() holds, and callers can check that.
• calc_l2() equals layer 0's direct estimate and stays within the documented bound in the Monte Carlo test.

Verification

Each fix has a deterministic test that fails before it and passes after it:

• pyramid_calc_card_counts_heap_keys_with_nonpositive_estimates
• pyramid_calc_l2_reads_layer_zero_directly_in_standard_mode
• pyramid_calc_l2_sums_stratum_estimates_in_terminal_mode

pyramid_top_layer_complete_tracks_the_top_heap covers the new accessor.

Monte Carlo tests run in under a second by default. Their _long variants are #[ignore]d.

• pyramid_calc_l2_meets_its_documented_bound: 150 trials, or 3000 in the long variant. It failed before the fix (39/150 misses) and passes after it.
• pyramid_calc_card_meets_its_documented_bound: 60 trials, or 2000 in the long variant, with 3x256 elephant and 2x32 mouse layers. It checks the miss rate against the Chebyshev bound, that the variance is at most 2 F0^2/k, and that the estimate is unbiased (mean error within four standard errors).

One e2e assertion is adjusted. tests/e2e/frameworks.rs::univmon_pyramid_weighted_metrics checked the distinct count against a 20% band with with_defaults() (heap 32). At heap 32 the unbiased estimator's standard deviation can reach sqrt(2/32) = 25% of F0. On that test's stream the estimate came out at -26%; the old filtered estimator happened to fall inside the band. Following what #221 did for UnivMon's documented matrix, the test now uses the default tiers with heap 128 and asserts top_layer_complete(). It passes.

Run for this PR:

• cargo fmt --check
• cargo clippy --workspace --all-targets -D warnings with all features, default features, octo-runtime and experimental
• cargo doc --all-features with -D warnings
• cargo test --lib with the same four feature sets
• the pyramid tests including the ignored long variants

The full e2e suite (tests/e2e) was not run for this PR. The only e2e test run was univmon_pyramid_weighted_metrics.

Evidence types: screenshots and performance measurements do not apply. Before/after behavior comes from the tests above.

Limitations

• As in fix(univmon)!: every query is the paper's Algorithm 2 G-sum #221, terminal mode reconstructs its candidate sets from the strata, so the distinct-count bound does not cover it. The terminal-mode L2 bound is a union bound over the strata, and the narrower mouse strata have a larger delta.
• The Monte Carlo tests use Zipf streams only. fix(univmon)!: every query is the paper's Algorithm 2 G-sum #221's UnivMon tests also cover uniform and single-heavy-key streams.

[zzylol]

Closing: UnivMonPyramid (univmon_optimized.rs) was removed from main (#240).