refactor(univmon)!: ASAPv1 only — drop the proto envelope field, add goldens

CHANGELOG.md

@@ -66,6 +66,9 @@ signals a backwards-compatible change.
   worker's seed and the aggregator's from one seed. The unseeded constructors
   draw from the thread generator. The seed is not serialized, and the ASAPv1
   payload is unchanged.
+- **ASAPv1 golden fixtures for `UnivMon`** (kind_id `0x10 0x00`):
+  `univmon_str_l3_2x4_h5`, `univmon_i64_l3_2x4_h5` and
+  `univmon_empty_l3_2x4_h5`, checked in `tests/asapv1_golden.rs`.
 
 ### Changed
 
@@ -88,6 +91,15 @@ signals a backwards-compatible change.
   as an empty window like any other, and that window takes a place in
   `max_windows`.
 
+### Removed
+
+- **UnivMon's proto format.** `UnivMon` serializes only as ASAPv1
+  (`serialize_to_bytes` / `deserialize_from_bytes`, kind_id `0x10 0x00`).
+  Removed the `univmon` field (15) of `SketchEnvelope` (now reserved) and with
+  it `sketch_envelope::SketchState::Univmon`; `UnivMonState` and
+  `UnivMonLayer` stay in `proto/univmon/univmon.proto` only as
+  `HydraCell.univmon`.
+
 ### Fixed
 
 - `KLLDynamic` panicked on every quantile query once a NaN had entered the

docs/asapv1_wire_format.md

@@ -972,7 +972,7 @@ The encode side enforces rules 3 and 4 as well, and rejects a sketch holding the
 
 **A layer's `CountL2HH` is inlined, not nested.** The layer's `counts` and `l2` are appended to the outer sketch's own positional arrays, and its `rows` / `cols` / `seed_index` come from the outer metadata plus the layer's position. No nested envelope, magic, version or metadata map is emitted per layer.
 
-**Metadata vs payload.** `layer_size`, `sketch_row`, `sketch_col` and `heap_size` are `init_univmon`'s four arguments — configuration that shapes the payload, so the descriptor carries them and every derived length is checked rather than re-stored. `key_type` is a structural param in the §3.5 sense: it fixes the element type of `keys`, and the payload cannot be read without it. One `key_type` covers every layer; a pyramid whose layers mix `HeapItem` variants has no single one and fails to serialize, as a Space-Saving summary does. Structural-param order is `layer_size, sketch_row, sketch_col, heap_size, key_type`.
+**Metadata vs payload.** `layer_size`, `sketch_row`, `sketch_col` and `heap_size` are `init_univmon`'s four arguments — configuration that shapes the payload, so the descriptor carries them and every derived length is checked rather than re-stored. `key_type` is a structural param in the §3.5 sense: it fixes the element type of `keys`, and the payload cannot be read without it. One `key_type` covers every layer; a pyramid whose layers mix `HeapItem` variants has no single one and fails to serialize, as a Space-Saving summary does. Structural-param order is `layer_size, sketch_row, sketch_col, heap_size, key_type`. A pyramid with no heap entries emits `key_type = "u64"`.
 
 UnivMon carries the hash-spec group — it hashes, and a consumer must reproduce that hash to query a key — but carries **no seed-index key**. It hashes the bottom-layer finder at `BOTTOM_LAYER_FINDER` unconditionally: a fixed part of the algorithm, not a profile choice, exactly as Space-Saving's index 0 is (§3.5). The per-layer counter seed index *is* a real value, but it equals the layer's position, so it is derived and not stored, and the encoder rejects a layer whose seed index is not its position. `UnivMon` has no hasher type parameter, so its metadata is derived from `DefaultXxHasher`'s `HashProfile` — read live, never hardcoded — and a custom-profile envelope is different bytes and is rejected on decode.
 
@@ -1341,14 +1341,14 @@ Good direction (more compact, higher fidelity, less Rust-internal duplication),
 2. **Golden byte-vector fixtures** in one shared repo, [`sketchlib-golden-bytes`](https://github.com/ProjectASAP/sketchlib-golden-bytes), which each implementation mounts at `asapv1_golden/`; both languages decode and re-encode them byte-identically. These replace the `portable`-as-oracle round-trip test.
 3. **This registry**, mirrored, never independently allocated.
 
-Fixtures exist for six `kind_id`s — HLL's three estimators, Count-Min, Count Sketch and compact KLL — and `sketchlib-go` mirrors those. Every other kind here has a payload and **no fixture**, so this document is its only contract; `asapv1_golden/README.md` lists the gap. `portable` carries its own Go goldens.
+`asapv1_golden/README.md`'s Coverage section lists which `kind_id`s have fixtures; for every other kind, this document is the only contract. `portable` carries its own Go goldens.
 
 **Hash profile on the Go side.**
 Rust derives the hash spec from a generic `HashProfile` bound on the hasher type; Go has no generic hasher type, so there is nothing to derive from.
 On the Go side the profile is simply **written into** the metadata on encode and **read from** it on decode.
 Go MUST validate the profile it reads (same fail-closed intent as Rust): a sketch is only mergeable/queryable if its `hash_profile_id` + seeds match the profile Go is prepared to reproduce.
 
-Sequencing: (2) covers six `kind_id`s, and the one `native bytes == portable bytes` test is HLL-only, since no other `portable` type emits ASAPv1 bytes.
+Sequencing: the one `native bytes == portable bytes` test is HLL-only, since no other `portable` type emits ASAPv1 bytes.
 Retire `portable` once the fixtures cover the rest.
 
 ---

docs/features.md

@@ -112,7 +112,7 @@ MessagePack (`rmp-serde`) support. **serde support** means the type derives `Ser
 | EHSketchList | Yes | Yes |
 | ExponentialHistogram | No | Yes |
 
-Protobuf (prost): `.proto` definitions exist for CountMin, Count, HLL, DDSketch, KLL, Elastic, Coco, Hydra, and UnivMon. Rust conversion code is in progress.
+Protobuf (prost): `SketchEnvelope` carries CountMin, Count, HLL, DDSketch, KLL, Elastic, Coco, and Hydra state; `UnivMonState` exists only inside `HydraCell`. Rust conversion code is in progress.
 
 ### API Stability
 

docs/message_pack_format.md

@@ -169,10 +169,8 @@ decode → re-encode them byte-identically. The `kind_id` registry is mirrored
 verbatim with Go's `wire/asapmsgpack/magic_ids.go`, never independently
 allocated.
 
-That proof covers the six `kind_id`s a fixture exists for — HLL's three
-estimators, Count-Min, Count Sketch and compact KLL. Every other implemented
-kind has **no ASAPv1 golden and therefore no ASAPv1 drift guard**;
-[its README](https://github.com/ProjectASAP/sketchlib-golden-bytes#coverage) lists what is covered.
+That proof covers only the `kind_id`s with a fixture, which
+[its README's Coverage section](https://github.com/ProjectASAP/sketchlib-golden-bytes#coverage) lists.
 The older `portable` path is guarded separately, by the `sketchlib-go` goldens
 of its own listed above.
 

docs/tests.md

@@ -2131,6 +2131,9 @@ Test file: [`tests/asapv1_golden.rs`](../tests/asapv1_golden.rs)
 | `hll_hip_p12_matches_golden` | The HIP variant's three running scalars travel on the wire beside its registers. | Builds `HllSketch::from_raw(HllVariant::Hip, 12, regs, 1.5, 2.5, 3.0)` over the shared P12 pattern, verifies `to_msgpack()` equals `asapv1_golden/hll_hip_p12.hex`, that `HyperLogLogHIPP12::deserialize_from_bytes` re-serializes the fixture identically, and that `HllSketch::from_msgpack` reports the registers, `HllVariant::Hip`, `hip_kxq0` `1.5`, `hip_kxq1` `2.5` and `hip_est` `3.0`. |
 | `kll_f64_k200_matches_golden` | A deterministic f64 KLL serializes to its golden bytes, which round-trip unchanged. | Verifies `KLL::<f64>::init_kll_with_seed(200, 42)` fed `1..=50` (below the level-0 capacity, so no compaction fires) serializes to `asapv1_golden/kll_f64_k200.hex`, and that decoding the fixture re-serializes to the same bytes and reports `quantile(0.0)` `1.0` and `quantile(1.0)` `50.0`. |
 | `kll_i64_k200_matches_golden` | A deterministic i64 KLL serializes to its golden bytes, which round-trip unchanged. | Verifies `KLL::<i64>::init_kll_with_seed(200, 42)` fed `1..=50` serializes to `asapv1_golden/kll_i64_k200.hex`, and that decoding the fixture re-serializes to the same bytes and reports `quantile(0.0)` `1.0` and `quantile(1.0)` `50.0`. |
+| `univmon_empty_l3_2x4_h5_matches_golden` | A freshly constructed pyramid has one encoding, pinned by its golden. | Verifies `UnivMon::init_univmon(5, 2, 4, 3)` serializes to `asapv1_golden/univmon_empty_l3_2x4_h5.hex`; reading the envelope by hand gives kind_id `0x10 0x00`, shape `(3, 2, 4, 5)`, `key_type` `u64`, 24 zero counts, 6 zero `l2` accumulators, `heap_lens` `[0, 0, 0]`, no keys or heap counts, candidate flags `[true, true, true]`, `bucket_size` `0` and `update_mode` `0`; the decode matches the source's shape, weight, flags, counters, per-layer `get_l2()` and heaps, and re-serializes identically. |
+| `univmon_i64_l3_2x4_h5_matches_golden` | The `i64` key type reaches the bytes, and only it differs from the string fixture. | Builds the same layers as the string fixture with heap keys `i64::MIN`, `-1`, `-129` on layer 0, `128` on layer 1 and `4294967296`, `7` on layer 2, verifies the bytes equal `asapv1_golden/univmon_i64_l3_2x4_h5.hex`, that the hand-read envelope carries shape `(3, 2, 4, 5)` and those keys under `key_type` `i64` with the same counts, `l2`, heap lengths, heap counts, flags, weight and mode, that the decode matches the source and re-serializes identically, that layer 0's heap holds `HeapItem::I64(i64::MIN)`, and that the fixture differs from the string one. |
+| `univmon_str_l3_2x4_h5_matches_golden` | A known 3-layer pyramid with string keys serializes to its golden bytes and decodes back to that state. | Builds `UnivMon::init_univmon(5, 2, 4, 3)` (heap 5, 2 rows, 4 columns, 3 layers: pairwise distinct), writes layer 0's cells `[[0, 127, 128, 65536], [-1, -33, -32768, -2147483648]]`, layer 1's `[[3, -2, 0, 1], [0, 0, 5, -4]]` and layer 2's `[[0, 7, 0, 0], [-6, 0, 0, 0]]` through `L2HH::apply_delta`, seats `alpha`=`65536`, `beta`=`300`, `delta`=`128` in layer 0's heap, `gamma`=`5` in layer 1's and `epsilon`=`9`, `zeta`=`2` in layer 2's, calls `set_total_weight(70_000)` and `mark_layer_candidates_incomplete(1)`, and verifies the bytes equal `asapv1_golden/univmon_str_l3_2x4_h5.hex`. Reading the envelope by hand gives kind_id `0x10 0x00`, shape `(3, 2, 4, 5)`, `key_type` `string`, the 24 cells in layer then row-major order, `l2` `[4294999809, 4611686019501130818, 14, 41, 49, 36]`, `heap_lens` `[3, 1, 2]`, the six keys and heap counts `[65536, 300, 128, 5, 9, 2]` in that order, flags `[true, false, true]`, `bucket_size` `70000` and `update_mode` `1`; the decode matches the source and re-serializes identically. |
 
 ### MessagePack Envelope Compatibility
 
@@ -2202,7 +2205,7 @@ Test file: [`tests/xtest_consumer.rs`](../tests/xtest_consumer.rs)
 
 | test_name | test_description | what_is_tested |
 | --- | --- | --- |
-| `cross_language_proto` | Protobuf envelopes read from the cross-language fixture directory answer their point queries in band. | Asserts nothing unless fixtures are available: with no `XTEST_DIR` and no discoverable producer directory holding `tests/cross_language/xtest_producer_test.go`, it prints a skip line and returns. With fixtures it decodes nine `SketchEnvelope` files and records a pass or failure for each — `countmin.pb` min-frequency for `item:42` at least `101`, `kll.pb` p50 within 5% of `5000` and p99 within 5% of `9900`, `ddsketch.pb` the same two quantiles within 2%, `hll.pb` an ErtlMLE cardinality inside `[40_000, 65_000]`, `countsketch.pb` `cs:hot` at least `200`, `coco.pb` `coco:hot` at least `500`, `elastic.pb` `elephant` at least `900`, `univmon.pb` a cardinality inside `[1_000, 15_000]`, `hydra.pb` `hydra:42` at least `51` — using the hash seeds `0xcafe3553`, `0x6a09e667` and `0xbb67ae85`, and, when `countmin_sampled.pb` and `hll_sampled.pb` exist, requires `effective_sample_p` of `0.1` and a `rescale_count` by `1/p` within 5% of `100_000` and 6% of `200_000`. A single `assert!` at the end fails if any check failed; a missing or mistyped envelope panics on the spot. |
+| `cross_language_proto` | Protobuf envelopes read from the cross-language fixture directory answer their point queries in band. | Asserts nothing unless fixtures are available: with no `XTEST_DIR` and no discoverable producer directory holding `tests/cross_language/xtest_producer_test.go`, it prints a skip line and returns. With fixtures it decodes eight `SketchEnvelope` files and records a pass or failure for each — `countmin.pb` min-frequency for `item:42` at least `101`, `kll.pb` p50 within 5% of `5000` and p99 within 5% of `9900`, `ddsketch.pb` the same two quantiles within 2%, `hll.pb` an ErtlMLE cardinality inside `[40_000, 65_000]`, `countsketch.pb` `cs:hot` at least `200`, `coco.pb` `coco:hot` at least `500`, `elastic.pb` `elephant` at least `900`, `hydra.pb` `hydra:42` at least `51` — using the hash seeds `0xcafe3553`, `0x6a09e667` and `0xbb67ae85`, and, when `countmin_sampled.pb` and `hll_sampled.pb` exist, requires `effective_sample_p` of `0.1` and a `rescale_count` by `1/p` within 5% of `100_000` and 6% of `200_000`. A single `assert!` at the end fails if any check failed; a missing or mistyped envelope panics on the spot. |
 
 ### HLL Custom Precision
 

proto/sketchlib.proto

@@ -9,7 +9,6 @@
 //   hll/hll.proto                    — HyperLogLogState, HLLVariant
 //   kll/kll.proto                    — KLLState, CoinState
 //   ddsketch/ddsketch.proto          — DDSketchState
-//   univmon/univmon.proto            — UnivMonState, UnivMonLayer
 //   hydra/hydra.proto                — HydraState, HydraCounterType, HydraCell
 //
 // Design goals:
@@ -34,7 +33,6 @@ import "countsketch/countsketch.proto";
 import "hll/hll.proto";
 import "kll/kll.proto";
 import "ddsketch/ddsketch.proto";
-import "univmon/univmon.proto";
 import "hydra/hydra.proto";
 import "cocosketch/cocosketch.proto";
 import "elasticsketch/elasticsketch.proto";
@@ -71,12 +69,13 @@ message SketchEnvelope {
     HyperLogLogState hll          = 12;
     KLLState         kll          = 13;
     DDSketchState    ddsketch     = 14;
-    UnivMonState     univmon      = 15;
     HydraState       hydra        = 16;
     CocoSketchState  coco         = 17;
     ElasticState     elastic      = 18;
   }
 
+  reserved 15;
+  reserved "univmon";
   reserved 19 to 31;  // Reserved for future sketch types.
 
   // Geometric / NitroSketch sampling probability for the producing sketch.

src/proto/generated/sketchlib.v1.rs

@@ -892,7 +892,7 @@ pub struct SketchEnvelope {
     /// The sketch payload. Exactly one field must be set.
     #[prost(
         oneof = "sketch_envelope::SketchState",
-        tags = "10, 11, 12, 13, 14, 15, 16, 17, 18"
+        tags = "10, 11, 12, 13, 14, 16, 17, 18"
     )]
     pub sketch_state: ::core::option::Option<sketch_envelope::SketchState>,
 }
@@ -911,8 +911,6 @@ pub mod sketch_envelope {
         Kll(super::KllState),
         #[prost(message, tag = "14")]
         Ddsketch(super::DdSketchState),
-        #[prost(message, tag = "15")]
-        Univmon(super::UnivMonState),
         #[prost(message, tag = "16")]
         Hydra(super::HydraState),
         #[prost(message, tag = "17")]