refactor(elastic)!: ASAPv1 only — drop the proto format, add goldens

CHANGELOG.md

@@ -66,6 +66,8 @@ signals a backwards-compatible change.
   worker's seed and the aggregator's from one seed. The unseeded constructors
   draw from the thread generator. The seed is not serialized, and the ASAPv1
   payload is unchanged.
+- **ASAPv1 golden fixtures for Elastic**: `elastic_4b_2x4` and
+  `elastic_4b_2x4_stale` in `asapv1_golden/`, checked by `tests/asapv1_golden.rs`.
 
 ### Changed
 
@@ -88,6 +90,15 @@ signals a backwards-compatible change.
   as an empty window like any other, and that window takes a place in
   `max_windows`.
 
+### Removed
+
+- **BREAKING: Elastic's protobuf form.** `proto/elasticsketch/elasticsketch.proto`,
+  its message `ElasticState` (`asap_sketchlib::proto::sketchlib::ElasticState`),
+  and the `SketchEnvelope` oneof variant `elastic`
+  (`sketch_envelope::SketchState::Elastic`, field 18, now reserved) are gone.
+  `Elastic::serialize_to_bytes` / `deserialize_from_bytes` (ASAPv1, `0x0b 0x00`)
+  is Elastic's only serialization.
+
 ### Fixed
 
 - `KLLDynamic` panicked on every quantile query once a NaN had entered the

docs/asapv1_wire_format.md

@@ -1341,14 +1341,14 @@ Good direction (more compact, higher fidelity, less Rust-internal duplication),
 2. **Golden byte-vector fixtures** in one shared repo, [`sketchlib-golden-bytes`](https://github.com/ProjectASAP/sketchlib-golden-bytes), which each implementation mounts at `asapv1_golden/`; both languages decode and re-encode them byte-identically. These replace the `portable`-as-oracle round-trip test.
 3. **This registry**, mirrored, never independently allocated.
 
-Fixtures exist for six `kind_id`s — HLL's three estimators, Count-Min, Count Sketch and compact KLL — and `sketchlib-go` mirrors those. Every other kind here has a payload and **no fixture**, so this document is its only contract; `asapv1_golden/README.md` lists the gap. `portable` carries its own Go goldens.
+`asapv1_golden/README.md`'s Coverage section lists which `kind_id`s have fixtures; for every other kind this document is the only contract. `portable` carries its own Go goldens.
 
 **Hash profile on the Go side.**
 Rust derives the hash spec from a generic `HashProfile` bound on the hasher type; Go has no generic hasher type, so there is nothing to derive from.
 On the Go side the profile is simply **written into** the metadata on encode and **read from** it on decode.
 Go MUST validate the profile it reads (same fail-closed intent as Rust): a sketch is only mergeable/queryable if its `hash_profile_id` + seeds match the profile Go is prepared to reproduce.
 
-Sequencing: (2) covers six `kind_id`s, and the one `native bytes == portable bytes` test is HLL-only, since no other `portable` type emits ASAPv1 bytes.
+Sequencing: (2) covers the kinds that README lists, and the one `native bytes == portable bytes` test is HLL-only, since no other `portable` type emits ASAPv1 bytes.
 Retire `portable` once the fixtures cover the rest.
 
 ---

docs/features.md

@@ -112,7 +112,7 @@ MessagePack (`rmp-serde`) support. **serde support** means the type derives `Ser
 | EHSketchList | Yes | Yes |
 | ExponentialHistogram | No | Yes |
 
-Protobuf (prost): `.proto` definitions exist for CountMin, Count, HLL, DDSketch, KLL, Elastic, Coco, Hydra, and UnivMon. Rust conversion code is in progress.
+Protobuf (prost): `.proto` definitions exist for CountMin, Count, HLL, DDSketch, KLL, Coco, Hydra, and UnivMon. Rust conversion code is in progress.
 
 ### API Stability
 

docs/message_pack_format.md

@@ -169,10 +169,9 @@ decode → re-encode them byte-identically. The `kind_id` registry is mirrored
 verbatim with Go's `wire/asapmsgpack/magic_ids.go`, never independently
 allocated.
 
-That proof covers the six `kind_id`s a fixture exists for — HLL's three
-estimators, Count-Min, Count Sketch and compact KLL. Every other implemented
-kind has **no ASAPv1 golden and therefore no ASAPv1 drift guard**;
-[its README](https://github.com/ProjectASAP/sketchlib-golden-bytes#coverage) lists what is covered.
+That proof covers only the `kind_id`s a fixture exists for, which
+[its README's Coverage section](https://github.com/ProjectASAP/sketchlib-golden-bytes#coverage)
+lists; every other implemented kind has **no ASAPv1 drift guard**.
 The older `portable` path is guarded separately, by the `sketchlib-go` goldens
 of its own listed above.
 

docs/tests.md

@@ -2126,6 +2126,8 @@ Test file: [`tests/asapv1_golden.rs`](../tests/asapv1_golden.rs)
 | `cs_i32_regular_2x4_matches_golden` | The i32 counter type reaches the metadata, and an i32 golden is not an i64 sketch. | Verifies `Count::<Vector2D<i32>, RegularPath>::from_storage` over `[[0, 127, 128, 65536], [-1, -33, -32768, -2147483648]]` cast to `i32` serializes to `asapv1_golden/cs_i32_regular_2x4.hex`, that decoding it recovers the eight cells and re-serializes identically, that the i32 and i64 regular fixtures have equal length yet differ, and that the i32 fixture fails to decode as `Count<Vector2D<i64>, RegularPath>`. |
 | `cs_i64_fast_2x4_matches_golden` | The fast-path mode reaches the bytes, so the same matrix encodes differently per path. | Verifies `Count::<Vector2D<i64>, FastPath>::from_storage` over `[[0, 127, 128, 65536], [-1, -33, -32768, -2147483648]]` serializes to `asapv1_golden/cs_i64_fast_2x4.hex`, that decoding it recovers the eight cells and re-serializes identically, and that the fixture differs from `cs_i64_regular_2x4.hex` built from the same matrix. |
 | `cs_i64_regular_2x4_matches_golden` | A known signed Count-Sketch matrix serializes to its checked-in golden bytes. | Verifies `Count::<Vector2D<i64>, RegularPath>::from_storage` over `[[0, 127, 128, 65536], [-1, -33, -32768, -2147483648]]` — sweeping the msgpack integer widths in both directions — serializes to `asapv1_golden/cs_i64_regular_2x4.hex`, and that decoding the fixture reports `rows()` `2`, `cols()` `4`, the eight cells, and a byte-identical re-serialization. |
+| `elastic_4b_2x4_matches_golden` | A known Elastic heavy table and light layer serialize to their golden bytes. | Sets a 4-bucket `2x4` sketch's buckets to a free one carrying the eviction flag, a 31-byte flow id with votes `127`/`128`, an empty flow id with votes `1`/`65535` and the eviction flag, and a 32-byte flow id with votes `i32::MAX`/`256` and the flag, and its light cells to `[[0, 255, 65536, i32::MAX], [-1, -33, -32768, i32::MIN]]`; verifies the bytes equal `asapv1_golden/elastic_4b_2x4.hex`, and that decoding the fixture recovers every bucket, `bktlen` `4`, the light dimensions and cells, and a byte-identical re-serialization. |
+| `elastic_4b_2x4_stale_matches_golden` | `stale_copies` reaches the bytes, and nothing else differs. | Builds the same state on a 2-bucket sketch after `expand_heavy()`, verifies the bytes equal `asapv1_golden/elastic_4b_2x4_stale.hex`, that decoding recovers the same state and re-serializes identically, and that the two fixtures have equal length and differ in exactly one byte, `0xc2` against `0xc3`. |
 | `hll_classic_p12_matches_golden` | A known P12 Classic register pattern serializes to its golden bytes on both the portable and native paths. | Builds `HllSketch::from_raw(HllVariant::Regular, 12, regs, 0.0, 0.0, 0.0)` where `regs` is 4,096 zeros with `[0] = 1`, `[1] = 7`, `[100] = 42` and `[4095] = 3`, verifies `to_msgpack()` equals `asapv1_golden/hll_classic_p12.hex`, that `HyperLogLogP12::<Classic>::deserialize_from_bytes` on the fixture recovers those registers and re-serializes identically, and that `HllSketch::from_msgpack` reports the same registers, `precision` `12` and `HllVariant::Regular`. |
 | `hll_ertl_mle_p12_matches_golden` | The Datafusion variant tag reaches the bytes for the same P12 register pattern. | Builds `HllSketch::from_raw(HllVariant::Datafusion, 12, regs, 0.0, 0.0, 0.0)` over the shared P12 pattern (`[0] = 1`, `[1] = 7`, `[100] = 42`, `[4095] = 3` in 4,096 registers), verifies `to_msgpack()` equals `asapv1_golden/hll_ertl_mle_p12.hex`, that `HyperLogLogP12::<ErtlMLE>::deserialize_from_bytes` recovers the registers and re-serializes identically, and that `HllSketch::from_msgpack` reports `HllVariant::Datafusion`. |
 | `hll_hip_p12_matches_golden` | The HIP variant's three running scalars travel on the wire beside its registers. | Builds `HllSketch::from_raw(HllVariant::Hip, 12, regs, 1.5, 2.5, 3.0)` over the shared P12 pattern, verifies `to_msgpack()` equals `asapv1_golden/hll_hip_p12.hex`, that `HyperLogLogHIPP12::deserialize_from_bytes` re-serializes the fixture identically, and that `HllSketch::from_msgpack` reports the registers, `HllVariant::Hip`, `hip_kxq0` `1.5`, `hip_kxq1` `2.5` and `hip_est` `3.0`. |
@@ -2202,7 +2204,7 @@ Test file: [`tests/xtest_consumer.rs`](../tests/xtest_consumer.rs)
 
 | test_name | test_description | what_is_tested |
 | --- | --- | --- |
-| `cross_language_proto` | Protobuf envelopes read from the cross-language fixture directory answer their point queries in band. | Asserts nothing unless fixtures are available: with no `XTEST_DIR` and no discoverable producer directory holding `tests/cross_language/xtest_producer_test.go`, it prints a skip line and returns. With fixtures it decodes nine `SketchEnvelope` files and records a pass or failure for each — `countmin.pb` min-frequency for `item:42` at least `101`, `kll.pb` p50 within 5% of `5000` and p99 within 5% of `9900`, `ddsketch.pb` the same two quantiles within 2%, `hll.pb` an ErtlMLE cardinality inside `[40_000, 65_000]`, `countsketch.pb` `cs:hot` at least `200`, `coco.pb` `coco:hot` at least `500`, `elastic.pb` `elephant` at least `900`, `univmon.pb` a cardinality inside `[1_000, 15_000]`, `hydra.pb` `hydra:42` at least `51` — using the hash seeds `0xcafe3553`, `0x6a09e667` and `0xbb67ae85`, and, when `countmin_sampled.pb` and `hll_sampled.pb` exist, requires `effective_sample_p` of `0.1` and a `rescale_count` by `1/p` within 5% of `100_000` and 6% of `200_000`. A single `assert!` at the end fails if any check failed; a missing or mistyped envelope panics on the spot. |
+| `cross_language_proto` | Protobuf envelopes read from the cross-language fixture directory answer their point queries in band. | Asserts nothing unless fixtures are available: with no `XTEST_DIR` and no discoverable producer directory holding `tests/cross_language/xtest_producer_test.go`, it prints a skip line and returns. With fixtures it decodes eight `SketchEnvelope` files and records a pass or failure for each — `countmin.pb` min-frequency for `item:42` at least `101`, `kll.pb` p50 within 5% of `5000` and p99 within 5% of `9900`, `ddsketch.pb` the same two quantiles within 2%, `hll.pb` an ErtlMLE cardinality inside `[40_000, 65_000]`, `countsketch.pb` `cs:hot` at least `200`, `coco.pb` `coco:hot` at least `500`, `univmon.pb` a cardinality inside `[1_000, 15_000]`, `hydra.pb` `hydra:42` at least `51` — using the hash seeds `0xcafe3553` and `0xbb67ae85`, and, when `countmin_sampled.pb` and `hll_sampled.pb` exist, requires `effective_sample_p` of `0.1` and a `rescale_count` by `1/p` within 5% of `100_000` and 6% of `200_000`. A single `assert!` at the end fails if any check failed; a missing or mistyped envelope panics on the spot. |
 
 ### HLL Custom Precision
 

proto/sketchlib.proto

@@ -37,7 +37,6 @@ import "ddsketch/ddsketch.proto";
 import "univmon/univmon.proto";
 import "hydra/hydra.proto";
 import "cocosketch/cocosketch.proto";
-import "elasticsketch/elasticsketch.proto";
 
 option go_package = "github.com/ProjectASAP/sketchlib-go/proto/sketch_envelope;sketch_envelopepb";
 
@@ -74,9 +73,11 @@ message SketchEnvelope {
     UnivMonState     univmon      = 15;
     HydraState       hydra        = 16;
     CocoSketchState  coco         = 17;
-    ElasticState     elastic      = 18;
   }
 
+  reserved 18;
+  reserved "elastic";
+
   reserved 19 to 31;  // Reserved for future sketch types.
 
   // Geometric / NitroSketch sampling probability for the producing sketch.

src/proto/generated/sketchlib.v1.rs

@@ -823,33 +823,6 @@ pub struct CocoSketchState {
     #[prost(bool, repeated, tag = "5")]
     pub has_keys: ::prost::alloc::vec::Vec<bool>,
 }
-/// ElasticState is the portable state of an Elastic Sketch.
-///
-/// The Elastic Sketch has a heavy part (one bucket per index) and a light
-/// Count-Min layer. The heavy part is serialized as parallel arrays.
-/// The light layer is stored as a CountMinState with FLOAT64 counters.
-#[derive(Clone, PartialEq, ::prost::Message)]
-pub struct ElasticState {
-    /// Number of heavy buckets.
-    #[prost(uint32, tag = "1")]
-    pub bucket_count: u32,
-    /// Per-bucket flow identifiers (strings), length = bucket_count.
-    /// Empty string means the bucket is unoccupied.
-    #[prost(string, repeated, tag = "2")]
-    pub flow_ids: ::prost::alloc::vec::Vec<::prost::alloc::string::String>,
-    /// Positive vote counts per bucket, length = bucket_count.
-    #[prost(sint32, repeated, tag = "3")]
-    pub vote_pos: ::prost::alloc::vec::Vec<i32>,
-    /// Negative vote counts per bucket, length = bucket_count.
-    #[prost(sint32, repeated, tag = "4")]
-    pub vote_neg: ::prost::alloc::vec::Vec<i32>,
-    /// Whether each bucket is in eviction state, length = bucket_count.
-    #[prost(bool, repeated, tag = "5")]
-    pub evictions: ::prost::alloc::vec::Vec<bool>,
-    /// Light Count-Min layer (FLOAT64 counters).
-    #[prost(message, optional, tag = "6")]
-    pub light: ::core::option::Option<CountMinState>,
-}
 /// SketchEnvelope is the portable container for all sketch transfers.
 /// Producers fill exactly one field inside sketch_state.
 /// Consumers check format_version first, then dispatch on the oneof.
@@ -892,7 +865,7 @@ pub struct SketchEnvelope {
     /// The sketch payload. Exactly one field must be set.
     #[prost(
         oneof = "sketch_envelope::SketchState",
-        tags = "10, 11, 12, 13, 14, 15, 16, 17, 18"
+        tags = "10, 11, 12, 13, 14, 15, 16, 17"
     )]
     pub sketch_state: ::core::option::Option<sketch_envelope::SketchState>,
 }
@@ -917,7 +890,5 @@ pub mod sketch_envelope {
         Hydra(super::HydraState),
         #[prost(message, tag = "17")]
         Coco(super::CocoSketchState),
-        #[prost(message, tag = "18")]
-        Elastic(super::ElasticState),
     }
 }