Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ signals a backwards-compatible change.
law that sketch claims — merge algebra, path equivalence, the paper's own
model, and the wire / ASAPv1 round trip. Shared generators, the `grid`
reader and the `round_trip!` macro live in `tests/pbt/support.rs`; four
modules sit behind `experimental`. 435 tests under `--all-features`, 367
modules sit behind `experimental`. 433 tests under `--all-features`, 365
under the default feature set. Every law was mutation-checked: the
implementation was broken deliberately, the law confirmed red, and confirmed
not to take unrelated laws down with it; a law no semantically real mutation
Expand All @@ -66,6 +66,13 @@ signals a backwards-compatible change.
worker's seed and the aggregator's from one seed. The unseeded constructors
draw from the thread generator. The seed is not serialized, and the ASAPv1
payload is unchanged.
- **HyperLogLog `from_storage` constructors.** `HyperLogLogImpl::from_storage(registers)`
and `HyperLogLogHIPImpl::from_storage(registers, kxq0, kxq1, est)` build a
sketch from known register storage (and HIP running state).
- **HLL P14 golden fixtures** `hll_classic_p14`, `hll_ertl_mle_p14` and
`hll_hip_p14` in `asapv1_golden/`, checked in both directions by
`tests/asapv1_golden.rs`. Their registers touch the first, a middle and the
last index and hold 51, the largest P14 rank.

### Changed

Expand All @@ -88,6 +95,20 @@ signals a backwards-compatible change.
as an empty window like any other, and that window takes a place in
`max_windows`.

### Removed

- **HyperLogLog's protobuf and portable formats.** HyperLogLog serializes only
as ASAPv1 (`serialize_to_bytes` / `deserialize_from_bytes`). Removed: the
portable `HllSketch`, `HllSketchDelta` and `HllVariant` (module
`message_pack_format::portable::hll` and their crate-root re-exports), with
`registers_from_state`, `decode_sparse_registers` and
`encode_sparse_registers`; the `MessagePackCodec` impls for
`HyperLogLogImpl` and `HyperLogLogHIPImpl` (`message_pack_format::native::hll`);
the `HLLDelta` proto message (`proto::sketchlib::HllDelta`); and the `hll`
field (12) of `SketchEnvelope`, now reserved. `HyperLogLogState`,
`HLLSparseRegisters` and `HLLVariant` remain only as the `HydraCell` HLL
state.

### Fixed

- `KLLDynamic` panicked on every quantile query once a NaN had entered the
Expand Down
2 changes: 2 additions & 0 deletions docs/api/api_hyperloglog.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ but improves accuracy.
```rust
fn new() -> Self
fn default() -> Self
fn from_storage(registers: Registers) -> Self // HyperLogLogImpl
fn from_storage(registers: Registers, kxq0: f64, kxq1: f64, est: f64) -> Self // HyperLogLogHIPImpl
```

## Insert / Update
Expand Down
2 changes: 1 addition & 1 deletion docs/asapv1_wire_format.md
Original file line number Diff line number Diff line change
Expand Up @@ -1348,7 +1348,7 @@ Rust derives the hash spec from a generic `HashProfile` bound on the hasher type
On the Go side the profile is simply **written into** the metadata on encode and **read from** it on decode.
Go MUST validate the profile it reads (same fail-closed intent as Rust): a sketch is only mergeable/queryable if its `hash_profile_id` + seeds match the profile Go is prepared to reproduce.

Sequencing: (2) covers six `kind_id`s, and the one `native bytes == portable bytes` test is HLL-only, since no other `portable` type emits ASAPv1 bytes.
Sequencing: (2) covers six `kind_id`s.
Retire `portable` once the fixtures cover the rest.

---
Expand Down
2 changes: 1 addition & 1 deletion docs/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ MessagePack (`rmp-serde`) support. **serde support** means the type derives `Ser
| EHSketchList | Yes | Yes |
| ExponentialHistogram | No | Yes |

Protobuf (prost): `.proto` definitions exist for CountMin, Count, HLL, DDSketch, KLL, Elastic, Coco, Hydra, and UnivMon. Rust conversion code is in progress.
Protobuf (prost): `.proto` definitions exist for CountMin, Count, DDSketch, KLL, Elastic, Coco, Hydra, and UnivMon. Rust conversion code is in progress.

### API Stability

Expand Down
2 changes: 1 addition & 1 deletion docs/library_map.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
- **`src/message_pack_format/`** - Serialization plumbing ([message_pack_format.md](./message_pack_format.md)). The current format is **ASAPv1**, specified in [asapv1_wire_format.md](./asapv1_wire_format.md)
- `envelope.rs` — the shared, sketch-agnostic ASAPv1 framing (magic/version/`kind_id` + length prefixes, `encode`/`split`); every `wire.rs` under `src/sketches/` and `src/sketch_framework/` calls into it
- `codec.rs` — the `MessagePackCodec` trait; `error.rs` — the unified `Error`
- `portable/` — **deprecated**, being retired. The older per-sketch wire types (`CountMinSketch`, `HllSketch`, …); ASAPv1 (per-sketch `wire.rs`) is now what `sketchlib-go` mirrors, not these
- `portable/` — **deprecated**, being retired. The older per-sketch wire types (`CountMinSketch`, `KllSketch`, …); ASAPv1 (per-sketch `wire.rs`) is now what `sketchlib-go` mirrors, not these
- `native/` — **deprecated**, being retired. Older `MessagePackCodec` shims over `src/sketches/` byte serialization

## Documentation
Expand Down
2 changes: 1 addition & 1 deletion docs/message_pack_format.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ being **phased out** in favor of the per-sketch `wire.rs` + the shared
`envelope.rs`.

- `portable/` holds per-sketch wire types that predate the envelope. They
carry their own byte-parity goldens against `sketchlib-go` — HLL, Count-Min,
carry their own byte-parity goldens against `sketchlib-go` — Count-Min,
Count Sketch and KLL (both directions) — separate from `asapv1_golden/`.
- `native/` is a set of thin `MessagePackCodec` shims over the sketches'
`serialize_to_bytes` / `deserialize_from_bytes`. Every type it wraps emits the
Expand Down
Loading
Loading