[zzylol]

Part of #147 (first work item: record measurement conditions on each row).

Why

The per-phase cost model (#145) applies each cost-table row to real deployments whose instances look nothing like the benchmark's: a window of λ·x/card(G) items instead of 1M, latencies in seconds instead of zipf ranks, card(G) keys instead of 100k. The row says nothing about the conditions it was measured under, so the optimizer can neither rescale it nor tell how far it is extrapolating.

What

Every row of approxbench atomic-costs output (rqe_atomic_costs.json, and the ASAPQuery table from export_atomic_costs.sh) gains a measured_at block. No new measurements: it is derived from the workload description already in each grid record. Nothing consumes it yet; the size sweep and model-side work in #147 will.

How

reduce_one reads the record's workload. In a table description the last column is the value and the ones before it are keys; a one-column table's column is both.

Field	Source
items_per_instance	row_num (synthetic) or records_loaded (external)
keys_per_instance	measured groups_per_instance when the comparator reports it; else the product of the key columns' populations — an upper bound on keys actually drawn
value_range	value column's [lo, hi] (zipf [1, population], uniform [lo, hi]) plus shift; null for normal/Pareto and for monotonic (counter) columns, whose values are a running sum of draws
merge_operand_items	items_per_instance / merge_shards of the cost pass
distribution	the value column's distribution verbatim (kind, skew, population, seed)

measured_at is Option with #[serde(default)], so tables written before it still load.

Before this PR

A row carried only identity, four costs and accuracy:

{"sketch": "kll-percall", "sketch_config": {"algorithm": "kll-percall", "params": {"k": 200}},
 "mem_bytes_per_instance": 6400.0, "insert_cpu_secs": 2.5e-08, "merge_cpu_secs": 6.76e-06,
 "query_cpu_secs": 6.5e-06, "query_accuracy": {"mean_rank_err": 0.00164, "max_rank_err": 0.00734, ...}}

Whether those 6.4 KB and 6.5 µs were measured on 1M or 1k items, on zipf ranks or latencies, was only recoverable by digging the workload out of rqe_atomic_costs_grid.jsonl by hand.

After this PR

The same row, plus:

"measured_at": {
  "items_per_instance": 1000000,
  "keys_per_instance": 100000,
  "value_range": [1.0, 100000.0],
  "merge_operand_items": 62500,
  "distribution": {"kind": "zipf", "skewness": 1.1, "population_size": 100000, "seed": 42}
}

A grouped exact row reports the measured group count, and a counter column no range:

"sketch": "exact-increase", ...
"measured_at": {"items_per_instance": 200000, "keys_per_instance": 9950, "value_range": null,
                "merge_operand_items": 12500,
                "distribution": {"kind": "uniform", "lower_bound": 0.0, "upper_bound": 10.0, "seed": 3}}

Evidence

Ran this branch's reducer on the 2026-10-06 export (out_10_6_26_1342/rqe_atomic_costs_grid.jsonl, 18 rows). Every row gets measured_at:

Rows	items	keys	value_range	operand
exact-sum/min/max	200,000	9,950 (measured)	[1, 1000]	12,500
exact-increase	200,000	9,950 (measured)	null (counter)	12,500
kll, dd, hll, cms/cs-heap, univmon-topk	1,000,000	100,000 (bound)	[1, 100000]	62,500
univmon-cardinality	200,000	100,000 (bound)	[1, 100]	12,500

The 100k key bound is loose: the same export's HLL rows estimate ~62–64k distinct keys actually drawn.

Verification

• Unit tests (aqpbm-core/src/atomic_costs.rs):
  • measured_at_reads_a_single_column_workload: one column is both key and value; operand = items / shards.
  • measured_at_reads_a_labelled_workload: keys = product of label populations, shift moves the range, a measured group count wins over the bound.
  • measured_at_has_no_value_range_for_a_counter_column: a monotonic column records no range.
  • a_row_without_measured_at_still_loads: older tables deserialize.
• End-to-end tests: the export run above (reduction only; no new benchmark run).
• Other checks: cargo test --workspace (505 passed), cargo clippy --workspace --all-targets -- -D warnings.

Architectural decisions

• Per row, not per profile. ASAPQuery's document keeps the workload once per profile; this table mixes five datasets (zipf plus four column specs), so each row carries its own.
• Derived, not measured. All fields come from the existing workload description. Exact distinct-key counts would need a new measurement; the bound is documented instead.
• Always emitted. Rejected: an opt-in flag for the rqe export only. ASAPQuery's strict loader was updated instead (ASAPQuery#796, merged).
• Plain fields plus the distribution verbatim, no new enums for dataset kind or skew.

Limitations and follow-up

• keys_per_instance for unlabelled synthetic data is the population, an upper bound.
• value_range is the generator's domain, not the observed min/max.
• Consumers: the items-per-instance size sweep and model-side rescaling in Tracking: make the rqe-optimizer cost export fit the per-phase cost model #147.

Human review — do not complete with an agent

• The MVP boundary is correct.
• New conceptual layers or public interfaces are necessary.
• The before/after description matches the intended product behavior.
• Human reviewer:
• Decision and rationale:

🤖 Generated with Claude Code

[zzylol]

Grilling outcome (agreed with @zzylol):

• Q5: a monotonic (counter) value column now records value_range: None; its values are a running sum of draws. Found by running this reducer on the 10/6 export, where exact-increase showed [0, 10]. New test.
• Q6: keys_per_instance stays the documented upper bound (100k population vs ~62–64k distinct drawn, per the export's HLL rows); measured group counts still win.
• Q7: ASAPQuery's loader (deny_unknown_fields) gets an optional measured_at in a separate ASAPQuery PR, to merge before its table is regenerated.

Checked on the real export grid: all 18 rows get measured_at with the expected items / keys / operand sizes.

🤖 Generated with Claude Code