Skip to content

study: denser KLL/DDSketch grid and Hydra W 65536; §6.3 rerun - #213

Merged
zzylol merged 5 commits into
mainfrom
eval/dense-grid
Oct 14, 2026
Merged

zzylol merged 5 commits into
mainfrom
eval/dense-grid

Conversation

@zzylol

@zzylol zzylol commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Denser config grid, measured for the new configs only, and the §6.3 rerun on it, on #210's workload (42 / 62 / 104 RQEs).

Code (study_saturation.py)

  • KLL k ∈ {25, 50, 100, 200, 400, 800} (was 50/200/800). DDSketch α adds 0.1.
  • The Hydra eval sweep adds W = 65536 for hydra-hll and hydra-kll. hydra-univmon-cardinality is capped at 16384 (HYDRA_MAX_W): its grid at 65536 is about 20 GB, and its error has already saturated.
  • --sketches / --configs measure only new configs, so they can be appended to an existing study (merge_study_parts.py refuses duplicate points).
  • Hydra cost rows wider than 16384 merge 4 shards instead of 16, because 16 grids of 3.2 GB OOM. Merge cost is still per merge.
  • HLL stays at lg_k 12/14/16, the precisions asap_sketchlib builds.

Inputs (new configs only, appended)

  • Curves on clnode130, merged into autosketch-vs-asap-inputs/saturation/. Raw records are in raw/dense_*.
  • 7 cost rows measured serially on idle clnode138: 63 rows in all.
  • 999 Hydra runs at W = 65536 on clnode109/155/167/178: 7,778 rows in all.

Results: the 12 workloads at 95% (results, figures, plans) and at 90% and 99%, on clnode109/155/167/178. 0 RQEs dropped. One sanity check fails by solver tolerance: all-shared8 at 99%, Fargate frontier, 0.01% (see the targets README).

95%, cheapest plan ASAP, CPU (vCPU), old → new ASAP, Fargate ($/h), old → new
mixed 2.38 → 2.35 0.170 → 0.149 (−13%)
multi-grouping 0.221 → 0.220 0.0137 → 0.0134
mixed + multi-grouping 2.60 → 2.57 0.184 → 0.162 (−12%)
  • CPU cost barely moves. Insert CPU per item is flat across configs (KLL ≈ 30 ns, DDSketch ≈ 19 ns).
  • Memory-weighted cost falls up to 13%. Plans pick a KLL or DDSketch closer to the target size.
  • 90% still equals 95%. HLL still bottoms out at lg_k=12, and DDSketch α = 0.1 (0.048 error) meets both targets.
  • Hydra is never chosen. W = 65536 brings its error on groups holding ≥ 1% from 0.283 to 0.039 (hydra-hll, http), but it is the most expensive grid on every axis.

🤖 Generated with Claude Code

zzylol and others added 2 commits October 14, 2026 01:52
…sketches/--configs to add configs to an existing study

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FhcWJExZcmVqS6r6rEtjis
…B do not fit)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol changed the base branch from eval/accuracy-target-sweep to main October 14, 2026 01:52
…, Hydra W 65536)

Inputs: the new configs' curves appended to autosketch-vs-asap-inputs (raw
records in raw/dense_*), 7 cost rows (63 in all), and 999 Hydra runs at
W = 65536 (7,778 in all). Results: the 12 workloads at 95% (with plans) and
at 90% and 99%, on #210's tables.

CPU cost falls 0-1.4%, Fargate cost up to 13% (smaller KLL/DDSketch nearer
the target). 90% still equals 95%. Hydra is never chosen, even at W = 65536.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol marked this pull request as ready for review October 14, 2026 02:19
…ce and claims

- filtered_configs()/check_filters(): --sketches or --configs skips the
  exact rows; a misspelled name exits; tests for both.
- The wide-merge comment says per fold, with operands 4x larger.
- rqe-optimizer README: the -hydra row's 2026-10-14 rerun, the one
  solver-tolerance violation, e2e-rate10 planned on the pre-#213 grid.
- -hydra README: rerun provenance moved to Reproduce; hydra-kll's figure
  marked unmerged, with the all-merge-counts value.
- raw README: measurement dates, clnode130.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…comparability; rewrap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol merged commit 3fa808b into main Oct 14, 2026
2 checks passed
milindsrivastava1997 added a commit that referenced this pull request Oct 14, 2026
…ry (#217)

Regenerates rqe-optimizer/results/e2e-rate10 (classic, all, multigroup at
shared = 1, 10 samples/s) on main: the denser grid's cost table and curves
(#213), instant RQEs reading one scrape (#215), and one RQE per schema query
(#216). Tables, --runs 3 results and --runs 1 plans; the README gives the
RQE counts, costs, classic deployments and ASAPQuery's
--data-ingestion-interval-ms 100.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant