Skip to content

rqe-optimizer: hydra-kll accuracy when a query merges several windows #166

Closed as not planned

Description

@milindsrivastava1997

Found in review of #160 (S4b of #143).

Problem

accuracy_at (#154) reads merge_accuracy at the merge counts on either side of n = L/x, and falls back to the single-window query_accuracy when the row has none. The hydra-kll rows have an empty merge_accuracy. A fixed-size shared sketch folding n windows of every group holds about n times the items in the same grid, so a quantile RAQE with L = 10x can pass accuracy_ok_for while the true rank error exceeds the SLA.

Options

  • Measure: have scripts/export_rqe_optimizer_costs.sh record hydra-kll merge accuracy at several merge counts, as rqe-optimizer: check accuracy at the query's merge count #154 does for the other families.
  • Refuse until measured: make families with one_fixed_size_sketch_for_all_groups ineligible when n > 1 and merge_accuracy is empty.

Probably both: refuse now and loosen once the rows exist. Re-measure after the no-fan-out hydra-kll change, since fan-out changes the error.

🤖 Generated with Claude Code

Activity

  1. removed their assignment
    on Oct 7, 2026
  2. zzylol commented on Oct 10, 2026

    @zzylol
    Collaborator

    Closing as a duplicate of #193 §5.6 (docs/rqe_optimizer_hydra.md).

    • Merged-grid accuracy and cost are measured at the shard counts workloads need, with records split by sample.
    • CMS, CountSketch and HLL grids get exactness checks.
    • KLL and UnivMon cells get merge curves.
    • Until those land, a Hydra candidate can't serve a multi-window query.

    (decided by @zzylol)

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions