Skip to content

DRB-8: Unify bench/ E5 and perf/ schema-v2 performance authority #196

Description

@metaphorics

Problem

Two live performance policy surfaces coexist and contradict each other. The repository must decide whether they are deliberately separate evidence tracks or one supersedes the other, then assign a single non-conflicting owner for host conditions, argv model, baselines, thresholds, and artifact directories.

Exact conflicts

  • Host conditions: bench/compiler-rules.toml:20-32 requires performance governor, disabled swap, and pinned affinity (schema bamti.compiler-rules/v1). perf/hosts/bh1.toml:24-30 records powersave governor, swap_total_kib = 403979000, and affinity 0-19 (schema = 2). These are mutually exclusive default host profiles.
  • Baseline directories: bench/compiler-rules.toml:139 expects bench/baselines/stage1.json; perf/benchmarks.toml:15,26,36,46,56,66,76,86,96,106,116,127 expects perf/baselines/s0.json..s11.json.
  • Argv model: bench/compiler-rules.toml:58-68,76-78,90-98,105-112,137-138 uses runtime flags like ["-r", "--jit", "--json"]. perf/benchmarks.toml:185 uses ["check", "{fixture}"], which conflicts with docs/reference/cli.md:65 (the previous check interface is not part of the current public CLI).
  • Consumers: Cargo.toml:11-21 registers bench/jit_benchmarks.rs, bench/stage0_evidence.rs, and bench/stage1_regression_guard.rs; crates/bamts-verification/src/lib.rs:24-26 and main.rs:165 wire perf_stage0, perf_jit, perf_guard; crates/bamts-verification/src/bin/perf_budget.rs consumes schema-v2 perf/hosts/bh1.toml. verification/catalog-inputs.json and completion-program.toml:937-973 lock E5 leaves to bench/ paths.

Scope of change

  • Decide whether bench/ (E5) and perf/ (schema-v2 / PB-1 / BH1) are distinct tracks with non-overlapping host IDs, or whether one schema is the migration target.
  • If distinct, document the boundary: which measurements, gates, binaries, and CI lanes use which surface, and ensure no gate mixes the two policies.
  • If unified, pick the surviving schema, migrate all consumers, baselines, thresholds, manifests, and CI/completion references, then archive the superseded surface.

Source-of-truth predecessors

  • Host conditions: the chosen single policy manifest (bench/compiler-rules.toml, perf/hosts/bh1.toml, or a merged perf/policy.toml).
  • Baseline/scorecard paths: the surviving surface's manifest + perf_budget / bamts-verification consumers.
  • Argv content reconciliation: DRB-3 (CLI contract drift) owns whether check is a public or internal-only command.
  • Missing baselines: DRB-12.

Acceptance

  • grep across Cargo.toml, crates/, verification/, .outline/gates/, Justfile, and .github/workflows/ shows no mixed references to both bench/compiler-rules.toml and perf/hosts/bh1.toml for the same measurement gate.
  • One and only one named file owns host conditions, argv model, baseline paths, and thresholds for each active performance leaf.
  • cargo check and cargo test -p bamts-verification pass with the chosen single surface (or with explicitly separated E5/PB host IDs and distinct binary entry points).
  • perf_budget / bamts-verification perf_* emits a deterministic error (not silent policy picking) when host conditions or required baselines are missing.

Blockers

  • DRB-3 (CLI contract drift) for the argv = ["check", ...] content.
  • DRB-12 (missing planned artifacts) cannot classify baselines until this decision is made.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    wayfinder:taskDecision prerequisite ticket

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions