Skip to content

Rework benchmarks#2517

Open
l0r1s wants to merge 16 commits intodevnet-readyfrom
rework-benchmarks
Open

Rework benchmarks#2517
l0r1s wants to merge 16 commits intodevnet-readyfrom
rework-benchmarks

Conversation

@l0r1s
Copy link
Collaborator

@l0r1s l0r1s commented Mar 19, 2026

Summary

Converts the benchmark and weight system to the standard FRAME WeightInfo pattern. This lays the groundwork for using parameterized weights where extrinsic cost scales with input size (e.g., number of contributors, batch size), which was not possible with hardcoded values.

Before: Extrinsics had Weight::from_parts(X, 0) hardcoded directly in #[pallet::weight(...)] annotations. CI parsed benchmark stdout and patched values in source files.

After: Each pallet has a weights.rs file, a WeightInfo trait, and extrinsics reference T::WeightInfo::fn_name(). CI uses a syn-based Rust tool to compare old vs new weight values with a threshold.

Note: we generate weights using --runtime + --genesis-builder flags (not --chain) because the runtime uses custom host functions that require the full node binary.

Pallet changes

  • pallet_subtensor — 51 extrinsics converted to T::WeightInfo::fn(), 18 without benchmarks kept hardcoded
  • pallet_admin_utils — 55 extrinsics converted, 19 kept hardcoded
  • pallet_drand — all 3 extrinsics converted
  • pallet_shield — all 2 extrinsics converted
  • pallet_commitments — all 2 extrinsics converted, stale weights.rs fixed (set_rate_limit renamed to set_max_space)

For each pallet: added pub mod weights, type WeightInfo in Config, wired SubstrateWeight<Runtime> in runtime, added type WeightInfo = () to all test mocks.

New tooling (support/weight-tools/)

Lightweight crate (~4s build, no runtime dependency):

  • weight-compare — parses two weights.rs files with syn, compares base weights (threshold-based), reads/writes (exact), and per-component slopes for parameterized weights. Used by CI.

Scripts

  • scripts/benchmark_all.sh — rewritten to generate weights.rs via --output/--template, supports single-pallet runs and SKIP_BUILD=1
  • scripts/benchmark_action.sh — rewritten to use weight-compare with 40% drift threshold
  • Removed 4 redundant per-pallet scripts/benchmark.sh files

CI

  • run-benchmarks.yml — cleaned up, extracted repeated skip-label check into .github/scripts/check-skip-label.sh, added weight-compare build step
  • apply-benchmark-patch.yml — uses a GitHub App token instead of GITHUB_TOKEN so the pushed commit re-triggers all CI workflows. Requires GH_APP_ID (repo variable) and GH_APP_PRIVATE_KEY (repo secret) to be configured once.

Other

  • .maintain/frame-weight-template.hbs synced with upstream polkadot-sdk
  • docs/benchmarks-and-weights.md — guide covering new pallet setup, adding extrinsics, parameterized weights, CI workflow

@l0r1s l0r1s changed the title WIP Rework benchmarks [WIP] Rework benchmarks Mar 19, 2026
@l0r1s l0r1s marked this pull request as ready for review March 19, 2026 21:32
@l0r1s l0r1s added the skip-cargo-audit This PR fails cargo audit but needs to be merged anyway label Mar 19, 2026
@l0r1s l0r1s changed the title [WIP] Rework benchmarks Rework benchmarks Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-cargo-audit This PR fails cargo audit but needs to be merged anyway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant