feat(test): support sharding tests across CI runners

[peter-trost]

Description

Adds --shard-index and --total-shards to very_good test and very_good dart test, so a test suite can be split across multiple CI runners with a strategy.matrix.

strategy:
  matrix:
    shard: [1, 2, 3]
steps:
  - run: very_good test --shard-index ${{ matrix.shard }} --total-shards 3

Opening as a draft because @ryzizub is assigned to #1538 — happy to hand this over, close it, or adapt to a different design if work is already underway or a different approach is preferred.

Why this approach

The test optimizer already discovers every test file, so sharding is a partition of that list rather than new machinery.

• Sorted, then round-robin. Directory.listSync order is filesystem dependent, so without sorting two runners could disagree on the partition and either skip or duplicate tests. Sorting first makes the partition deterministic across machines; dealing out round-robin keeps shards balanced by file count.
• Non-optimized tests are sharded too. Files tagged skip_very_good_optimization run as standalone files alongside the optimizer entrypoint. Leaving them unsharded would re-run all of them on every runner, partly defeating the purpose.
• No shard-specific filenames. The issue proposed generating .test_optimizer_1_of_3.dart. That turned out to be unnecessary — each runner generates its own .test_optimizer.dart containing only its slice, and the file is cleaned up afterwards anyway. Same result, no filename plumbing.
• Empty shards succeed. A shard with no test files (more shards than test files) would otherwise fail with No tests were found, breaking builds on oversized matrices. It now reports success, matching the existing "no test folder" behaviour.

Interaction with coverage

--min-coverage is rejected with a usage error when sharding. Each shard only exercises a subset of the codebase, so its coverage is not representative of the whole suite and would fail the build spuriously. The error explains the alternative: collect per-shard coverage with --coverage, merge the lcov reports, and enforce the threshold once in a separate job.

Sharding also requires the optimizer, so it is rejected with --no-optimization and with --platform (which disables the optimizer). Both cases exit with ExitCode.usage and an actionable message.

This is the part I'd most like a maintainer opinion on — a documented merge-then-check workflow is the other reasonable option, and #804 may change what's possible here.

Testing

• Hook unit tests (11 new): complete/disjoint partition, non-optimized sharding, determinism, balance, empty shards, and the pass-through case.
• Command tests (15 new): all validation paths and shard pass-through, for both test and dart test.
• Runner tests (2 new): shard vars reach the generator; empty shard short-circuits without invoking the test runner.
• Manual end-to-end against a real package (7 optimizable + 2 tagged test files): verified all tests run exactly once across 3 and 12 shards, that shards are stable across repeated runs, and that the Flutter path shards correctly. Also exhaustively verified the partition is complete, disjoint, and balanced for suite sizes 0–40 across 1–8 shards.

dart analyze --fatal-infos --fatal-warnings . is clean, and the touched lib/ files are at 100% line coverage.

Note

The full suite passes with -j 1 (529 tests). At -j 8 a handful of unrelated tests fail due to a pre-existing working-directory race between test files — I confirmed the same failures on a clean main checkout, so it is not introduced here.

Closes #1538

🤖 Generated with Claude Code