Repository navigation
CI mutation runs one job per project #311
Description
Requirement
The CI Mutation job runs one Stryker job per mutation target project in parallel, so a pull request waits for the slowest project instead of one solution-wide sweep. The latest master run took 15:24, of which 6:35 was five identical coverage captures (Stryker 5.0 repeats the capture over every test for each project in solution mode; see ADR 0010).
Acceptance criteria:
- A first step lists the target projects with
scripts/mutation-targets.sh(from the scoped-run issue), so a new project is mutated without a workflow edit and none can be left out silently (fix(ci): Restore before Stryker and fail when it drops a project #306). - One matrix job per project runs
scripts/stryker-sweep.sh --project <Name>.csprojat the configuredconcurrency: 4and fails below the 100% thresholds.CI gatekeepsneeds: mutation. - Each job posts its own markdown summary and uploads its JSON report.
- A merge job combines the reports with a new
scripts/mutation-report.sh merge. It refuses (non-zero exit) when a source file appears in two reports, a test id maps to two names, or the schema versions differ or are unknown. Each refusal is pinned by a fixture test. - The badge and Pages are built from the merged JSON. Pages serves our own small page loading the merged JSON into a pinned
mutation-testing-elementsviewer, not Stryker's generated HTML. mutation-guard.shaccepts--projectonly in the matrix job andmutation-changed.sh.- The weekly divergence check and the documented local full sweep stay single solution-mode runs.
First caller
The CI Mutation job on every pull request and master push, and publish / deploy-pages, which consume the merged report.
Design
Design note: #311 (comment), approved in #311 (comment). Rationale: ADR 0010 (docs/adr/0010-scoped-mutation-runs-per-project.md; the decision's "ADR 0009" became 0010 because 0009 was taken).
Activity
bitbiter-dev commentedon Oct 24, 2026
Design note
Problem and constraints: Stryker 5.0's solution mode captures per-test coverage once per mutated project, and each capture runs all 1,620 tests of all 6 assemblies, ignoring the project. Five captures took 6:35 of a 15:24 CI run and recorded identical coverage. The gate (100%), the badge, the Pages report and the job summary need one result per run. The project list must not drift from the solution: #306 showed a project can drop out of mutation with only a warning.
Options:
coverage-analysis: all. Captures coverage once, but every mutant then runs every test: 4:57 against 1:08 on Infrastructure alone.Recommendation: option 3. It is the only option that shortens CI, and the merge is mechanical: the reports follow the public mutation-testing-report-schema v2, source files are disjoint across projects (0 overlap in all pairs), and test ids are content hashes, identical across reports (0 conflicts across 4,336 shared ids).
First caller: the
Mutationjob;publishanddeploy-pagesread the merged JSON, so the merge's output format is that schema, unchanged.Hard to reverse: nothing; reverting to option 1 is a workflow change.
bitbiter-dev commentedon Oct 24, 2026
Decision
2026-10-24, the maintainer, in a
/grill-with-docssession:IsMutationTargetMSBuild convention throughscripts/mutation-targets.sh, the same list the scoped run uses, so local and CI agree.concurrencystays pinned at 4, so each job uses its whole runner and a local run measures like CI.github-actions commentedon Oct 24, 2026
Moved back to
needs-triage:docs/agents/design-process.md("What every issue carries") requires these parts beforeready-for-agent:Add them, then label the issue
ready-for-agentagain.ci: Mutation runs one job per project and merges the reports (#311) (#…