Skip to content

ooxml.js is at 96.91% mutation score, 211 mutants short of 100 #1296

Description

@Mearman

A complete local Stryker run over the whole package (concurrency 2, no incremental cache, so every valid mutant got a result) puts ooxml.js at 96.91% of 6839 valid mutants: 6599 killed, 29 timeout, 187 survived, 24 with no coverage. So 211 mutants are still undetected and the package is short of the genuine 100 the rest of the campaign has been aiming for. break is set to 95, derived from that measurement by the rule on PackageStrykerOptions.breakThreshold, not picked.

Nothing is suppressed to get there. grep -rn "Stryker disable" src/ returns nothing, and every mutant closed so far was either killed by a real test or removed by restructuring the code so the mutation had nowhere to apply.

Where the 211 actually are. These ten files are the whole of it: 43 of the package's mutated files are at a genuine 100 and the remaining 3 carry no mutants at all.

File score survived no coverage
typed/xlsx/build.ts 87.66 67 1
typed/pptx/read.ts 83.77 53 9
typed/xlsx/conditional-format.ts 90.51 32 11
typed/xlsx/styles.ts 93.63 20 0
typed/docx/write.ts 99.45 6 0
typed/docx/read.ts 99.40 5 0
typed/shared/drawingml.ts 99.40 1 0
typed/xlsx/comments.ts 98.82 1 0
typed/xlsx/content.ts 98.38 1 2
typed/xlsx/drawings.ts 98.69 1 1

The bulk of the work is the top four. The six single-survivor files are probably an afternoon between them, and the 24 no-coverage mutants are the cheapest thing here: they are code no test reaches at all, so they need a test that gets there rather than a test that distinguishes anything subtle.

The method that works, and it is the only one that should be used: where a survivor represents a real behavioural difference, write a test that pins it; where the mutation genuinely cannot change observable behaviour, restructure the code so the mutation has no AST node to apply to, rather than leaving it and annotating it. Removing a redundant guard whose condition is already implied by the one beside it is the common shape. Do not reach for a disable comment and do not weaken an assertion to make a mutant die.

Two things worth knowing before starting on xlsx/build.ts, both carried over from the earlier triage in stryker.config.ts:

Several of its survivors were disproven by hand. Applying the exact mutation the report names (declarations.length > 0 changed to true, for one) and running pnpm exec vitest run --config vitest.mutation.config.ts src/typed/xlsx/build.test.ts, which is the same runner config Stryker itself uses, fails real tests every time. So a [Survived] verdict on this file is a claim to check rather than proof a test is missing, and the triage should start by reproducing each one manually.

Three of its survivors were checked and are genuine equivalents: definedNameKey's localSheetId ?? "" fallback (the undefined-segment key of a workbook-global name is only ever added to carriedNames, never looked up, because every derivation lookup passes a numeric sheet index), and buildCellElement's cell.font !== undefined || and format === undefined && decoration === undefined (an explicitly-passed all-undefined decoration interns to the same signature and the same default index as no decoration at all, so the output is byte-identical). Those three need the code restructured, not a test.

The survivors in docx/write.ts and docx/read.ts were also each verified equivalent by hand, so those two files need restructuring rather than new tests. The details are in the comment on packages/ooxml.js/stryker.config.ts.

To reproduce the measurement: cd packages/ooxml.js && pnpm exec stryker run stryker.config.ts --concurrency 2. It takes about seventy minutes on a loaded machine. Scoping to one file with --mutate is much quicker while iterating, and the per-file numbers from a scoped run match the full run exactly, which is how the figures above were cross-checked.

Note that CI cannot currently confirm any of this on a pull request, for the reason in #1294.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions