Skip to content

[Bug]: [mutate] Enum case values are never mutation-tested #1978

Open

Description

@mechgw

What Happened

The value of a string-backed enum case is never effectively mutation-tested, even when tests check it. Changing 'paid' in the example below would break the test, but --mutate never reports it. There are four independent reasons, all in pestphp/pest-plugin-mutate:

  1. The mutator is dropped. The mutator for non-empty strings, NotEmptyStringToEmpty, is not in a default set. When it is requested explicitly, MutationGenerator::filterMutators() drops it, because it only keeps mutators found in MutatorMap, which is built from the default sets. So no mutation is created at all.
  2. mutates() and --class ignore enums. MutationGenerator::doesNotContainClassToMutate() matches only class and trait, so mutates(Status::class) creates no mutation for an enum.
  3. The case declarations have no line coverage. The mutation is then reported as uncovered, without running any test. With --covered-only, the enum file is skipped entirely.
  4. Separately, on Linux and macOS, when the covered files share only /, every mutation is reported as uncovered (basePath of --coverage-php is empty and the paths stay relative).

How to Reproduce

enum Status: string
{
    case Paid = 'paid';
}
it('stores the paid status', function () {
    expect(Status::Paid->value)->toBe('paid');
});

Run pest --mutate --mutator=StringNotEmptyStringToEmpty: no mutation is created for 'paid'. With mutates(Status::class) in the test file, no mutation is created either.

Sample Repository

mechgw/pest-plugin-mutate@5.x...fix/enum-case-mutations (fixtures and tests for every case)

Pest Version

5.3.2 (pest-plugin-mutate 5.0.2)

PHP Version

8.4.21

Operation System

Linux, Windows

Notes

I would like to fix this in a series of small pull requests to pestphp/pest-plugin-mutate, each one useful on its own:

The last part is the largest, so I would rather hear your opinion before opening it. The approach is:

  • for a mutated case value, the plugin runs the tests whose code references the case (Status::Paid, self::Paid, static::Paid, and from(), tryFrom(), cases() as a reference to every case), including the hooks and the helper functions of the same test file. If those tests do not detect the mutation, it falls back to all the tests originally selected;
  • the result comes from the JUnit report of that run: a failing test means tested, a passing run means untested, and a missing report or an unknown exit code is reported as uncovered with the reason printed, never as tested;
  • with --covered-only, a case value is mutated only when the selected tests reference it. Cases referenced only from helpers in other files or from closures stored in variables are skipped, and this limitation is covered by a test;
  • in parallel runs, the workers record the tests they execute, so the plugin knows which tests were actually selected.

The complete change, with tests on Linux and Windows, is on the branch linked above. It is a large diff, so please read it only as a preview. I will split it and adapt it to whatever direction you prefer.

Would you accept this approach, or would you rather solve point 3 differently?

Activity

  1. mechgw commented on Oct 18, 2026

    @mechgw
    Author

    Opened pestphp/pest-plugin-mutate#50 for point 2 without waiting for pestphp/pest-plugin-mutate#22: both change the same line, but they do not contradict each other, and #50 describes what changes once #22 is merged.

  2. theofidry commented on Oct 20, 2026

    @theofidry

    Out of curiosity did you find meaningful tests and mutations from this?

    Because:

    it('stores the paid status', function () {
        expect(Status::Paid->value)->toBe('paid');
    });

    Would only be writing different a native PHP construct, not testing behaviour. It would be akin to test that this is not a class or whether it's final.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions