test: Give each in-memory EF test its own service provider so mutants cannot leak through EF's caches

Problem

Under mutation testing, one in-memory EF test can change the outcome of another. Stryker's MTP runner keeps one test host alive across many mutants. EF Core keeps the built model and every compiled query in an internal service provider that AnichronDbContexts.InMemory() caches for the whole process. A mutant that is active when a query is first compiled therefore stays baked into that query for every later mutant tested on the same host.

How it shows up: in the single-threaded divergence sweep, the block-removal mutant of EfBurstRepository.RemoveCheckAsync (src/Anichron.Core/Data/Repository/BurstRepository.cs:93) is reported Survived. That happened in run #6 and run #7, and locally on master. The query-filter mutant of AnichronDbContext.cs:98 had already compiled BurstChecks.CountAsync() with an inverted filter on that host, so RemoveCheckAsync_CheckedPhoto_DeletesItsRowAtTheNextSaveAsync passes with the method body removed. The CI-concurrency sweep kills it. The divergence check takes the single-threaded sweep as its reference, so it fails. Full diagnosis and measurements: diagnosis comment.

The same can happen for every HasQueryFilter lambda in AnichronDbContext.cs (lines 55, 75, 98, 172, 205, 228). It works in both directions: a false survivor (this case), or a query-filter mutant that cannot take effect because the query was compiled before it ran.

Desired behavior

Every in-memory test gets its own EF internal service provider, so its own store, model and query cache. Every context the test creates shares that provider. Nothing EF compiles in one test is reused by another test in the same process, whatever mutant is active.

Acceptance criteria

The single-threaded sweep of a Mutation divergence run on the branch reports the BurstRepository.cs RemoveCheckAsync block-removal mutant Killed, at 100.00%. On master today a --concurrency 1 sweep reports it Survived at 99.94%, its only survivor. No separate local --concurrency 1 sweep is needed.
A Mutation divergence run (workflow_dispatch on the branch) passes with equal detected counts. The PR states both counts (concurrency 4 and concurrency 1).
Anichron.Testing.Tests.Unit pins the helper's contract: contexts built from one options value share a store (a row saved through one is read through another), and two options values never share an internal service provider.
No test passes a database name to share a store. Shared stores come only from sharing one options value, and the databaseName parameter is gone.
freshModel is gone, because every options value now builds its own model. SoftDeleteQueryFilterTests and AnichronDbContextsTests are adjusted to match. NpgsqlModel() is unchanged.
The time of the unit suite (dotnet test) before and after, the median of 3 runs each on the same machine, is stated in the PR. It is reported for information; there is no budget. The ~19 ms per test measured below is the expected order.
CLAUDE.md (the EF bullet under Testing, and the "Mutants that flip between runs" paragraph) and docs/mutation-testing.md describe the new rule and why it exists: test-host reuse across mutants leaks per-process caches.

First caller

The existing in-memory tests, first of all BurstRepositoryTests.RemoveCheckAsync_CheckedPhoto_DeletesItsRowAtTheNextSaveAsync, the test this issue caught. The call sites that change are the 19 that share a store by database name, in BurstRepositoryTests (6), MediaAssetRepositoryTests (4), BurstDetectionTests (7) and AnichronDbContextsTests (2). Call sites that already reuse one options value (AssetInteractionWriterTests, AssetViewReaderTests, MediaAssetReadRepositoryTests) and single-context InMemory() callers keep their code.

Design

Approved by the maintainer (2026-10-25): option 1 of the design note, one EF internal service provider per options value. Also decided:

InMemory() stays as a no-parameter shortcut for new(InMemoryOptions()); multi-context tests share one InMemoryOptions() value.
No architecture rule or other guard against tests building their own in-memory options. Add one only if a test ever bypasses the helper.
No ADR: the rule and its reason go in CLAUDE.md and docs/mutation-testing.md.
The PR also closes Mutation gate: RemoveCheckAsync block-removal mutant is reported surviving in about half of the sweeps #312 (Fixes #312).

Out of scope

A Stryker upstream report (MTP host reuse leaks per-process state across mutants). Decided not to file one.
Other per-process state, such as static readonly initialisers, which CLAUDE.md already covers.

Happened again in divergence run #7, on branch fix/stryker-cold-restore (#306). Both sweeps now mutate every project and each tests 1595 mutants. This BurstRepository.cs:93 block-removal mutant is the only difference: Killed at concurrency 4, Survived at concurrency 1. That makes 2 out of 2 runs, so the single-threaded sweep misses this kill every time; it isn't occasional. That should make it easier to reproduce locally.

Diagnosis: reproduced locally, and the cause is EF's query cache in a reused test host

Result: the line-93 block-removal mutant survives the single-threaded sweep because EF Core caches compiled queries per process, and Stryker's MTP runner reuses one test host across many mutants. A query-filter mutant tested earlier on the same host leaves its inverted filter in the cached BurstChecks.Count() query, so the killing test passes with the method body removed. This is a measurement fault (a false survivor). The test is not missing anything, and it is a different defect from the documented upstream one. No code changed.

Reproduction (master `bdb9e0d`, Stryker 5.0.0, macOS, private `TMPDIR`)

Run	Mutant `BurstRepository.cs:93` (block removal)
`dotnet stryker -m "**/Anichron.Core/Data/Repository/BurstRepository.cs" --concurrency 1`	id 360, Killed
`dotnet stryker --concurrency 1` (full sweep, `--verbosity debug --log-to-file`)	id 360, Survived; score 99.94%, its only survivor

| dotnet stryker -m ".../BurstRepository.cs" -m ".../AnichronDbContext.cs" --concurrency 1 | id 360, Survived, the only survivor; mutant 49 (below) tested at log line 45586, 360 at 47278, no Core.Tests.Unit host restart in between |

The full single-threaded sweep reproduces it on a laptop, so it is not CI-specific. Scoping to BurstRepository.cs alone does not reproduce it. Adding only AnichronDbContext.cs, which brings in the query-filter mutant, does.

Mechanism

AnichronDbContext.cs:98: entity.HasQueryFilter(c => !c.Asset.IsSoftDeleted);. Stryker mutates the lambda body (LogicalNotExpression; id 49 locally, 48 in CI run Configure appsettings.json and environment variable wiring #7's single-threaded sweep). In the expression tree that becomes IsActive(49) ? c.Asset.IsSoftDeleted : !c.Asset.IsSoftDeleted.
EF evaluates that IsActive(49) call once, when it compiles a query, and stores the compiled query in the query cache of the internal service provider. AnichronDbContexts.InMemory() leaves service-provider caching on, so that cache is shared by the whole test-host process.
Mutant 49's covering tests include RemoveCheckAsync_CheckedPhoto_DeletesItsRowAtTheNextSaveAsync. Running it with 49 active compiles db.BurstChecks.CountAsync() with the inverted filter.
The MTP runner keeps the Anichron.Core.Tests.Unit host warm across mutants. From the debug log: the host last started during mutant 8's run (log line 51683), ran mutants 10…359 including 49 (line 52022), and was not restarted before mutant 360 (line 54038).
When mutant 360 runs, the cached count query still filters to soft-deleted assets, so it returns 0 even though the row was never removed. The test's Should().Be(0) passes, and the run is reported "success" (Survived).

At concurrency 4 the mutants are spread over four runners with separate hosts. The host that tests the line-93 mutant there presumably hadn't run the filter mutant since it started (CI has no debug log to show which host ran what), so the cache was clean and the mutant is killed. With one runner the order is fixed, which is why this happened in 2 out of 2 runs.

The EF behaviour, isolated

A scratch console app on EF Core InMemory 10.0.13, with a stand-in MutantControl and the same filter shape:

48-first:  mutant 48 active: count=0 (expect 0)   mutant 294 active: count=0 (expect 1)   ← stale filter
294-first: mutant 294 active: count=1 (expect 1)  mutant 48 active: count=1 (expect 0)    ← filter mutant can't take effect

Each count uses a new database and a new context. Only the process-wide query cache carries over. So the defect works both ways: it can create a false survivor (this issue), or make a query-filter mutant itself look survived if the query was compiled before that mutant ran. Here mutant 49 is still killed through other tests (SoftDeleteQueryFilterTests, Worker BurstDetectionTests), but that depends on the order.

Scope: what else is exposed

Any mutant inside a HasQueryFilter lambda (AnichronDbContext.cs lines 55, 75, 98, 172, 205, 228) can contaminate later mutants on the same host whose killing tests run the same query shape, or be masked itself. This is the same family as the "EF's cached model" note in CLAUDE.md, but it goes through the query cache, not the model cache, and ModelBuilder mutants aren't needed to trigger it. (Mutants in ordinary LINQ query lambdas in repository code are presumably safe, because EF turns closure and method-call subtrees in the main query into parameters re-evaluated on each execution. That was not measured here.)

Fix options (for triage; nothing implemented)

⚠️ freshModel: true is not a fix, and would make this worse. Measured in the scratch app: with service-provider caching off, the seed context and the test context get separate in-memory stores, so the seeded row is invisible (IgnoreQueryFilters().Count() is 0). The test would then pass against the removed body every time. The same applies to any test that seeds in one context and reads in another.

caching	database root	outcome (both orders)
on (today)	default	stale filter carried between mutants
off	default	store split, seed invisible: count always 0
off	one shared `InMemoryDatabaseRoot`	correct in both orders

Test-side: give the InMemory helper a mode that turns service-provider caching off and passes one InMemoryDatabaseRoot to every context of a test (UseInMemoryDatabase(name, root)). That drops the process-wide query and model caches while keeping the store shared. It could be used narrowly (tests whose killing assertion runs through a query filter) or by default for InMemory(). The cost is a model build per context, not yet measured on the suite.
Upstream (Stryker): report that the MTP runner's process reuse leaks per-process caches across mutants. The injected IsActive call in a mutated expression tree is evaluated once by EF and cached. A stand-alone repro like the scratch app above would make a small upstream issue.
Not recommended: a // Stryker disable on the line-93 mutant. It is killable and is killed whenever the cache is clean, so it is not equivalent.

The debug logs from these runs were local and are not kept. The log line numbers above come from them.

Design note

Problem and constraints:

Stryker's MTP runner reuses one test host across many mutants. Any per-process cache that holds the result of mutated code carries it into later mutants.
EF Core keeps the model and compiled queries in an internal service provider. AnichronDbContexts.InMemory() currently lets EF cache that provider for the whole process.
Constraints:
- Tests that seed through one context and read through another must keep seeing one store.
- No new process-wide static state in test infrastructure.
- The unit suite stays fast enough to run as the pre-push gate.
- The fix lives in Anichron.Testing, which is excluded from mutation (ADR 0005, rule 4) and tested by Anichron.Testing.Tests.Unit.

Measured with a scratch app on EF Core (InMemory) 10.0.13 and the real AnichronDbContext, using a stand-in MutantControl whose active id changes between "tests":

Setup	Seed visible to a second context	Compiled query reused by the next test	Cost per test (warm)
Today: cached provider	yes	yes (the defect)	0.1 ms
`EnableServiceProviderCaching(false)` (today's `freshModel: true`)	no: each context gets its own store	no	model build per context
Caching off + one shared `InMemoryDatabaseRoot`	yes	no	model build per context
One internal provider per test (`UseInternalServiceProvider`)	yes	no	~19 ms (model build per test)

Options:

One internal service provider per options value. InMemoryOptions() builds new ServiceCollection().AddEntityFrameworkInMemoryDatabase().BuildServiceProvider() and passes it with UseInternalServiceProvider(...). A test that needs several contexts shares one options value (new AnichronDbContext(options)), as AssetInteractionWriterTests and others already do. The databaseName and freshModel parameters go away.
Cost: about 19 ms per test for the model build. The 19 named call sites in 4 files change. Nothing new to name or learn: "contexts share a store when they share options" is how EF itself works.
Explicit per-test store type. A new InMemoryStore in Anichron.Testing owns the provider and hands out contexts (store.Create()).
Cost: the same runtime cost as option 1, plus a new type that only wraps what one options value already is, and the same call-site changes.
Look up a provider by database name in a static dictionary. InMemory(name) returns contexts on the provider registered for that name.
Cost: no call-site changes. But it is itself the process-wide static state that caused this, it keeps one provider and model per test alive for the host's lifetime, and a test that reuses a name silently shares state again.
Caching off + shared InMemoryDatabaseRoot per test.
Cost: correct, but the model is built for every context rather than every test, which is slower than option 1. It still needs a per-test handle to carry the root, so it changes the same call sites.
Change the pattern instead: restart the test host for every mutant (or ask upstream for it).
Cost: Stryker 5.0.0 has no option for it. A host start per mutant would multiply sweep time several times over. This belongs in the upstream report, not in this repo.

Recommendation: option 1. It is the smallest interface that gives each test its own EF universe. It adds no type and no static state, and it removes two parameters instead of adding one. It pays the model build once per test rather than once per context. And it makes the right way to share a store (share the options) the only way.

First caller: BurstRepositoryTests.RemoveCheckAsync_CheckedPhoto_DeletesItsRowAtTheNextSaveAsync (seed context plus test context). It must keep seeing its seed and must kill the line-93 mutant in a full --concurrency 1 sweep. Its shape, two contexts from one options value, is the one every multi-context test takes.

Hard to reverse: nothing. Test infrastructure only, with no production code, schema or contract change.

Implementation sketch:

public static DbContextOptions<AnichronDbContext> InMemoryOptions()
    => new DbContextOptionsBuilder<AnichronDbContext>()
        .UseInMemoryDatabase(Guid.NewGuid().ToString())
        .UseInternalServiceProvider(new ServiceCollection().AddEntityFrameworkInMemoryDatabase().BuildServiceProvider())
        .Options;

public static AnichronDbContext InMemory() => new(InMemoryOptions());

Call sites that passed a name become var options = AnichronDbContexts.InMemoryOptions(); plus new AnichronDbContext(options) for each context. Anichron.Testing may need a package reference for Microsoft.Extensions.DependencyInjection (EF already depends on it transitively). The contract tests can read the provider through options.FindExtension<CoreOptionsExtension>()!.InternalServiceProvider.

Won't fix. Option 1 was implemented in #319 and did remove the leak, but the cost is too high:

The unit suite went from 10.0 s to 23.7 s.
Mutation — Anichron.Core went from 5–8 min to more than 20 min, and timed out.

Each test builds the model again and compiles every query again, and Stryker pays that for every mutant. Paying that across the whole suite to fix one false survivor isn't worth it.

What remains: a HasQueryFilter mutant can still leave a compiled filtered query cached for later mutants on the same test host. That has been seen only once (#312). #312 is fixed narrowly instead: its killing test asserts with IgnoreQueryFilters(), a query shape the filter mutant never compiles, and docs/mutation-testing.md records the pattern. If the leak shows up in other tests, test/305-inmemory-provider (#319) is the full fix, with the measured costs above.

Navigation Menu

Repository navigation

test: Give each in-memory EF test its own service provider so mutants cannot leak through EF's caches #305

Problem

Desired behavior

Acceptance criteria

First caller

Design

Out of scope

Diagnosis: reproduced locally, and the cause is EF's query cache in a reused test host

Reproduction (master `bdb9e0d`, Stryker 5.0.0, macOS, private `TMPDIR`)

Mechanism

The EF behaviour, isolated

Scope: what else is exposed

Fix options (for triage; nothing implemented)

Design note

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

test: Give each in-memory EF test its own service provider so mutants cannot leak through EF's caches #305

Description

Problem

Desired behavior

Acceptance criteria

First caller

Design

Out of scope

Activity

bitbiter-dev commented on Oct 24, 2026

bitbiter-dev commented on Oct 24, 2026

Diagnosis: reproduced locally, and the cause is EF's query cache in a reused test host

Reproduction (master bdb9e0d, Stryker 5.0.0, macOS, private TMPDIR)

Mechanism

The EF behaviour, isolated

Scope: what else is exposed

Fix options (for triage; nothing implemented)

bitbiter-dev commented on Oct 25, 2026

Design note

bitbiter-dev commented on Oct 25, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Reproduction (master `bdb9e0d`, Stryker 5.0.0, macOS, private `TMPDIR`)