fix(build): give PIT the same SLF4J exclusion as Surefire, and cover FlashAttn - #414
Conversation
…FlashAttn Two separate failures were stacked on the `Java Tests Ubuntu` job, and the first was hiding the second. 1. PIT aborted with "5 tests did not pass without mutation ... requires a green suite" — on a suite Surefire had just reported green (1729 tests, 0 failures). Cause: two SLF4J providers on the test classpath. slf4j-simple arrives at runtime scope because it is the binding shipped in the fat jar, logback-classic at test scope because LogCaptor requires logback specifically; SLF4J's ServiceLoader picks one arbitrarily and every LogCaptor assertion fails when it picks slf4j-simple. #411 added the binding and the Surefire `classpathDependencyExcludes` for exactly this, but PIT builds its OWN classpath and never reads Surefire's configuration, so the mutation run still saw both. pitest-maven has the same parameter, modelled on Surefire's; it is now set to match. Nothing about the artifact changes. It went unseen because every publish run between #411 and now was cancelled; run 897 is the first to complete since. 2. With that cleared, PIT actually ran and surfaced the survivor that run 887 had already reported before #411 ever landed: "Mutation score of 99 is below threshold of 100". `net.ladenthin.llama.args.*` is on the PIT target list at threshold 100, and FlashAttn — added in #408 — was the one enum in that package with no test, so `getArgValue()`'s `return argValue` had no killer. Adds FlashAttnTest in the shape every sibling enum already uses. The values are a wire contract, not labels: since llama.cpp b10273 `--flash-attn` takes a mandatory value, so a wrong or empty token makes the parser eat the next argv entry and the load fails naming a flag the caller never set. Verified locally, both directions: the failure reproduces on the parent commit with the exact same five test names, and with the fix PIT reports 319/319 mutations killed (100%) and `mvn clean verify` is green at 1486 tests. CLAUDE.md now names the PIT half of the rule explicitly, next to the existing "spotbugs/spotless bind to verify" trap, including why srcmorph solves the same clash by excluding at the dependency instead (there both providers arrive transitively). This does NOT address the macOS 15 failures in the same run; those are a separate, bump-introduced regression and are reported alongside this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
Review SummaryThis PR fixes two stacked test failures in the Java Tests Ubuntu job with a clear, focused approach. Both changes are correct and follow established patterns. ✅ Issues Fixed1. PIT SLF4J Provider Clash — Excellent diagnosis. The root cause is spot-on: PIT builds its own classpath and doesn't read Surefire's configuration, so the Surefire-only exclusion left both SLF4J providers on the PIT classpath. The fix—mirroring the Surefire exclusion in pitest-maven configuration—is the right approach. The CLAUDE.md documentation clearly explains this non-obvious trap and why it matters. 2. FlashAttn Test Coverage — The new test follows the established pattern perfectly, extending AbstractCliArgEnumTest and providing all three enum constants with correct arg values and enum count. The docstring accurately emphasizes that these are wire contracts, not cosmetic labels, which is an important constraint since llama.cpp b10273 made the flag value-mandatory. Code Quality
Verification Notes✅ Test pattern consistency verified against SamplerTest and other enum tests ScopeThe PR body appropriately notes that macOS 15 failures are separate (a bump-introduced regression). This PR stays focused on the PIT/mutation testing surface, which is the right scope boundary. No issues found. Ready to merge. |
|



Summary
Fixes the
Java Tests Ubuntufailure in run 33861339600. Two failures were stacked there and the first was hiding the second.1. PIT aborted with "5 tests did not pass without mutation … requires a green suite" — on a suite Surefire had just reported green (1729 tests, 0 failures). Cause: two SLF4J providers on the test classpath.
slf4j-simplearrives at runtime scope (it is the binding shipped in the fat jar),logback-classicat test scope (LogCaptor requires logback specifically); SLF4J'sServiceLoaderpicks one arbitrarily, and every LogCaptor assertion fails when it picks slf4j-simple. The log says it outright:#411 added the binding and the Surefire
classpathDependencyExcludesfor exactly this — but PIT builds its own classpath and never reads Surefire's configuration, so the mutation run still saw both.pitest-mavenhas the same parameter, explicitly modelled on Surefire's ("List of classpath entries, formatted asgroupId:artifactId…"); it is now set to match. Nothing about the artifact changes: the jar and the fat jar still ship slf4j-simple.That it went unseen is its own small lesson: every publish run between #411 and now was cancelled, so run 897 is the first to complete since the binding changed.
2. With that cleared, PIT actually ran and surfaced the survivor that run 887 had already reported before #411 ever landed —
Mutation score of 99 is below threshold of 100.net.ladenthin.llama.args.*is on the PIT target list at threshold 100, andFlashAttn(added in #408) was the one enum in that package with no test, sogetArgValue()'sreturn argValuehad no killer. AddsFlashAttnTestin the shape every sibling enum already uses. Those values are a wire contract, not labels: since llama.cpp b10273--flash-attntakes a mandatory value, so a wrong or empty token makes the parser eat the following argv entry and the load fails naming a flag the caller never set.Test plan
ChatResponseParserTest.testParseResponse_emitsTimingLine,CompletionResponseParserTest.testParseCompletionResult_emitsTimingLine, and three inTimingsLoggerTest)multiple SLF4J providersline, the five tests pass, and the pre-existing survivor becomes visible (99%)FlashAttnTestadded: PIT 319/319 mutations killed (100%),BUILD SUCCESSmvn clean verifygreen — 1486 tests, 0 failuresCLAUDE.mdupdated — the PIT half of the rule is now named next to the existing "spotbugs/spotless bind toverify" trap, including why srcmorph solves the same clash by excluding at the dependency instead (there both providers arrive transitively)The same run also fails
Java Tests macOS 15 arm64 (Metal)and(no Metal). That is a separate, bump-introduced regression and is not touched here. What I established:-DGGML_NATIVE=OFF(macOS 14 is the host-native build). Ubuntu and both Windows jobs run these same tests fine.LlamaModelis opened while one is already resident:LlamaModelTest.testSpeculativeDecoding— the draft model (AMD-Llama-135m-code.Q2_K) throws ~30 ms intollama_model_load:E llama_model_load: error loading model: vector."vector"is libc++'swhat()forstd::out_of_range/std::length_error, which is why the message is that terse and why it is macOS-shaped.MemoryManagementTest.testPromptCacheCompleteMissAfterWarmup— a secondcodellama-7bload fails incommon_init_from_paramsafter the model itself loaded, with nollama_model_loaderror line. The Metal job also logsggml_metal_log_allocated_size: warning: current allocated size is greater than the recommended max working set size, so runner memory pressure is real — but it was green on the same runners at b10731.I cannot reproduce either on Linux, and the job log has no further native detail. The cheapest decisive next step is a bisect: pin back to
b10792on a probe branch and let CI say whether the five b10792→b10797 commits caused it (#28323touchesllama-model-loader.cppandllama-hparams.*, which is the suspicious one). Say the word and I'll push that probe — or, if you would rather not spend a run, pinning back tob10792outright is a safe landing spot, at the cost of the s390xq5_1fix.Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.md🤖 Generated with Claude Code
https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
Generated by Claude Code