feat(filter): finding-class priors + witness-based corroboration (#468)

[justn-hyeok]

Summary

Follow-up to #468. Two research-backed additions targeting the FP class captured in today's n=3 baseline runs against `fp-moderator-regex` and `quota-manager-dual`.

Check 7 — finding-class priors

`packages/core/src/pipeline/finding-class-scorer.ts` holds an ordered table of empirically FP-heavy claim patterns with category-specific multipliers:

class	multiplier	targets
redos	×0.6	"catastrophic backtracking", "ReDoS", "exponential time"
zero-width	×0.5	"zero-width space", "invisible unicode character"
may-throw	×0.7	"may throw", "uncaught exception", "without try/catch"
missing-validation	×0.7	"missing input validation", "unsanitized input", "unvalidated argument"
generic-potential	×0.85	"potential security risk", "could be exploited"

Each class's multiplier is folded into `doc.confidence` and the matched class id is recorded on `confidenceTrace.classPrior`. First match wins (specific classes before catch-all).

Witness-based echo dampener

Modifies `computeL1Confidence` — when 3+ co-located findings share the same evidence fingerprint (first 80 normalised chars of joined `evidence[]`), apply a ×0.75 dampener. Catches "correlated failure cascades" where multiple reviewers latch onto the same superficial cue instead of corroborating independently. Research ref: Microsoft "Tree of Reviews" (2024).

Triggers only when:

1. 3+ co-located docs have non-empty `evidence[]`
2. 3+ distinct fingerprints emerge (i.e. evidence is actually populated)
3. Majority (≥half rounded down) share the same fingerprint

Existing tests use empty evidence arrays → dampener stays off → zero behaviour change for prior test suite.

Measurement

No live bench run this session — budget-conscious. Unit tests pin the regression against actual 2026-04-20 FP samples captured from runs 24669968011 / 24670411622 / 24670855655 (all against `fp-moderator-regex`). Next session's first bench run will measure real impact.

Test plan

• 17 finding-class-scorer tests (every class's pattern + negative cases for real bugs)
• 6 witness echo-dampener tests
• `hallucination-filter.test.ts` helper padding rewritten to avoid accidentally matching "may-throw"/"missing-validation" priors
• Full suite: 3402 → 3425 passing
• `pnpm typecheck` clean

Risks

1. Prior table drift — the multipliers reflect our current OpenRouter reviewer pool. Swapping to a significantly different model mix may invalidate the numbers. One central table (`FINDING_CLASS_PRIORS`) is the tuning point.
2. Genuine bugs in FP-heavy classes — real ReDoS / may-throw bugs will take the prior penalty. This is tolerable because `generic-potential` has the lightest dampener (×0.85), evidence-heavy real findings likely also score high on check 6, and suggestion-verifier still runs for CRITICAL+. If a real bug is silenced, the Phase 2 bench measurement will show the recall regression.
3. Echo dampener false triggers — reviewers with genuinely identical concise evidence "line 10 has off-by-one" could both hash to the same prefix. Mitigated by requiring 3+ matching docs (not 2+), and existing corroboration still counts them.

[github-actions]

🟠 CodeAgora: NEEDS HUMAN REVIEW

1 verify · 4 ignore | 5 reviewers · 1 debates

The only discussed CRITICAL issue has mixed confidence and only reaches 48%, with one reviewer explicitly disputing that it is a blocking security defect and framing it as a robustness concern rather than a proven vulnerability. The evidence presented suggests a possible missing guard around activeReviewers in computeL1Confidence, but it is not sufficiently substantiated to confidently reject the change, especially given the triage guidance that low-confidence critical findings should be routed for human review rather than treated as blocking. The unconfirmed single-reviewer issues also remain unvalidated, so the overall change is not clearly unsafe, but it is not proven safe enough to merge without human confirmation.

Verify

	File	Issue	Confidence
🔴	packages/core/src/pipeline/confidence.ts:68	Missing null guard for activeReviewers in computeL1Confidence	🟡 48%

4 suggestion(s)

• packages/core/src/pipeline/hallucination-filter.ts:238 — Potential undefined reference in hallucination-filter.ts
• packages/core/src/tests/hallucination-filter.test.ts:87 — Test inconsistency in hallucination-filter.test.ts
• packages/core/src/pipeline/finding-class-scorer.ts:114 — Potential null/undefined string concatenation in matchFindingClass
• packages/core/src/pipeline/confidence.ts:120 — Echo dampener condition may be too lenient

Issue distribution (4 file(s))

File	Issues
packages/core/src/pipeline/confidence.ts	████████████ 2
packages/core/src/pipeline/hallucination-filter.ts	██████ 1
packages/core/src/tests/hallucination-filter.test.ts	██████ 1
packages/core/src/pipeline/finding-class-scorer.ts	██████ 1

Agent consensus log (1 discussion(s))

✅ d001 — 1 round(s), consensus → CRITICAL

Verdict: CRITICAL — Majority consensus (2/3 agree)

---

_{CodeAgora · Session: 2026-04-20/001}

[github-actions]

🔴 CRITICAL — Missing null guard for activeReviewers in computeL1Confidence

Confidence: 🟡 48%

Problem: In packages/core/src/pipeline/confidence.ts:68

The function computeL1Confidence calculates agreementRate using division by activeReviewers:

const agreementRate = Math.min(100, Math.round((agreeing / activeReviewers) * 100));

If activeReviewers is passed as 0 or undefined, this produces NaN or Infinity, corrupting downstream confidence calculations.

Evidence:

1. Line 68 shows division by activeReviewers without guard
2. No validation at function entry
3. The function is exported and could be called with any values

Suggested change

	if (typeof activeReviewers !== 'number' || activeReviewers <= 0) {
	return doc.confidence ?? 50;
	}

✅ Discussion d001 — 1 round(s), consensus

Verdict: CRITICAL — Majority consensus (2/3 agree)

_{Flagged by: r3  |  CodeAgora}

[github-actions]

🟡 WARNING — Potential undefined reference in hallucination-filter.ts

Confidence: 🔴 24%

Problem: In packages/core/src/pipeline/hallucination-filter.ts:238-244

The classMatch variable may be used after it's potentially used in the confidence penalty logic, but there's no null check when building the confidenceTrace object.

Evidence:

1. Line 228: const classMatch = matchFindingClass(doc);
2. Lines 229-231: Uses classMatch with a conditional check
3. Lines 238-244: Builds confidenceTrace with classMatch spread

Suggestion: Either:

1. Only add classPrior to trace when a penalty was actually applied (when classMatch.multiplier < 1.0), OR
2. Keep the current behavior but update the trace output to clarify "matched" vs "applied"

---

_{Flagged by: r3  |  CodeAgora}

[github-actions]

🟡 WARNING — Test inconsistency in hallucination-filter.test.ts

Confidence: 🔴 22%

Problem: In packages/core/src/tests/hallucination-filter.test.ts:87-94 and 282-289

The test text was changed from "unvalidated argument" to "raw numeric argument" to avoid false positive matching on the missing-validation check. However:

1. The zero-width prior includes pattern /\bu\+200b\b/i - the new text "raw numeric" contains "u" followed by "+" followed by "200b" in sequence ("raw numeric argument into setTimeoutWrapperu + 200b")? No, this doesn't match.

2. Actually, the real issue: The test change weakens the test coverage - now a finding that says "raw numeric argument" would NOT be caught by missing-validation, but a finding saying "unvalidated argument" WOULD be. This could allow real FP findings to slip through.

Evidence:

1. Old text: "unvalidated argument" - matches /\bunvalidated\b/i
2. New text: "raw numeric argument" - does NOT match any pattern
3. The test expects 80 confidence (no penalty), but this change means the test doesn't actually verify the missing-validation check is working

Suggested change

	`At src/utils.ts:10 inside the introduced call: ${snippet} ` +
	const classMatch = matchFindingClass(doc);
	if (classMatch && classMatch.multiplier < 1.0) {
	const penalized = Math.round((doc.confidence ?? 50) * classMatch.multiplier);
	doc.confidence = penalized;
	}
	// ... later ...
	doc.confidenceTrace = {
	...(classMatch ? { classPrior: classMatch.id } : {}),
	};

_{Flagged by: r3  |  CodeAgora}

[github-actions]

🟡 WARNING — Potential null/undefined string concatenation in matchFindingClass

Confidence: 🔴 30%

Problem: In packages/core/src/pipeline/finding-class-scorer.ts:114

The matchFindingClass function concatenates issueTitle and problem without checking if they are null or undefined. If either field is null/undefined, it would create literal "null" or "undefined" strings in the haystack, potentially leading to unexpected regex matches.

Evidence:

1. Line 114 constructs haystack as: const haystack = ${doc.issueTitle}\n${doc.problem};
2. No null/undefined checks before string interpolation
3. If doc.issueTitle is null, haystack becomes "null\n[problem]"
4. A regex like /null/i (not present but could be added later) would match unexpectedly

Suggested change

	for (const prior of FINDING_CLASS_PRIORS) {
	const haystack = `${doc.issueTitle ?? ''}\n${doc.problem ?? ''}`;

_{Flagged by: r5  |  CodeAgora}

[github-actions]

🟡 WARNING — Echo dampener condition may be too lenient

Confidence: 🔴 20%

Problem: In packages/core/src/pipeline/confidence.ts:120

The echo dampener triggers when distinctCount <= Math.floor(fingerprints.length / 2). For 3 fingerprints, this allows 1 distinct fingerprint (2 identical) to trigger the dampener, which may not represent a true "majority" echo cascade.

Evidence:

1. For 3 fingerprints: Math.floor(3/2) = 1, so distinctCount <= 1 triggers
2. This means 2 identical + 1 different triggers dampener
3. The comment says "majority share the same" but 2/3 is not a majority

Suggestion: Change condition to require strict majority: distinctCount < Math.ceil(fingerprints.length / 2) or distinctCount * 2 < fingerprints.length

_{Flagged by: r5  |  CodeAgora}