Skip to content

fix: gate contradiction hunt BM25 sensitivity#19

Closed
bob-codedaptive wants to merge 1 commit into
develop/1.0.xfrom
codex/fix-bm25-hunt-sensitive-data-leak
Closed

fix: gate contradiction hunt BM25 sensitivity#19
bob-codedaptive wants to merge 1 commit into
develop/1.0.xfrom
codex/fix-bm25-hunt-sensitive-data-leak

Conversation

@bob-codedaptive

Copy link
Copy Markdown
Member

Motivation

  • The BM25 corpus lane could surface candidate drawer IDs from the corpus index without applying the normal sensitivity ceiling, allowing borderline candidates to carry raw content prefixes that leak restricted/secret content to MCP callers.
  • The change enforces the same elevated-or-below sensitivity ceiling used by recall so BM25 probes, BM25 hits, and final candidate evaluation cannot produce snippet-bearing results for higher-sensitivity drawers.

Description

  • Add a visibility predicate that returns true only for drawers with adjectiveSensitivity <= Elevated and apply it in the Swift hunter to gate probe drawers, BM25 hit drawers, and the final candidate-pair evaluation path (packages/kits/GeniusLocusKit/Sources/GeniusLocusKit/Brain/ContradictionHunt.swift).
  • Add the same closure in the Rust coordinator and use it to skip BM25 probe drawers, BM25 source hits, and any candidate pairs that contain non-visible drawers (packages/kits/GeniusLocusKit/rust/src/coordinator.rs).
  • The gating prevents restricted/secret drawers from being admitted into the snippet-producing borderline path while preserving existing BM25 candidate generation and strong-cue capture behavior.

Testing

  • git diff --check was executed and reported no whitespace/patch errors.
  • swift test --package-path packages/kits/GeniusLocusKit --skip-build was attempted but blocked by network dependency fetch errors (SwiftPM GitHub fetches returned HTTP 403), so unit tests could not be run here.
  • cargo check --manifest-path packages/kits/GeniusLocusKit/rust/Cargo.toml was attempted but blocked by network dependency fetch errors (crates.io index fetch returned HTTP 403), so Rust build checks could not be completed here.

Codex Task

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution to MOOTx01. Before it can be merged we need you to sign our Contributor License Agreement. You can read it here: CLA.md. To sign, post a comment on this pull request with exactly the following text:


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3575a0cb4e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

query: query, limit: Self.huntBM25CandidateK)
else { continue }
for hit in hits where hit.sourceID != probeDrawer.id {
let hitDrawers = (try? await estate.hydrateBodies(ids: [hit.sourceID])) ?? []

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Batch BM25 hit hydration before filtering

When the corpus lane has many hits, this performs a separate hydrateBodies storage read for every BM25 result before the existing batched candidate hydration below; with the default 50 probes × 20 hits, and larger on-demand sweeps, that can turn one chunked by-id load into hundreds or thousands of round trips while re-reading the same drawers. Consider collecting the hit source IDs and applying the sensitivity check from a single batch load/map instead.

Useful? React with 👍 / 👎.

@bob-codedaptive

Copy link
Copy Markdown
Member Author

loses on two counts: no tests (fails the parity-regression bar), and it adds a performance regression — a per-hit hydrateBodies call that defeats the intentional batched late-hydration. Its extra "defense-in-depth" gates protect nothing the chokepoint gate already covers (restricted bodies are only used as internal BM25 query strings upstream, never disclosed).

  • Keeper has no residual gap for the disclosure finding — the gate excludes a pair if either endpoint exceeds .elevated, before any snippet is built. Complete in both legs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant