Skip to content

feat(retrieval): ship the provenance champion as the retrieval default - #572

Merged
plind-junior merged 2 commits into
testfrom
feat/koth-engine-lane
Jul 28, 2026
Merged

feat(retrieval): ship the provenance champion as the retrieval default#572
plind-junior merged 2 commits into
testfrom
feat/koth-engine-lane

Conversation

@plind-junior

Copy link
Copy Markdown
Member

follow-up to #569: the competition round-1 hardening and the champion promotion.

  • the engine-lane winner (feat(competition): provenance-rank engine-lane submission #567) ships in-package as vouch.strategies.provenance and becomes the reigning champion: contrib/strategies/baseline.py delegates to it, so challengers must now beat real provenance-aware ranking, not identity order.
  • new KBs get the champion as the final reorder stage via the starter config (retrieval.strategy); existing KBs keep byte-identical ordering until they add the key; strategy: null opts out. the bench is unaffected (throwaway KBs write their own config).
  • engine gate hardening from review: actions pinned to immutable shas, delete-only strategy prs classified as normal instead of failing messy.

validation: full suite (1833 passed), mypy, ruff; paired bench verifies the champion at +0.05 composite over identity baseline (band 0.0381) with the ci-exact loop.

pin checkout and setup-python to immutable commit shas (mutable tags in
a pull_request_target workflow are a supply-chain surface), and classify
a delete-only strategy pr as normal instead of engine - the filename
shape matched but there is nothing to fetch at the head sha, so the
gate failed messy instead of passing through.
the engine-lane winner (pr #567) moves in-package as
vouch.strategies.provenance and becomes what challengers must beat:
contrib/strategies/baseline.py now delegates to it instead of returning
identity order. new KBs get it as the final reorder stage via the
starter config (retrieval.strategy); existing KBs keep byte-identical
ordering until they add the key, and strategy: null opts out. the bench
is unaffected - its throwaway KBs write their own config, so kits and
strategies keep being measured explicitly.

rerank-isolation tests opt out of the strategy stage, since the champion
is final-say and would re-sort the asserted window.
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@plind-junior, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 14 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 49592b60-0f0b-46e1-9cb9-079e3bf7861b

📥 Commits

Reviewing files that changed from the base of the PR and between d0c6f8d and 6f95ad5.

📒 Files selected for processing (8)
  • .github/workflows/koth-engine-gate.yml
  • CHANGELOG.md
  • contrib/strategies/baseline.py
  • src/vouch/storage.py
  • src/vouch/strategies/__init__.py
  • src/vouch/strategies/provenance.py
  • tests/test_retrieval_backend.py
  • tests/test_strategy.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/koth-engine-lane

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added docs documentation, specs, examples, and repo guidance ci github actions and automation storage kb storage, migrations, schemas, and proposals retrieval context, search, synthesis, and evaluation tests tests and fixtures size: S 50-199 changed non-doc lines labels Jul 28, 2026
@plind-junior
plind-junior merged commit b806941 into test Jul 28, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci github actions and automation docs documentation, specs, examples, and repo guidance retrieval context, search, synthesis, and evaluation size: S 50-199 changed non-doc lines storage kb storage, migrations, schemas, and proposals tests tests and fixtures

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant