feat(competition): relevance-guard engine-lane submission - #574
Conversation
one uniform suspect penalty scaled by query relevance, so hearsay, possessive fragments, stored instructions, and stale conflicting values fall out of a tight candidate pool in damage order; conflict collapse marks values contradicted by a change-of-state memory; past-tense questions suspend collapse because history is the answer there. verified with the ci-exact loop over seeds 1-12: challenger 0.7083 vs champion 0.6583, diff +0.0500, band 0.0381 - dethroned.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
pull_request_target checks out the DEFAULT branch when no ref is given, so the engine gate scored from main - which carries the workflow but not the scorer or the strategies - and died on a missing file (pr #574's first run). the kit gate got this exact pin in 683e0a3; the engine gate now matches, and the seed identity comes from the checked-out base tip instead of github.sha for the same reason.
pull_request_target checks out the DEFAULT branch when no ref is given, so the engine gate scored from main - which carries the workflow but not the scorer or the strategies - and died on a missing file (pr #574's first run). the kit gate got this exact pin in 683e0a3; the engine gate now matches, and the seed identity comes from the checked-out base tip instead of github.sha for the same reason.
pull_request_target checks out the DEFAULT branch when no ref is given, so the engine gate scored from main - which carries the workflow but not the scorer or the strategies - and died on a missing file (pr #574's first run). the kit gate got this exact pin in 683e0a3; the engine gate now matches, and the seed identity comes from the checked-out base tip instead of github.sha for the same reason.
|
koth engine lane - dethroned {
"date": "2026-07-28",
"base_sha": "b11ac52a1fbd1172887eb924a8e170aec15eef5a",
"seeds": [
67078370866574,
116114090607205,
108999320470477,
114845009661335,
248394666013973,
108843862080367,
274140852163751,
57307557147568
],
"lane": "engine",
"champion": {
"scores": [
0.5,
0.7,
0.5,
0.7,
0.7,
0.8,
0.7,
0.7
],
"mean": 0.6625
},
"challenger": {
"scores": [
0.5,
0.7,
0.6,
0.7,
0.7,
0.9,
0.8,
0.7
],
"mean": 0.7
},
"mean_diff": 0.037500000000000006,
"se": 0.018298126367784998,
"band": 0.03586432768085859,
"dethroned": true
}engine code is NOT auto-merged. a winning strategy earns the
|
|
koth engine lane - dethroned {
"date": "2026-07-28",
"base_sha": "b11ac52a1fbd1172887eb924a8e170aec15eef5a",
"seeds": [
67078370866574,
116114090607205,
108999320470477,
114845009661335,
248394666013973,
108843862080367,
274140852163751,
57307557147568
],
"lane": "engine",
"champion": {
"scores": [
0.5,
0.7,
0.5,
0.7,
0.7,
0.8,
0.7,
0.7
],
"mean": 0.6625
},
"challenger": {
"scores": [
0.5,
0.7,
0.6,
0.7,
0.7,
0.9,
0.8,
0.7
],
"mean": 0.7
},
"mean_diff": 0.037500000000000006,
"se": 0.018298126367784998,
"band": 0.03586432768085859,
"dethroned": true
}engine code is NOT auto-merged. a winning strategy earns the
|
engine-lane entry for season 1 (#573): relevance-guarded ranking.
one general idea carried through: every class of suspect memory — hearsay, third-person possessive fragments, stored instructions, stale values contradicted by a change-of-state memory — gets a single uniform penalty scaled by how much the suspect matches the query. suspects fall out of a tight candidate pool in damage order, so the item the question is actually about is the one excluded first. past-tense questions ("what was ... before it changed?") suspend the collapse, because there the stale side is the answer. no benchmark-keyed logic: no value lists, no template matching, no category dispatch.
local verification with the ci-exact loop:
per the lane rules the gate scores this in a sandbox and it never auto-merges; on a human merge above the band, the ledger sweep promotes it to reigning champion and the threshold ratchets.