Skip to content

feat: restore evidence-triggered event router - #5

Draft
Hubujiu wants to merge 33 commits into
mainfrom
experiment/progressive-ladders
Draft

feat: restore evidence-triggered event router#5
Hubujiu wants to merge 33 commits into
mainfrom
experiment/progressive-ladders

Conversation

@Hubujiu

@Hubujiu Hubujiu commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Summary

  • replace the rejected E0-E3 / R0-R3 specialist tree with the compact Core/Direct plus Debugging, Decision, and Implementation event router
  • keep retrieval orthogonal: targeted, bounded lexical, structural, and bounded exhaustive/authoritative evidence
  • keep requirements interviewing explicit-only and remove unsupported specialist runtime leaves
  • preserve the rejected experiment and its lessons under evolution/
  • publish current-only v1.5 n=3 public and held-out evidence

Release result

Candidate runtime: 30ac7e70b425b3f02f7bf4e21cb4809f0e4d6c2c

  • public matrix: 294/294 determinate
  • Delivery: 54/54
  • Debug: 40/42 (correctness 42/42; sibling safety 40/42)
  • Decision: 29/30
  • Native Behavior: 52/54
  • public event reasoning: 113/114
  • public Router after contract adjudication: 107/114 (reasoning 113/114, retrieval 108/114)
  • held-out: 61/66 quality cells, 18/22 tasks stable-pass across 3 frozen repositories
  • held-out traces: 66/66 valid; 0/66 spontaneous requirements-interview activation

The recorded raw Router score was 99/114. Three frozen Retrieval expectations contradicted the active STRUCTURAL relationship-mapping definition; the adjudicated count changes expectations only, not model output. Held-out active reasoning expectations were also decoupled from rejected capability_path metadata. Both corrections have contract tests and are documented in benchmarks/results/v1.5/.

No no-skill, Ponytail, combined-skill, or prior-version arm was rerun. Historical v1.2 is an offline, non-paired reference only.

Validation

  • python -m unittest discover -s benchmarks -p 'test_*.py' — 55 passed
  • pwsh -NoProfile -File benchmarks/run.ps1 -SelfTest — 39 passed
  • pwsh -NoProfile -File benchmarks/run.ps1 -ProgressiveSelfTest — 8 passed
  • public full current-only n=3 — 294 cells, 0 indeterminate
  • held-out current-only n=3 — 66 cells, 0 indeterminate
  • PR CI validates SKILL.md with skills-ref and reruns the harness tests on Ubuntu

Evidence boundary

The report preserves every residual failure and its adjudication. No runtime wording was changed after the final n=3 run, and no case-specific noun was added to improve a benchmark score.

@Hubujiu Hubujiu changed the title experiment: validate progressive capability tree feat: restore evidence-triggered event router Aug 31, 2026
@Hubujiu
Hubujiu marked this pull request as ready for review August 31, 2026 12:00
@Hubujiu
Hubujiu marked this pull request as draft August 31, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant