diff --git a/AGENTS.md b/AGENTS.md
index a008d0c..28bd34f 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,23 +1,50 @@
# Practical Coding
-This repository is an Agent Skill. If you are a coding agent reading this file from a checkout, apply the skill as follows.
+This repository is an Agent Skill. If you are a coding agent reading this file from a checkout, apply the Skill from [`SKILL.md`](SKILL.md).
-Read [`SKILL.md`](SKILL.md) for the route-agnostic shortest-path Core, the three-branch Event Router, and the Retrieval Policy. The Core always applies. Ordinary well-specified work stays Direct with no reasoning reference and no worker.
+Practical Coding is an adaptive-rigor system, not a task classifier. The Core always applies. Start from the cheapest sufficient action and add stricter reasoning only when a present blocker requires it.
-The Event Router handles only unresolved blockers that change how the task must be reasoned about:
+## 1. Decision Gate
-| Trigger | Reasoning module |
+Before execution, determine whether a material unresolved choice blocks or materially changes the next safe action.
+
+- If no, continue to execution.
+- If yes, read [`references/decision.md`](references/decision.md) and resolve only that decision frontier.
+
+A request-, repository-, or authority-settled choice is input, not a Decision. Cheap reversible choices use the project or platform default. Resolve discoverable facts before asking the user; only genuinely user-owned scope, compatibility, cost, preference, or risk choices should remain as questions.
+
+After the choice is settled, continue with the Core. Do not assume the logical end of Decision removes `decision.md` from model context.
+
+## 2. Execution Escalation
+
+Direct is the default execution state: Core only.
+
+| Present blocker | Extra rigor |
|---|---|
-| An observed failure, regression, or incorrect behavior still lacks an evidenced cause | [`references/debugging.md`](references/debugging.md) |
-| A material unresolved user-owned choice about architecture, whether or which external dependency/implementation to adopt, APIs, data models, or compatibility would change the next action | [`references/decision.md`](references/decision.md) |
-| An unknown contract/invariant, an unresolved material risk boundary (security/permissions, irreversible side effects, persistence/migration, concurrency/transactions, compatibility), or insufficient evidence for a risky material claim blocks safe execution | [`references/implementation.md`](references/implementation.md) |
+| Observed failure or regression still lacks an evidenced cause | [`references/debugging.md`](references/debugging.md) |
+| Safe execution is blocked by an unknown contract/invariant, unresolved material risk boundary, or unresolved sufficient evidence for a risky claim | [`references/implementation.md`](references/implementation.md) |
+
+Debugging and Implementation are escalation profiles, not sequential stages. Do not classify by task nouns, file count, code size, or apparent difficulty. A diagnosed bug can be Direct. A security, persistence, migration, concurrency, or compatibility edit can be Direct when the governing boundary, affected surface, and sufficient check are already established.
+
+If one loaded profile resolves its blocker and a materially different blocker later appears, reassess from the Core. Do not accumulate another large reasoning reference in the root merely because the task continued; isolate substantial follow-up work when the context saved exceeds handoff cost.
+
+## 3. Retrieval Policy
+
+Retrieval is independent from Decision and execution rigor. Use the cheapest sufficient available path:
+
+1. current context / known path / known symbol;
+2. bounded or ranked source discovery, falling back to ordinary filename/text/symbol search;
+3. an already-available structural index only for relationship-heavy questions where it materially reduces exploration;
+4. current-source verification for material conclusions.
+
+Read [`references/navigation.md`](references/navigation.md) only when broad retrieval itself is substantial enough to need the detailed procedure. Host-native ranked search, FFF-style retrieval, and `DeusData/codebase-memory-mcp` are optional capabilities, not project requirements. Missing capabilities fall back without installing tooling or changing project configuration solely for retrieval.
-Load exactly one first-match reasoning module in addition to the Core. A choice already settled by the request or repository is input, not a Decision event. A security, persistence, migration, concurrency, or compatibility noun is not itself an Implementation event when the governing boundary, affected surface, and sufficient check are already established. Do not treat file count, task nouns, search needs, or the existence of another library as routing evidence. If a different blocker appears later, reassess it without accumulating another reasoning reference in the root; use the Core when sufficient or isolate substantial follow-up work when the saved context exceeds handoff cost.
+Retrieval levels are cost bounds rather than exact semantic labels. A cheap bounded search may be acceptable where a targeted read would also suffice; an unnecessary structural exploration is not.
-Navigation is not a fourth Event Router branch. Code retrieval follows the cheapest sufficient available path: known source first, then bounded/ranked source discovery, then an already-available structural index only when relationship queries materially reduce exploration, followed by current-source verification for material claims. Routine targeted lookup needs no Navigation reference.
+## 4. Isolation Gate
-Read [`references/navigation.md`](references/navigation.md) only when broad retrieval itself is substantial enough to need the detailed procedure. Host-native ranked search, FFF-style retrieval, and `DeusData/codebase-memory-mcp` are optional capabilities, not project requirements. Use them only when already available; otherwise fall back to ordinary source search without installing tooling or changing project configuration solely for retrieval. An already-integrated structural backend may maintain or refresh its own index as part of normal use.
+The root owns user intent, authorization, repository state, integration, and the final completion claim. Keep the root to the Core plus at most one loaded reasoning reference at a time.
-For a substantial triggered event, prefer an isolated no-history worker only when its context savings exceed handoff cost; otherwise load the one selected reasoning reference in the root agent. Keep the root to the Core plus at most one loaded reasoning reference for the task. If broad mapping becomes expensive while another reasoning reference is already resident, prefer a read-only Navigation worker rather than loading a second large reference into the root.
+The root never reads [`references/delegation.md`](references/delegation.md). When isolation clearly saves more context than its handoff cost, dispatch one worker with `delegation.md` plus exactly one assigned reference and a compact capsule of settled choices, verified facts, scope, repository state, and success conditions.
-The root agent owns user intent, authorization, repository state, routing, integration, and the final completion claim. A worker reads [`references/delegation.md`](references/delegation.md) plus exactly one assigned reference and returns a compact capsule. Decision, Debugging, and Navigation workers are read-only. An Implementation worker may write only when its assignment explicitly includes implementation, must have a bounded scope, and must be the sole writer there. Treat a capsule as stale after relevant repository changes.
+Decision, Debugging, and Navigation workers are read-only. An Implementation worker may write only when explicitly assigned a bounded implementation scope and must be the sole writer there. Never use overlapping writers or worker pipelines. Treat a worker capsule as stale after relevant repository changes.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 295ab75..e282613 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -1,20 +1,48 @@
# Contributing
-Contributions should preserve Practical Coding as one compact Skill with a small route-agnostic Core, three independently loadable reasoning modules, and a retrieval policy that does not become a permanent prompt tax.
+Contributions should preserve Practical Coding as one compact **adaptive-rigor Skill**: a small route-agnostic Core, one pre-execution Decision Gate, two independently loadable execution-rigor profiles, and a Retrieval Policy that does not become a permanent prompt tax.
-- Keep `SKILL.md` as a small shortest-path Core plus Event Router and compact Retrieval Policy. Do not add routing intensity modes unless a mature cross-agent mechanism materially improves evidence.
-- The Event Router owns only unresolved reasoning blockers: Debugging, Decision, and Implementation. Navigation is retrieval, not a fourth reasoning route.
-- Do not retune Core or module wording from a single failed benchmark cell, and do not add case-specific bans named after benchmark tasks.
-- Keep the Direct Path real: simple, well-specified work must not require a reference, broad repository scan, or worker.
+- Keep `SKILL.md` small. The Core must contain only rules that apply to essentially every coding task.
+- Treat **Decision** differently from execution rigor. Load `decision.md` only when a material unresolved choice blocks or materially changes the next safe action.
+- Treat **Direct** as the default execution state, not a route or reference.
+- Treat **Debugging** and **Implementation** as alternative escalation profiles, not a `Direct → Debugging → Implementation` pipeline.
+- Debugging is justified only while an observed failure lacks an evidenced cause. If the cause is already established and the safe fix is known, stay Direct.
+- Implementation rigor is justified only while safe execution is blocked by an unknown contract/invariant, unresolved material risk boundary, or insufficient evidence for a risky claim. Security, persistence, migration, concurrency, compatibility, or file count alone are not triggers.
+- Do not retune Core or reference wording from a single failed benchmark cell, and do not add case-specific bans or trigger nouns named after public benchmark tasks.
+- Keep simple, well-specified work genuinely cheap: no reasoning reference, broad repository scan, plan document, or worker merely because the task involves code.
- Routine targeted source lookup must not require `references/navigation.md`. Load that reference only when broad retrieval itself is substantial enough to justify the prompt cost.
-- Preserve context isolation. The root should normally carry the Core plus at most one reasoning reference. If broad mapping becomes expensive while another reference is resident, prefer a read-only Navigation worker when saved context exceeds handoff cost.
-- Workers read `references/delegation.md` plus exactly one assigned reference and return compact evidence capsules, not transcripts.
-- Keep shared-workspace workers read-only by default. An Implementation worker may write only when its assignment explicitly includes implementation, with one bounded writer scope.
-- Prefer strengthening an existing module or retrieval primitive over adding another module. Verification remains part of Core or Implementation; do not create a mandatory Verification route.
+- Preserve context isolation. The root should normally carry the Core plus at most one large reasoning reference at a time. Logical state transitions do not remove already-read context.
+- If a later blocker needs another large reference, isolate substantial follow-up work only when saved context exceeds handoff cost. Do not create worker pipelines.
+- Workers read `references/delegation.md` plus exactly one assigned reference and return compact evidence capsules, not raw transcripts.
+- Keep shared-workspace workers read-only by default. An Implementation worker may write only when explicitly assigned one bounded writer scope.
+- Prefer strengthening an existing profile or retrieval primitive over adding another module. Verification remains part of Core or Implementation rigor; do not create a mandatory Verification stage.
- Do not introduce mandatory plans, execution documents, Git workflows, tests, reviews, documentation, or tool-specific ceremony as universal gates.
- Preserve reuse-before-invention, mature-implementation-first, risk-proportional verification, evidence-driven debugging, and resistance to speculative code and defensive bloat.
- Avoid new scripts, dependencies, configuration, generated project files, or persistent services unless they solve a demonstrated project need rather than merely making retrieval possible.
+## Decision Gate discipline
+
+A proposed Decision trigger must answer:
+
+> Without settling this choice, can the agent already know the next safe action?
+
+If yes, it is not a blocking Decision Gate. Repository conventions, authoritative constraints, or a cheap reversible default should settle ordinary choices without an interview.
+
+When a choice genuinely remains user-owned, ask only for the minimum scope, compatibility, cost, preference, or risk information that changes the next action. A Decision module should converge and return a compact execution capsule; it should not become a design-document generator.
+
+## Execution-rigor discipline
+
+When changing `debugging.md` or `implementation.md`, test both positive and negative boundaries:
+
+- unknown-cause failure → Debugging;
+- diagnosed failure → Direct;
+- unresolved material execution boundary → Implementation;
+- already-mapped risk boundary with known affected surface and sufficient check → Direct;
+- Debugging completed with no remaining boundary blocker → Direct;
+- Debugging completed but a materially different execution boundary remains unresolved → Implementation only as a new escalation, preferably isolated if a second large reference would accumulate.
+
+The amount of code, number of files, or perceived task difficulty is not a valid substitute for these blocker conditions.
+
## Retrieval backends
Practical Coding manages retrieval cost, not ownership of a particular search engine.
@@ -27,6 +55,31 @@ Practical Coding manages retrieval cost, not ownership of a particular search en
Do not reimplement mature retrieval engines inside Practical Coding merely to avoid an optional external capability. Conversely, do not turn an optional capability into a hard dependency or automatic installation side effect.
+### Retrieval benchmark discipline
+
+The v1.3 benchmark uses a minimum-sufficient / maximum-reasonable interval instead of requiring one unique exact Retrieval label.
+
+A benchmark-case change must distinguish:
+
+- **insufficient retrieval** — below the minimum context needed for safe action;
+- **acceptable retrieval** — within the frozen cost interval;
+- **excessive retrieval** — above the maximum reasonable cost for the task.
+
+Do not widen an interval after seeing a failed run merely to make the cell pass. Change it only before a validation cycle or after documenting a genuine benchmark-instrument defect.
+
+## Benchmark changes
+
+The canonical v1.3 runner is `benchmarks/run_catalog.py` (runner v2.1). `run_benchmarks.py` remains the v2.0 execution core for historical interpretability; do not silently rewrite old result directories or reinterpret old Router scores under the new schema.
+
+Any behavior-changing contribution should preserve or add regression coverage for the mechanism it changes. In particular, the current transition corpus must continue to cover:
+
+- Decision → Direct;
+- Decision → Implementation;
+- Debugging → Direct;
+- Debugging → Implementation.
+
+Before publishing claims, follow [`benchmarks/NEXT_VALIDATION.md`](benchmarks/NEXT_VALIDATION.md). Public cells that influenced wording are regression evidence, not independent held-out evidence.
+
## Mature implementation first
For any non-trivial capability with credible prior art:
diff --git a/README.md b/README.md
index 29fda4d..06c9c03 100644
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
- 
+ 
@@ -11,170 +11,192 @@
English · 简体中文
-> ## The right amount of engineering and context for every coding task.
+> ## The smallest sufficient engineering rigor for every coding task.
>
-> **Simple work stays direct. Unknown bugs get root-cause debugging. Risky changes get rigor. Code retrieval stops at the cheapest sufficient path.**
+> **Start cheap. Resolve only choices that block the next action. Add debugging or implementation rigor only when evidence says Direct is insufficient. Retrieve only the context worth paying for.**
-Practical Coding is a lean Agent Skill for coding assistants. It controls two costs independently:
+Practical Coding is a lean Agent Skill for coding assistants. It is not a four-way task classifier. It is an **adaptive engineering-rigor system** built around four independent controls:
-1. **Reasoning cost:** only unresolved blockers may load Debugging, Decision, or Implementation.
-2. **Context cost:** code discovery progresses from known source to bounded/ranked search to structural indexes only when each stronger rung is actually needed.
+1. **Core** — the minimum rules every coding task needs.
+2. **Decision Gate** — resolve a material choice only when it blocks or changes the next safe action.
+3. **Execution Escalation** — start Direct; add Debugging or Implementation rigor only for the blocker actually present.
+4. **Retrieval + Isolation** — pay only for repository context and extra contexts that materially help.
```bash
npx skills@latest add Hubujiu/practical-coding
```
-## What changes with v1.2
-
-Navigation is no longer a fourth Event Router branch. It is a retrieval policy shared by Direct work and every routed event.
-
-| Situation | Practical Coding behavior |
-|---|---|
-| Rename, CSS tweak, known local edit | **Direct Path** — Core only |
-| Observed bug with unknown cause | Core + **Debugging** |
-| Material unresolved architecture/API/dependency choice | Core + **Decision** |
-| Unknown contract or unresolved migration, permission, persistence, concurrency, compatibility, or other material risk boundary blocks safe work | Core + **Implementation** |
-| Need to locate code | Use the cheapest sufficient retrieval capability; no reasoning route is selected merely because search is needed |
-| Broad relationship-heavy mapping | Prefer an already-available structural index when it saves exploration; otherwise fall back to bounded source search |
-
-The main invariant is now:
-
-> **Core + at most one reasoning module; retrieval is orthogonal and capability-based.** Risk-related nouns do not trigger Implementation after the governing boundary, affected surface, and sufficient check are already established.
-
-Legacy `.practical-coding.yaml` files from v1.1 are no longer read by the Skill and can be removed. Retrieval capability is discovered from the current host/environment instead of stored as a project preference.
-
----
-
-## Architecture
+## v1.3 architecture
```mermaid
flowchart TB
- T[User coding task] --> C[Always-On Core]
- C --> E{Present unresolved reasoning blocker?}
- E -->|No| D[Direct Path]
- E -->|Observed failure lacks cause| G[Debugging]
- E -->|Material user-owned choice| A[Decision]
- E -->|Unknown contract / risk boundary| I[Implementation]
-
- D --> R{Need more code context?}
+ T[User coding task] --> C[Core]
+ C --> D{Material unresolved choice blocks next action?}
+ D -->|Yes| Q[Decision Gate]
+ Q --> C2[Choice settled / compact execution capsule]
+ D -->|No| C2
+
+ C2 --> E[Start Direct]
+ E --> S{Core sufficient now?}
+ S -->|Yes| V[Cheapest sufficient verification]
+ S -->|No: failure cause unknown| G[Debugging rigor]
+ S -->|No: safe boundary unknown| I[Implementation rigor]
+ G --> V
+ I --> V
+
+ C --> R[Retrieval Policy]
+ C2 --> R
G --> R
- A --> R
I --> R
+ R --> K[Known source / bounded search / structural retrieval]
- R -->|No| V[Cheapest focused verification]
- R -->|Known path/symbol| K[Targeted read]
- R -->|Location unknown| S[Bounded / ranked source search]
- R -->|Structural relationship needed| X[Available structural index]
-
- K --> V
- S --> V
- X --> Q[Verify material claims in current source]
- Q --> V
- V --> O[Evidence-based completion]
+ C2 --> X{Another large reference later?}
+ X -->|No| E
+ X -->|Yes and context savings exceed handoff| W[Isolated worker]
```
-### Always-On Core
+The important distinction is:
+
+> **Decision determines what the next action is. Execution rigor determines how much discipline that known action needs. Retrieval determines what code context is worth loading.**
+
+### Core
-The resident `SKILL.md` stays short and route-agnostic:
+The resident `SKILL.md` stays route-agnostic:
- define the smallest observable success;
-- stop at the first implementation rung that works;
-- reuse established project primitives and contracts;
-- add no speculative abstractions, options, wrappers, configuration, or scaffolding;
+- reuse the nearest established primitive or contract;
+- avoid speculative abstractions, options, wrappers, configuration, and scaffolding;
- make the smallest coherent reachable change;
-- prefer deletion and boring code;
-- add tests, fallback, validation, comments, or documentation only when a current requirement, contract, project rule, or necessary verification requires them;
-- run the cheapest focused check once;
+- add tests, validation, fallback, comments, or documentation only for a current requirement, established contract, project rule, or necessary verification;
+- run the cheapest focused check once after the final edit;
- claim only what fresh evidence supports.
-### Three reasoning modules
+### Decision Gate
-| Module | Trigger | Purpose |
-|---|---|---|
-| [`debugging.md`](references/debugging.md) | An observed failure still lacks an evidenced cause | Reproduce → earliest broken state → supported cause → root-cause fix |
-| [`decision.md`](references/decision.md) | A material user-owned choice remains open and changes the next action | Resolve the smallest real decision frontier |
-| [`implementation.md`](references/implementation.md) | Safe execution is blocked by an unknown contract/invariant, material risk boundary, or insufficient evidence for a risky claim | Map the boundary, preserve guarantees, and choose sufficient evidence |
+Decision is no longer a peer of Debugging and Implementation.
-The Event Router chooses only among these three. File count, task nouns, search needs, or the mere existence of another library do not select a reasoning module.
+Ask first:
----
+> **Does a material unresolved choice block or materially change the next safe action?**
+
+If yes, load [`decision.md`](references/decision.md). Resolve repository facts and authoritative constraints before asking the user. Only genuinely user-owned scope, compatibility, cost, preference, or risk choices remain as questions.
-## Retrieval: context selection instead of another workflow
+If the request, repository, authoritative evidence, or a cheap reversible default already settles the choice, execution starts immediately.
-Retrieval answers a different question from the Event Router:
+### Execution Escalation
-> **What is the cheapest way to obtain the code context needed for the current task?**
+Direct is the default state, not a module.
-The ladder is deliberately progressive:
+| Current blocker | Rigor |
+|---|---|
+| The next safe action is already known | **Direct — Core only** |
+| An observed failure exists but its cause is not evidenced | Core + [`debugging.md`](references/debugging.md) |
+| Safe execution is blocked by an unknown contract/invariant, unresolved material risk boundary, or insufficient evidence for a risky claim | Core + [`implementation.md`](references/implementation.md) |
-1. **Current context / known target** → read only the known source.
-2. **Unknown location** → use an already-available bounded or ranked source-search primitive.
-3. **No ranked primitive** → fall back to ordinary filename, text, and symbol search such as host search, `rg`, `grep`, or `find`.
-4. **Relationship-heavy question** → use an already-available structural index only when it materially reduces repeated exploration.
-5. **Material conclusion** → verify against current source; source is authoritative.
+Debugging and Implementation are **alternative escalation profiles**, not `Direct → Debugging → Implementation` stages.
-Stop at the first sufficient rung.
+A diagnosed bug may be Direct. A one-line persistence or permission edit may need Implementation rigor. A large multi-file edit may still be Direct when the contract, affected surface, and sufficient check are already established.
-### FFF-style retrieval and Codebase Memory are complementary
+### Retrieval Policy
-| Capability | Best at | Role in Practical Coding |
-|---|---|---|
-| Host-native / FFF-style ranked retrieval | Finding likely files and text candidates with bounded output and ranking signals | Cheap candidate discovery when already available |
-| Ordinary `rg` / filename / symbol search | Exact text, names, small repositories, universal fallback | Zero-special-backend fallback |
-| [`DeusData/codebase-memory-mcp`](https://github.com/DeusData/codebase-memory-mcp) or another structural index | Callers, callees, imports, implementations, dependency edges, cross-file flow | Optional structural retrieval when already available |
+Retrieval remains orthogonal to reasoning rigor:
-Practical Coding does **not** require `@ff-labs/pi-fff`, FFF, Codebase Memory, `.practical-coding.yaml`, or any persistent graph service. It also does not automatically install retrieval tooling merely because a stronger backend would be convenient. Missing capabilities degrade to the next available rung.
+1. current context / known source;
+2. bounded or ranked source discovery;
+3. structural index only when relationship-heavy exploration materially benefits;
+4. current-source verification for material claims.
-`references/navigation.md` contains the detailed broad-retrieval procedure. Routine targeted lookup does not load it.
+Host-native search, FFF-style ranked retrieval, ordinary `rg`/filename/symbol search, and [`DeusData/codebase-memory-mcp`](https://github.com/DeusData/codebase-memory-mcp) are capabilities, not project requirements. Missing stronger tooling falls back without changing project configuration solely for retrieval.
----
+`references/navigation.md` is loaded only for substantial broad retrieval.
-## Context isolation
+### Retrieval is scored as a cost interval
-A textual instruction such as “return to Direct” cannot remove a reference that is already in the model context. Practical Coding therefore treats context isolation as a real resource decision:
+v1.2 exposed an important benchmark flaw: an exact label can punish a reasonable search that is only one cheap rung broader. The v1.3 benchmark therefore separates:
-- Direct work and small routed events use no worker.
-- The root should normally carry the Core plus at most one reasoning reference.
-- Routine source search uses host tools directly without loading Navigation.
-- If broad mapping becomes expensive while Debugging, Decision, or Implementation is already resident, a read-only Navigation worker is preferred only when the context saved exceeds handoff cost.
-- Workers return compact evidence capsules, not raw search transcripts or graph dumps.
+- **minimum sufficient retrieval**, and
+- **maximum reasonable retrieval cost**.
-This is how progressive disclosure remains a context optimization rather than only a file-organization convention.
+For example, a known concept whose exact file is not available may reasonably use either a targeted read or bounded search. Structural exploration remains excessive unless the task actually needs relationships.
+
+### Context isolation
+
+A prompt cannot unload a reference that is already in model context. Therefore "return to Direct" is a logical state transition, not a context reset.
+
+The root keeps the Core plus at most one large reasoning reference at a time. If Decision is already resident and execution later needs a different substantial profile, or broad mapping would add large context, use an isolated worker only when the saved context exceeds handoff cost. Workers receive a compact capsule of settled choices, verified facts, scope, repository state, and success conditions.
---
-## Why not just install Ponytail + Superpowers together?
+## Benchmark contract in v1.3
-Practical Coding is influenced by both projects, but its differentiator is the control policy.
+The old v1.2 classifier:
-| Question | Ponytail + Superpowers | Practical Coding |
-|---|---|---|
-| Tiny obvious edit | Two broad philosophies remain available to the host/model | **Core only** |
-| Unknown bug | Multiple applicable process rules may coexist | **Debugging only** |
-| High-risk change | Rigor exists, but selection belongs to separate systems | **Implementation only when the risk boundary is unresolved** |
-| Code discovery | Depends on host/tool behavior | **Explicit cheapest-sufficient retrieval ladder** |
-| Context footprint | Independent systems may accumulate | **Core + at most one reasoning reference; broad retrieval isolated only when worth it** |
+```text
+REASONING = NONE | DECISION | DEBUGGING | IMPLEMENTATION
+RETRIEVAL = NONE | TARGETED | BOUNDED | STRUCTURAL
+```
-Practical Coding is therefore not `ponytail.md + superpowers.md`. It is an adaptive policy for deciding **how much engineering reasoning and how much repository context are worth paying for now**.
+is replaced by:
+
+```text
+DECISION = CLEAR | REQUIRED
+EXECUTION = BLOCKED | DIRECT | DEBUGGING | IMPLEMENTATION
+RETRIEVAL = minimum sufficient .. maximum reasonable
+```
+
+Invariant:
+
+```text
+DECISION=REQUIRED => EXECUTION=BLOCKED
+DECISION=CLEAR => EXECUTION in DIRECT | DEBUGGING | IMPLEMENTATION
+```
+
+Four explicit transition regressions are added:
+
+- Decision → Direct
+- Decision → Implementation
+- Debugging → Direct after diagnosis
+- Debugging → Implementation only when diagnosis exposes a still-unresolved material boundary
+
+Native behavior cases also verify that a settled Decision is not reopened and a diagnosed bug does not reload Debugging unnecessarily.
+
+The canonical benchmark runner is now v2.1. `run_benchmarks.py` remains the stable execution core; `case_catalog.py` adds the public case corpus and `adaptive_rigor.py` installs the v1.3 contract. This preserves the ability to interpret the committed v1.2 evidence without pretending the schemas are score-comparable.
---
-## Benchmark evidence
+## Evidence status
+
+No v1.3 model result is claimed before a fresh run.
-The final v1.2 evidence is published under [`benchmarks/results/v1.2/`](benchmarks/results/v1.2/): reasoning classification passed 114/114, independent Retrieval classification passed 106/114, Native Behavior passed 54/54, and the Practical-only Delivery/Decision/Debug regression passed 75/75. The v1.1 five-route results remain historical evidence and are not score-comparable with the v1.2 two-dimensional Router schema.
+The last committed validated baseline is v1.2 under [`benchmarks/results/v1.2/`](benchmarks/results/v1.2/):
-The published v1.1 results remain:
+- reasoning classification: **114/114**;
+- Retrieval exact classification: **106/114**;
+- Native Behavior: **54/54**;
+- Practical-only Delivery/Decision/Debug regression: **75/75**.
-| Suite | Practical v1.1 |
-|---|---:|
-| Delivery | **100% (27/27)** |
-| Decision | **100% (18/18)** |
-| Debug | **96.7% (29/30)** |
-| Router | **100% (114/114)** |
-| Native behavior | **100% (54/54)** |
-| Applicable total | **99.6% (242/243)** |
+Those results validate the v1.2 contract, not the v1.3 adaptive-rigor schema. The v1.3 candidate must rerun the affected Router/Behavior matrix and the current-vs-previous regression before release claims are updated.
-See the [v1.1 data](benchmarks/results/v1.1/README.md), [Chinese report](benchmarks/results/v1.1/REPORT_ZH.md), and [reproduction guide](benchmarks/REPRODUCING.md). A fresh v1.2 run is required before publishing new comparative claims.
+The GitHub Releases page currently has the tagged `v1.0.0` release; repository benchmark/Skill versions have advanced independently. The next tagged release should be created only after the v1.3 validation gate is complete.
+
+See [`benchmarks/REPRODUCING.md`](benchmarks/REPRODUCING.md) and [`benchmarks/NEXT_VALIDATION.md`](benchmarks/NEXT_VALIDATION.md).
+
+---
+
+## Why not just install Ponytail + Superpowers?
+
+Practical Coding is influenced by both, but its target is the **control policy** around specialist rigor.
+
+| Situation | Broad co-installed skills | Practical Coding |
+|---|---|---|
+| Tiny obvious edit | Multiple broad policies may remain applicable | **Core only** |
+| Unknown bug | Host/model chooses among overlapping process rules | **Debugging rigor only while cause is unknown** |
+| Risky change | Strong engineering rules exist but may be activated broadly | **Implementation rigor only while a material boundary is unresolved** |
+| Architecture choice | Can mix implementation reasoning with choice resolution | **Decision blocks execution only when the choice actually changes the next action** |
+| Repository discovery | Depends on host behavior | **Explicit cheapest-sufficient retrieval policy** |
+| Context growth | Independent references may accumulate | **Core + at most one large reasoning reference at a time** |
+
+This is still a hypothesis about integrated-stack efficiency until the planned combined-install benchmark is run. The repository does not claim universal superiority from specialist pairwise comparisons alone.
---
@@ -210,8 +232,6 @@ Project-local:
git clone https://github.com/Hubujiu/practical-coding.git .github/skills/practical-coding
```
----
-
## Repository structure
```text
@@ -221,12 +241,16 @@ practical-coding/
├── README.md
├── README_zh.md
├── references/
-│ ├── debugging.md
│ ├── decision.md
+│ ├── debugging.md
│ ├── implementation.md
│ ├── navigation.md
│ └── delegation.md
├── benchmarks/
+│ ├── run_benchmarks.py
+│ ├── case_catalog.py
+│ ├── adaptive_rigor.py
+│ └── run_catalog.py
├── examples/
├── agents/
└── docs/evaluations/
@@ -237,10 +261,10 @@ practical-coding/
- [DietrichGebert/ponytail](https://github.com/DietrichGebert/ponytail): YAGNI, native/stdlib-first thinking, deletion over addition.
- [obra/superpowers](https://github.com/obra/superpowers): systematic debugging, engineering rigor, verification, isolation.
- [mattpocock/skills](https://github.com/mattpocock/skills) / [Agent Skills Spec](https://agentskills.io): progressive disclosure and composable Skill structure.
-- [dmtrKovalenko/fff](https://github.com/dmtrKovalenko/fff): bounded/ranked code retrieval ideas such as frecency-aware candidate discovery.
+- [dmtrKovalenko/fff](https://github.com/dmtrKovalenko/fff): bounded/ranked code retrieval ideas.
- [DeusData/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp): structural code intelligence and graph-backed relationship queries.
-The differentiator is not ownership of those ideas. It is the policy that decides **when each capability is worth its implementation, retrieval, and context cost**.
+The differentiator is the policy that decides **how much engineering rigor, retrieval, and context are worth paying for now**.
## Contributing
diff --git a/README_zh.md b/README_zh.md
index 4b06743..5a96c2c 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -3,7 +3,7 @@
-
+
@@ -11,170 +11,196 @@
English · 简体中文
-> ## 每个编码任务，只支付它真正需要的工程强度和上下文成本。
+> ## 每个编码任务，只使用它真正需要的最小工程严格度。
>
-> **简单工作保持 Direct；未知 Bug 才进入根因调试；高风险修改才增加严谨度；代码检索在第一个足够的层级停止。**
+> **从最低成本开始；只有未决选择真的阻塞下一步时才先做 Decision；只有 Direct 不足时才增加 Debugging 或 Implementation 严格度；只加载值得支付的代码上下文。**
-Practical Coding 是一个轻量的编码 Agent Skill。它把两种成本分开控制：
+Practical Coding 是一个轻量的编码 Agent Skill。它不再把任务硬分成四类，而是一个 **自适应工程严格度系统**：
-1. **推理成本：** 只有真正未解决的 blocker 才允许加载 Debugging、Decision 或 Implementation。
-2. **上下文成本：** 代码检索从已知源码开始，按需升级到 bounded/ranked search，再到结构化索引；只有前一级不足时才继续。
+1. **Core**：所有编码任务都需要的最小规则。
+2. **Decision Gate**：只有实质未决选择会阻塞或改变下一安全动作时才先解决选择。
+3. **Execution Escalation**：默认 Direct；只有当前 blocker 真的需要时才增加 Debugging 或 Implementation 严格度。
+4. **Retrieval + Isolation**：只为真正有价值的仓库上下文和额外上下文支付成本。
```bash
npx skills@latest add Hubujiu/practical-coding
```
-## v1.2 的核心变化
+## v1.3 架构
-Navigation 不再是 Event Router 的第四条互斥分支，而变成 Direct 和所有 routed event 都可使用的 **Retrieval Policy**。
+```mermaid
+flowchart TB
+ T[用户编码任务] --> C[Core]
+ C --> D{实质未决选择是否阻塞下一步?}
+ D -->|是| Q[Decision Gate]
+ Q --> C2[选择已确定 / compact execution capsule]
+ D -->|否| C2
+
+ C2 --> E[从 Direct 开始]
+ E --> S{Core 现在是否足够?}
+ S -->|是| V[最便宜的充分验证]
+ S -->|否: 故障根因未知| G[Debugging rigor]
+ S -->|否: 安全边界未知| I[Implementation rigor]
+ G --> V
+ I --> V
+
+ C --> R[Retrieval Policy]
+ C2 --> R
+ G --> R
+ I --> R
+ R --> K[已知源码 / bounded search / structural retrieval]
-| 当前情况 | Practical Coding 行为 |
-|---|---|
-| 改名、CSS、已知局部修改 | **Direct Path**：只用 Core |
-| 已观察到 Bug，但根因未知 | Core + **Debugging** |
-| 架构/API/依赖等实质选择仍未确定 | Core + **Decision** |
-| 未知契约或尚未解决的安全、迁移、权限、持久化、并发、兼容性等重要边界阻塞安全执行 | Core + **Implementation** |
-| 只是需要找到相关代码 | 走最便宜的充分检索路径；“需要搜索”本身不会选择 reasoning module |
-| 需要大范围调用链/依赖关系映射 | 已有结构化索引能明显减少探索时才使用；没有就直接回退 bounded source search |
+ C2 --> X{后续是否还需要另一份大 reference?}
+ X -->|否| E
+ X -->|是且节省上下文超过 handoff 成本| W[隔离 worker]
+```
-新的核心不变量：
+核心区别是：
-> **Core + 最多一个 reasoning module；Retrieval 与 Event Router 正交。** 当 governing boundary、affected surface 和 sufficient check 已经确定时，仅仅出现安全、持久化、迁移、并发或兼容性名词并不会触发 Implementation。
+> **Decision 解决“下一步到底做什么”；Execution rigor 解决“已知下一步以后，需要多严格地执行”；Retrieval 解决“需要把哪些代码上下文加载进来”。**
-v1.1 遗留的 `.practical-coding.yaml` 不再被 Skill 读取，可以直接删除。Retrieval 能力改为根据当前宿主/环境中已经存在的工具动态选择，而不是保存为项目级偏好。
+### Core
----
+常驻 `SKILL.md` 只保留所有编码任务都适用的最小规则：
-## 架构
+- 先定义最小可观察成功条件；
+- 优先复用最近的既有 primitive、API 和 contract；
+- 不增加推测性的抽象、配置、wrapper、option 和脚手架；
+- 只做最小 coherent reachable change；
+- 测试、validation、fallback、注释、文档只在当前需求、既有 contract、项目规则或必要验证要求时添加；
+- 最终编辑后只跑一次最便宜、最聚焦的充分检查；
+- 只声明新证据真正支持的内容。
-```mermaid
-flowchart TB
- T[用户编码任务] --> C[Always-On Core]
- C --> E{是否存在当前未解决的推理 blocker?}
- E -->|否| D[Direct Path]
- E -->|故障仍无根因| G[Debugging]
- E -->|存在实质未决选择| A[Decision]
- E -->|未知契约 / 风险边界| I[Implementation]
-
- D --> R{是否还需要更多代码上下文?}
- G --> R
- A --> R
- I --> R
+### Decision Gate
- R -->|不需要| V[最便宜的聚焦验证]
- R -->|已知路径/符号| K[Targeted read]
- R -->|位置未知| S[Bounded / ranked source search]
- R -->|需要结构关系| X[已有 structural index]
+Decision 不再与 Debugging / Implementation 同级。
- K --> V
- S --> V
- X --> Q[重要结论回到当前源码验证]
- Q --> V
- V --> O[只基于新证据交付]
-```
+先问：
-### Always-On Core
+> **是否存在一个实质未决选择，会阻塞或显著改变下一安全动作？**
-常驻 `SKILL.md` 继续只保留所有编码任务都适用的最小规则：
+如果是，加载 [`decision.md`](references/decision.md)。先从仓库和权威来源解决可以发现的事实，只有真正属于用户的 scope、compatibility、cost、preference、risk tolerance 才需要提问。
-- 先定义最小可观察成功条件；
-- 实现上在第一个能工作的阶梯停止；
-- 复用已有 primitive、API 和 contract；
-- 不增加推测性的抽象、配置、wrapper、alias 或脚手架；
-- 只做最小 coherent reachable change；
-- 删除优先，普通代码优先；
-- validation、fallback、retry、测试、注释、文档只在真实需求、既有 contract、项目规则或必要验证要求时添加；
-- 最终只跑一次最便宜、最聚焦的检查；
-- 只声明最新证据真正支持的内容。
+如果请求、仓库、权威约束或便宜可逆的默认值已经把选择确定下来，就直接进入执行。
-### 三个 reasoning module
+### Execution Escalation
-| 模块 | 触发条件 | 目的 |
-|---|---|---|
-| [`debugging.md`](references/debugging.md) | 已观察故障仍缺少证据化根因 | 复现 → 最早错误状态 → 支持的根因 → 根因修复 |
-| [`decision.md`](references/decision.md) | 一个由用户决定的实质选择仍未解决，并会改变下一步 | 收敛最小真实 decision frontier |
-| [`implementation.md`](references/implementation.md) | 安全执行被未知 contract/invariant、重要风险边界或不足以支撑高风险结论的证据阻塞 | 映射边界、保留保证并确定充分证据 |
+Direct 是默认执行状态，而不是一个模块。
-Event Router 只在这三个模块之间选择。文件数量、任务名、需要检索代码、或者存在另一个 library，都不是 reasoning route 的触发条件。
+| 当前 blocker | 使用的严格度 |
+|---|---|
+| 下一安全动作已经明确 | **Direct：只用 Core** |
+| 已观察到故障，但根因仍没有证据 | Core + [`debugging.md`](references/debugging.md) |
+| 安全执行被未知 contract/invariant、未解决的重要风险边界或不足的高风险证据阻塞 | Core + [`implementation.md`](references/implementation.md) |
----
+Debugging 和 Implementation 是**并列的升级 profile**，不是 `Direct → Debugging → Implementation` 的强制流水线。
-## Retrieval：上下文筛选，而不是另一套 workflow
+已经诊断清楚的 Bug 可以直接修。只有一行的 persistence / permission 修改也可能需要 Implementation。反过来，多文件任务如果 contract、影响面和充分检查都已经明确，也可以 Direct。
-Retrieval 回答的是和 Event Router 不同的问题：
+### Retrieval Policy
-> **当前任务需要的代码上下文，怎样以最低成本获得？**
+Retrieval 与 Decision / execution rigor 独立：
-检索阶梯：
+1. 当前上下文 / 已知源码；
+2. bounded 或 ranked source discovery；
+3. 只有关系型探索确实值得时才使用 structural index；
+4. 重要结论回到当前源码验证。
-1. **当前上下文 / 已知目标** → 直接读取目标源码。
-2. **不知道位置** → 优先使用宿主已经提供的 bounded/ranked search。
-3. **没有 ranked primitive** → 回退普通 filename / text / symbol search，例如宿主搜索、`rg`、`grep`、`find`。
-4. **问题主要是结构关系** → 只有已有 structural index 能显著减少重复探索时才使用。
-5. **重要结论** → 回到当前源码验证，源码始终是权威来源。
+宿主原生搜索、FFF 式 ranked retrieval、普通 `rg` / filename / symbol search，以及 [`DeusData/codebase-memory-mcp`](https://github.com/DeusData/codebase-memory-mcp) 都只是能力，不是项目依赖。更强能力不存在时无损回退，不为了检索单独修改项目配置。
-在第一个足够的层级停止。
+只有大范围检索本身足够复杂时才加载 `references/navigation.md`。
-### FFF 式检索与 Codebase Memory 是互补关系
+### Retrieval 改为“成本区间”评分
-| 能力 | 最擅长 | 在 Practical Coding 中的角色 |
-|---|---|---|
-| 宿主原生 / FFF 式 ranked retrieval | 用有限输出和排序信号找到最可能相关的文件、文本候选 | 已经可用时作为低成本候选发现 |
-| 普通 `rg` / filename / symbol search | 精确文本、名称、小仓库、通用场景 | 零特殊后端的 fallback |
-| [`DeusData/codebase-memory-mcp`](https://github.com/DeusData/codebase-memory-mcp) 或其它 structural index | callers、callees、imports、implementations、依赖边、跨文件 flow | 已经可用且结构问题值得时使用 |
+v1.2 暴露了一个 benchmark 问题：如果要求唯一精确 Retrieval 标签，就可能把“只多搜了一小步、但仍然合理”的行为判错。
-Practical Coding **不要求** `@ff-labs/pi-fff`、FFF、Codebase Memory、`.practical-coding.yaml` 或任何常驻图谱服务，也不会仅仅因为“更强的后端可能方便”就自动安装检索工具。能力不存在就无损降级到下一层。
+v1.3 因此分成：
-`references/navigation.md` 保存更详细的大范围检索流程。普通 targeted lookup 不需要加载它。
+- **minimum sufficient retrieval**：至少需要多少检索；
+- **maximum reasonable retrieval cost**：最多允许升级到哪里。
----
+例如，已知概念但没有精确文件路径时，targeted read 和 bounded search 都可能合理；而没有必要的 structural exploration 仍然属于过度升级。
-## 上下文隔离
+### 上下文隔离
-“return to Direct” 这样的文字无法把已经读进模型上下文的 reference 真正移除，因此 v1.2 把隔离当成真实资源问题处理：
+“return to Direct” 只能改变逻辑状态，无法从模型上下文中真正移除已经读取的 reference。
-- Direct 和小型 routed event 不使用 worker；
-- Root 通常只携带 Core + 最多一个 reasoning reference；
-- 普通源码搜索直接使用宿主工具，不加载 Navigation；
-- 如果 Debugging / Decision / Implementation 已经驻留，而大范围 mapping 会产生明显上下文噪声，只有隔离收益大于 handoff 成本时才派只读 Navigation worker；
-- worker 返回 compact evidence capsule，而不是 raw grep、搜索日志或 graph dump。
+因此 Root 保持 **Core + 同一时刻最多一个大型 reasoning reference**。如果 Decision 已经驻留，后面又真的需要另一种大型严格度；或者 broad mapping 会产生大量上下文，只有隔离节省量明显超过 handoff 成本时才派 worker。
-这样 Progressive Disclosure 才真正是在节省上下文，而不只是把同一份大提示词拆成多个文件。
+Worker 接收 compact capsule：已确定选择、已验证事实、scope、repository state、success condition，而不是重放整段历史推理。
---
-## 为什么不直接同时安装 Ponytail + Superpowers？
+## v1.3 Benchmark 契约
-Practical Coding 的差异不在于“拥有更多规则”，而在控制策略。
+v1.2 的旧分类：
-| 问题 | Ponytail + Superpowers | Practical Coding |
-|---|---|---|
-| 很小且明确的修改 | 两套宽泛哲学仍交给宿主/模型协调 | **只用 Core** |
-| 未知 Bug | 多套流程规则可能同时相关 | **只加载 Debugging** |
-| 高风险改动 | 有严谨能力，但由不同系统各自触发 | **只有风险边界未解决才加载 Implementation** |
-| 代码检索 | 依赖宿主自己的工具行为 | **显式 cheapest-sufficient retrieval ladder** |
-| 上下文成本 | 独立系统可能累计 | **Core + 最多一个 reasoning reference；昂贵检索只在值得时隔离** |
+```text
+REASONING = NONE | DECISION | DEBUGGING | IMPLEMENTATION
+RETRIEVAL = NONE | TARGETED | BOUNDED | STRUCTURAL
+```
-所以 Practical Coding 不是 `ponytail.md + superpowers.md`，而是在决定：**此刻值得支付多少工程推理成本，以及多少代码库上下文成本。**
+改成：
+
+```text
+DECISION = CLEAR | REQUIRED
+EXECUTION = BLOCKED | DIRECT | DEBUGGING | IMPLEMENTATION
+RETRIEVAL = minimum sufficient .. maximum reasonable
+```
+
+约束：
+
+```text
+DECISION=REQUIRED => EXECUTION=BLOCKED
+DECISION=CLEAR => EXECUTION in DIRECT | DEBUGGING | IMPLEMENTATION
+```
+
+新增四类明确的状态转换回归：
+
+- Decision → Direct；
+- Decision → Implementation；
+- Debugging → Direct（根因已经确定后）；
+- Debugging → Implementation（只有诊断后仍存在未解决的重要执行边界时）。
+
+Native Behavior 还会检查：已经 settled 的 Decision 不会被重新打开，已经 diagnosed 的 Bug 不会无意义地重新加载 Debugging。
+
+canonical benchmark runner 升级为 v2.1：`run_benchmarks.py` 保留为稳定执行内核，`case_catalog.py` 提供公开 case corpus，`adaptive_rigor.py` 安装 v1.3 契约。这样既不会破坏 v1.2 历史证据，也不会假装两个 schema 可以直接比较分数。
---
-## Benchmark 证据
+## 当前证据状态
+
+**在重新跑 benchmark 之前，不声明任何 v1.3 模型结果。**
-最终 v1.2 证据已发布到 [`benchmarks/results/v1.2/`](benchmarks/results/v1.2/)：reasoning 分类 114/114、独立 Retrieval 分类 106/114、Native Behavior 54/54，Practical-only 的 Delivery/Decision/Debug 回归 75/75。v1.1 五路由结果继续作为历史证据保留，但不能与 v1.2 双维 Router schema 直接比较分数。
+最后一个已经提交并验证的 baseline 是 [`benchmarks/results/v1.2/`](benchmarks/results/v1.2/)：
-当前公开的 v1.1 结果仍为：
+- reasoning classification：**114/114**；
+- Retrieval exact classification：**106/114**；
+- Native Behavior：**54/54**；
+- Practical-only Delivery/Decision/Debug regression：**75/75**。
-| Suite | Practical v1.1 |
-|---|---:|
-| Delivery | **100%（27/27）** |
-| Decision | **100%（18/18）** |
-| Debug | **96.7%（29/30）** |
-| Router | **100%（114/114）** |
-| Native behavior | **100%（54/54）** |
-| 适用总计 | **99.6%（242/243）** |
+这些结果验证的是 v1.2，不是 v1.3 adaptive-rigor schema。v1.3 candidate 必须重新跑受影响的 Router/Behavior matrix，以及 current-vs-previous regression，才能更新 release claim。
-查看 [v1.1 数据](benchmarks/results/v1.1/README.md)、[中文报告](benchmarks/results/v1.1/REPORT_ZH.md) 和 [复现指南](benchmarks/REPRODUCING.md)。在发布新的对比结论之前，需要重新跑 v1.2。
+GitHub Releases 当前实际只有已打 tag 的 `v1.0.0`；仓库内部 Skill / benchmark 版本之后继续迭代。下一次正式 tag 应该等 v1.3 validation gate 完成后再创建。
+
+详见 [`benchmarks/REPRODUCING.md`](benchmarks/REPRODUCING.md) 和 [`benchmarks/NEXT_VALIDATION.md`](benchmarks/NEXT_VALIDATION.md)。
+
+---
+
+## 为什么不是简单安装 Ponytail + Superpowers？
+
+Practical Coding 借鉴两者，但目标是它们外层的**控制策略**。
+
+| 场景 | 多套宽泛 Skill 同时安装 | Practical Coding |
+|---|---|---|
+| 很小且明确的修改 | 多套策略都可能留给宿主/模型协调 | **只用 Core** |
+| 未知 Bug | 多种流程规则可能重叠 | **只在根因未知期间使用 Debugging rigor** |
+| 高风险修改 | 有严格规则，但可能被宽泛触发 | **只有重要边界未解决期间使用 Implementation rigor** |
+| 架构选择 | 选择和实现推理容易混在一起 | **只有选择真的阻塞下一动作时才让 Decision 阻塞执行** |
+| 代码检索 | 取决于宿主默认行为 | **显式 cheapest-sufficient retrieval policy** |
+| 上下文增长 | 多个 reference 可能累计 | **同一时刻 Core + 最多一个大型 reasoning reference** |
+
+关于 integrated-stack efficiency 的优势仍然只是待验证假设；在 combined-install benchmark 完成前，不把 pairwise specialist comparison 宣传成普适优越性。
---
@@ -210,8 +236,6 @@ git clone https://github.com/Hubujiu/practical-coding.git "$env:USERPROFILE\.age
git clone https://github.com/Hubujiu/practical-coding.git .github/skills/practical-coding
```
----
-
## 仓库结构
```text
@@ -221,12 +245,16 @@ practical-coding/
├── README.md
├── README_zh.md
├── references/
-│ ├── debugging.md
│ ├── decision.md
+│ ├── debugging.md
│ ├── implementation.md
│ ├── navigation.md
│ └── delegation.md
├── benchmarks/
+│ ├── run_benchmarks.py
+│ ├── case_catalog.py
+│ ├── adaptive_rigor.py
+│ └── run_catalog.py
├── examples/
├── agents/
└── docs/evaluations/
@@ -235,12 +263,12 @@ practical-coding/
## 灵感来源
- [DietrichGebert/ponytail](https://github.com/DietrichGebert/ponytail)：YAGNI、native/stdlib-first、删除优先。
-- [obra/superpowers](https://github.com/obra/superpowers)：系统化 debugging、工程严谨性、验证、任务隔离。
+- [obra/superpowers](https://github.com/obra/superpowers)：系统化 debugging、工程严谨性、验证和隔离。
- [mattpocock/skills](https://github.com/mattpocock/skills) / [Agent Skills Spec](https://agentskills.io)：Progressive Disclosure 和可组合 Skill 结构。
-- [dmtrKovalenko/fff](https://github.com/dmtrKovalenko/fff)：frecency 等面向 Agent 的 bounded/ranked code retrieval 思路。
+- [dmtrKovalenko/fff](https://github.com/dmtrKovalenko/fff)：bounded/ranked code retrieval 思路。
- [DeusData/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)：结构化代码智能与 graph-backed relationship query。
-真正的差异不是“谁发明了这些思想”，而是：**什么时候值得为哪一种能力支付实现、检索和上下文成本。**
+真正的差异是：**什么时候值得为哪一种工程严格度、检索能力和上下文成本付费。**
## 贡献
diff --git a/SKILL.md b/SKILL.md
index 6eb1a67..2809996 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -1,15 +1,15 @@
---
name: practical-coding
-description: "Use for implementing, fixing, refactoring, or reviewing code with the smallest correct change; routes only unresolved debugging, architecture/choice, or risk-boundary blockers, while code retrieval uses the cheapest sufficient available capability."
+description: "Use for implementing, fixing, refactoring, or reviewing code with adaptive engineering rigor: start from the smallest Core, resolve blocking choices before execution, escalate only for an unexplained failure or unresolved material execution boundary, and retrieve only the minimum sufficient code context."
license: MIT
metadata:
author: Hubujiu
- version: "1.2"
+ version: "1.3"
---
# Practical Coding
-One short, route-agnostic core for every coding task. Reasoning escalates only for unresolved blockers; code retrieval escalates independently only when cheaper context selection is insufficient.
+Use the smallest amount of engineering rigor and repository context that is sufficient for the current coding task. The Core always applies. Additional references are escalation profiles, not task categories or mandatory stages.
## Core
@@ -23,34 +23,47 @@ One short, route-agnostic core for every coding task. Reasoning escalates only f
- Run the cheapest focused check once after the final edit. Never repeat an unchanged check or substitute diff inspection for a required compile, type, or build gate. In an isolated workspace, routine installation of declared dependencies is allowed only as a bounded prerequisite; otherwise report missing prerequisites instead of changing the project solely for verification. If the check creates unrelated generated churn, inspect at most one diff, then leave and report it.
- State only what fresh evidence supports; keep unrequested explanation short.
-## Direct Path
+## Decision Gate
-The Core always applies. If no Event Router condition matches, proceed immediately with the Core alone. Targeted reads, filename/text/symbol searches, and use of an already-known project path are ordinary Direct work; they do not select a reasoning module.
+Before execution, ask one question:
-## Event Router
+> Does a material unresolved choice block or materially change the next safe action?
-The Router selects only whether one additional reasoning module must be loaded; it does not own retrieval strategy or add implementation rules. Route only on a present unresolved blocker. Settled facts and choices are inputs, not events. A security, persistence, migration, concurrency, or compatibility noun is not itself a blocker: when the governing boundary, affected surface, and sufficient check are already established, stay Direct. Use this first-match ladder:
+If no, continue immediately to Execution Escalation. If yes, read `references/decision.md` and resolve only that decision frontier.
-1. An observed failure, regression, or incorrect behavior still lacks an evidenced cause: read `references/debugging.md`. A symptom or named failing function is not a diagnosed cause.
-2. A material user-owned choice about architecture, whether or which external dependency or implementation to adopt, API, data model, or compatibility remains unresolved and would change the next action: read `references/decision.md`. A choice already specified and authorized by the user is settled input; research needed to resolve an open choice belongs inside Decision.
-3. Safe execution is blocked by an unknown contract or invariant, an unresolved material risk boundary such as security/permissions, irreversible effects, persistence/migration, concurrency/transactions, or compatibility, or insufficient evidence for a risky material claim: read `references/implementation.md`.
+A choice already settled by the request, repository, an authoritative constraint, or a cheap reversible default is input, not a Decision Gate. The existence of another library or implementation is not itself a blocker. Resolve discoverable facts before asking the user; ask only when a genuinely user-owned preference, compatibility policy, cost, scope, or risk tolerance still changes the next action.
-Read exactly that one reasoning module in addition to the Core. Resolve the blocker, then reassess only if a different blocker appears. Reassessment does not justify accumulating another reasoning reference in the root: handle a trivial later blocker with the Core, or isolate a substantial later event in a worker when the saved context exceeds handoff cost. Do not preload modules or load candidates together to compare them. Task nouns, file count, and the mere existence of an alternative library do not select a module.
+When the choice is settled, return to the Core and reassess execution. Settling a Decision does not itself justify loading another reasoning reference in the same root context.
+
+## Execution Escalation
+
+Start Direct: use the Core alone and stop as soon as it is sufficient. Direct is the default execution state, not a route or module.
+
+Escalate only for the blocker actually present:
+
+- **Debugging rigor:** when an observed failure, regression, incorrect behavior, or failed check exists and its cause is not yet evidenced, read `references/debugging.md`.
+- **Implementation rigor:** when safe execution is blocked by an unknown contract or invariant, an unresolved material risk boundary, or insufficient evidence for a risky material claim, read `references/implementation.md`. Material boundaries include security or permissions, irreversible effects, persistence or migration, concurrency or transactions, and compatibility.
+
+These are not sequential stages. Do not pass through Debugging before Implementation, or Implementation after Debugging, unless a new blocker actually appears. A diagnosed bug can be fixed Direct. A risky-looking change stays Direct when its governing boundary, affected surface, and sufficient check are already established. File count, task nouns, and apparent code size are not escalation evidence.
+
+If a loaded rigor profile resolves its blocker, continue with the Core. If a materially different blocker later requires another large reference, do not accumulate references in the root merely because the logical task continued; use the Isolation Gate when the saved context exceeds handoff cost.
## Retrieval Policy
-Retrieval is orthogonal to the Event Router. Use the cheapest available capability that can provide sufficient task-relevant code context, and stop at the first sufficient rung:
+Retrieval is independent from Decision and execution rigor. Use the cheapest available capability that can provide sufficient task-relevant code context, and stop at the first sufficient rung:
1. If current context, a known path, or a known symbol is sufficient, read only that source.
2. Otherwise use an already-available bounded or ranked source-search capability; fall back to ordinary filename, text, and symbol search.
3. When the unresolved question is primarily structural — callers, callees, imports, implementations, dependencies, or cross-file flow — prefer an already-available structural code index when it materially reduces exploration.
-4. If a stronger retrieval capability is unavailable, fall back without changing project configuration or installing/persisting tooling solely for retrieval.
+4. If a stronger retrieval capability is unavailable, fall back without changing project configuration or installing or persisting tooling solely for retrieval.
5. Verify material conclusions against current source; source is authoritative.
-Do not read `references/navigation.md` for routine lookups. Read it only when broad retrieval itself is substantial enough to require the detailed retrieval procedure. If a reasoning reference is already loaded, do not add Navigation to the same root context merely to search: use the short policy above, or isolate substantial broad mapping in a read-only worker when context savings exceed handoff cost.
+Retrieval levels are cost bounds, not semantic task classes. A slightly broader bounded search is acceptable when it remains cheap and avoids guessing; unnecessary structural exploration is not. Do not read `references/navigation.md` for routine lookups. Read it only when broad retrieval itself is substantial enough to require the detailed procedure. If another large reference is already resident, prefer the short policy above or isolate substantial broad mapping instead of adding Navigation to the same root context.
## Isolation Gate
-Direct work and a single routed event in a small context use no worker. The root never reads `references/delegation.md`. Keep the root to the Core plus at most one loaded reasoning reference for the task; do not build a sequence of references that only accumulates context.
+Direct work and one small escalated blocker use no worker. The root never reads `references/delegation.md`. Keep the root to the Core plus at most one loaded reasoning reference at a time; textual instructions such as "return to Direct" do not remove already-loaded context.
+
+When a later blocker or broad mapping task is substantial and isolation clearly saves more context than its handoff costs, dispatch one worker and tell that worker to read `references/delegation.md` plus exactly one assigned reference. Pass settled choices, verified facts, repository state, scope, and success conditions as a compact capsule rather than replaying prior reasoning.
-When isolation clearly saves more context than its handoff costs, dispatch one worker and tell that worker to read `references/delegation.md` plus exactly one assigned reference. Navigation workers are read-only. Decision and Debugging workers are read-only. An Implementation worker may write only when its assignment explicitly includes implementation, must have a bounded scope, and must be the sole writer there. Never use overlapping writers or worker pipelines.
+Navigation workers are read-only. Decision and Debugging workers are read-only. An Implementation worker may write only when its assignment explicitly includes implementation, must have a bounded scope, and must be the sole writer there. Never use overlapping writers or worker pipelines.
diff --git a/benchmarks/NEXT_VALIDATION.md b/benchmarks/NEXT_VALIDATION.md
index 7556c2a..cd25442 100644
--- a/benchmarks/NEXT_VALIDATION.md
+++ b/benchmarks/NEXT_VALIDATION.md
@@ -1,20 +1,26 @@
# Next validation protocol
-This document freezes the evidence requirements for the next Practical Coding validation cycle before new results are inspected. Its purpose is to prevent post-hoc metric selection, public-regression overfitting, and marketing claims stronger than the evidence.
+This document freezes the evidence requirements for the **Practical Coding v1.3 adaptive-rigor candidate** before new v1.3 model results are inspected. Its purpose is to prevent post-hoc metric selection, public-regression overfitting, and marketing claims stronger than the evidence.
-The current public release is **Practical Coding v1.1**. Its compact aggregates and evidence boundaries are in [`results/v1.1/`](results/v1.1/); the v1.0 release remains archived in [`results/v1.0/`](results/v1.0/).
+Version status is intentionally explicit:
+
+- latest tagged GitHub Release: `v1.0.0`;
+- last committed validated architecture baseline: **v1.2**, with compact evidence in [`results/v1.2/`](results/v1.2/);
+- current candidate architecture: **v1.3**, using Decision Gate + Execution Escalation + Retrieval cost bounds;
+- v1.0/v1.1/v1.2 result directories remain immutable historical evidence for their tested schemas.
## 1. Freeze before running
-Before any release-quality model run:
+Before any release-quality v1.3 model run:
1. commit the candidate and use a clean working tree;
-2. record the exact candidate commit and benchmark manifest hashes;
-3. do not change Skill text, tasks, scorers, or acceptance thresholds after seeing partial results from the same cycle;
-4. if an instrument bug is found, invalidate the affected run, fix the instrument, document why, and rerun the complete affected matrix;
-5. preserve the complete candidate Skill bundle and comparator pins.
+2. record the exact candidate commit, previous-baseline commit, runner bundle hash, Skill entrypoint hash, and complete Skill-bundle hash;
+3. freeze the v1.3 classification contract before inspecting model results: `DECISION`, `EXECUTION`, and Retrieval minimum/maximum bounds;
+4. do not change Skill text, tasks, scorers, retrieval bounds, or acceptance thresholds after seeing partial results from the same cycle;
+5. if an instrument bug is found, invalidate the affected run, fix the instrument, document why, and rerun the complete affected matrix;
+6. preserve the complete candidate Skill bundle and comparator pins.
-Documentation-only changes do not justify retuning against already-inspected public cells. New evidence should come from held-out tasks, stack/interference tests, or repeated independent failures.
+Documentation-only changes do not justify retuning against already-inspected public cells. New behavioral evidence should come from held-out tasks, stack/interference tests, transition tests, or repeated independent failures.
## 2. Required gate order
@@ -26,7 +32,41 @@ pwsh -NoProfile -File benchmarks/run.ps1 -SelfTest
A failing self-test blocks all benchmark claims.
-### Gate B — public regression/current-vs-previous gate
+### Gate B — adaptive-rigor and native-behavior regression
+
+Because v1.3 changes the control model itself, rerun the affected classifier and native behavior suites before interpreting delivery comparisons:
+
+```powershell
+pwsh -NoProfile -File benchmarks/run.ps1 `
+ -Profile full `
+ -Suite router `
+ -Suite behavior `
+ -Runs 3 `
+ -Workers 3 `
+ -RequireStableRanking
+```
+
+The legacy suite id remains `router` for CLI compatibility, but its v1.3 contract is not an Event Router. It scores:
+
+- `DECISION=CLEAR|REQUIRED`;
+- `EXECUTION=BLOCKED|DIRECT|DEBUGGING|IMPLEMENTATION`;
+- Retrieval as a **minimum sufficient to maximum reasonable** cost interval.
+
+Required state invariant:
+
+```text
+DECISION=REQUIRED => EXECUTION=BLOCKED
+DECISION=CLEAR => EXECUTION in DIRECT | DEBUGGING | IMPLEMENTATION
+```
+
+The matrix must include the explicit transition boundaries:
+
+- Decision → Direct;
+- Decision → Implementation;
+- Debugging → Direct after diagnosis;
+- Debugging → Implementation only when a material execution boundary remains unresolved.
+
+### Gate C — public regression/current-vs-v1.2 gate
Run whenever `SKILL.md` or `references/` behavior changes:
@@ -35,29 +75,32 @@ pwsh -NoProfile -File benchmarks/run.ps1 `
-Profile full `
-Runs 3 `
-Workers 3 `
- -BaselineRef `
+ -BaselineRef 88382d2b0c00fa278067a5933bbcacc86f46b56e `
-IncludeBaseline `
-RequireStableRanking
```
+This compares the candidate against the accepted v1.2 baseline while preserving the fact that v1.2 and v1.3 classification schemas are not score-comparable. Delivery/Decision/Debug behavior, build/reachability, safety, and efficiency can still be compared where the underlying task/scorer contract is unchanged.
+
Acceptance order remains correctness/safety → build/reachability → efficiency.
-### Gate C — private held-out evidence
+### Gate D — private held-out evidence
-The strongest generalization claim requires a task set that was not consulted while editing the Skill.
+The strongest generalization claim requires a task set that was not consulted while editing the Skill or its public regression corpus.
Minimum first held-out target:
- at least 20 real coding tasks;
-- include simple/direct tasks, root-cause bugs, risky multi-file changes, and architecture/navigation tasks;
+- include simple/direct tasks, diagnosed and undiagnosed bugs, risky changes, unresolved choices, multi-file low-risk changes, and relationship-heavy retrieval tasks;
- executable verification whenever possible;
-- seed state fails and oracle/reference state passes before model calls;
+- seed state fails and oracle/reference state passes before model calls where applicable;
- same fixed model/harness for `no-skill` and Practical arms;
-- at least three paired repetitions for publication-quality claims.
+- at least three paired repetitions for publication-quality claims;
+- report unnecessary escalation, missed escalation, and retrieval-cost overshoot in addition to task success.
## 3. Required combined-stack benchmark
-Before claiming that Practical Coding is experimentally better than installing Ponytail and Superpowers together, add this arm:
+Before claiming that Practical Coding is experimentally better than installing Ponytail and Superpowers together, include this arm:
```text
no-skill
@@ -67,7 +110,7 @@ Ponytail + Superpowers
Practical Coding
```
-The combined arm must install the **actual current Skills simultaneously** in the same harness rather than concatenate their text manually.
+The combined arm must install the **actual pinned/current Skills simultaneously** in the same harness rather than concatenate their text manually.
Measure at minimum:
@@ -75,31 +118,37 @@ Measure at minimum:
- total/uncached input tokens, output tokens, reasoning tokens;
- model time and tool calls;
- number of Skill/reference files loaded;
-- unnecessary process/module loads;
+- unnecessary rigor/profile loads;
- missed escalations;
-- number of planning/debugging/delegation phases entered;
-- final production/test LOC;
-- whether both broad coding/process policies are invoked on simple tasks.
+- Decision blocking when no choice actually blocks execution;
+- Debugging reload after the cause is already diagnosed;
+- Implementation escalation after the governing boundary is already established;
+- retrieval below the minimum sufficient bound or above the maximum reasonable bound;
+- worker/subagent dispatches;
+- final production/test LOC.
The task set must include at least:
1. trivial/local direct edits;
2. clear multi-file but low-risk changes;
3. unknown root-cause bugs;
-4. security/persistence/concurrency boundaries;
-5. unresolved architecture/dependency decisions.
+4. already-diagnosed bugs;
+5. security/persistence/concurrency boundaries, both mapped and unmapped;
+6. unresolved architecture/dependency decisions;
+7. settled decisions followed by Direct work;
+8. settled decisions followed by unresolved execution boundaries.
### Hypothesis being tested
The architectural hypothesis is not "Ponytail is bad" or "Superpowers is bad." It is:
-> Two independently broad Skills may provide useful capabilities but incur duplicated routing/process context and leave their interaction to the host/model, while Practical's single event router should preserve similar specialist rigor with less unnecessary process on tasks that do not need it.
+> Broad specialist Skills may provide useful capabilities but leave activation and interaction to the host/model. Practical's adaptive-rigor control policy should preserve specialist rigor when a real blocker exists while avoiding unnecessary process, reference loading, and repository context when the Core is already sufficient.
This remains a hypothesis until the combined arm is measured.
-## 4. Routing and interference ablation
+## 4. Rigor and interference ablation
-To attribute any gain to adaptive routing rather than prompt wording, test:
+To attribute any gain to adaptive activation rather than prompt wording, test:
```text
no-skill
@@ -113,35 +162,51 @@ Ponytail + Superpowers
Record:
-- unnecessary module loads;
+- unnecessary reference loads;
- missed escalations;
- references loaded and bytes/tokens injected;
-- route changes per task;
-- time/tokens before the correct route is reached;
-- worker/subagent dispatches.
+- Decision Gate false positives/negatives;
+- Execution state errors;
+- retrieval minimum/maximum violations;
+- time/tokens before the sufficient rigor level is reached;
+- worker/subagent dispatches;
+- whether a second large reference is accumulated in the root instead of isolated when isolation would be cheaper.
+
+## 5. Retrieval scoring discipline
+
+Do not retune Skill wording to force one exact retrieval label when multiple low-cost paths are reasonable.
+
+The v1.3 classifier scores two retrieval failures separately:
+
+- **insufficient retrieval:** actual level is below the declared minimum needed to act safely;
+- **excessive retrieval:** actual level exceeds the declared maximum reasonable cost.
+
+A task may therefore accept a range such as `TARGETED..BOUNDED`. Change a range only before inspecting a new validation cycle, or after an independently justified benchmark-instrument defect is documented. Do not widen a range merely to turn an observed failure green.
-## 5. Statistical language
+## 6. Statistical language
`n=3` is a stability gate, not proof that small differences are statistically resolved. For small deltas use language such as `numerically ahead`, `numerically behind`, or `tied on this matrix` and report task counts separately from repeated trials.
Future confidence intervals should bootstrap by task/case ID so repetitions of one task are not treated as independent tasks.
-## 6. Failure discipline
+## 7. Failure discipline
When a failure appears:
1. save the complete run first;
2. classify infrastructure vs scorer/oracle defect vs stochastic behavior vs genuine Skill behavior;
-3. do not add case-specific nouns merely to turn a public cell green;
-4. prefer a general invariant only after the same mechanism appears independently;
-5. rerun the complete affected gate after a behavior change.
+3. distinguish control failure (Decision/Execution), retrieval insufficiency, retrieval overshoot, and delivered-code failure;
+4. do not add case-specific nouns merely to turn a public cell green;
+5. prefer a general invariant only after the same mechanism appears independently;
+6. rerun the complete affected gate after a behavior change.
-## 7. Claim ladder
+## 8. Claim ladder
| Evidence completed | Allowed claim |
|---|---|
-| Public regression only | Stable / numerically competitive on the fixed public matrix |
+| v1.3 public regression only | Stable on the fixed adaptive-rigor regression matrix |
+| + current-vs-v1.2 behavioral comparison | Bounded before/after claims on unchanged Delivery/Decision/Debug task contracts |
| + combined Ponytail/Superpowers arm | Bounded claims about integrated-stack efficiency/quality on that task population |
| + private held-out paired run | Bounded generalization claims for the held-out population |
-Never collapse Delivery vs Ponytail, Decision vs grilling, Debug vs Superpowers, and the combined-stack comparison into a single universal score.
+Never collapse Delivery vs Ponytail, Decision vs grilling, Debug vs Superpowers, adaptive-rigor classification, and the combined-stack comparison into a single universal score.
diff --git a/benchmarks/README.md b/benchmarks/README.md
index c6f85ed..05548a4 100644
--- a/benchmarks/README.md
+++ b/benchmarks/README.md
@@ -1,24 +1,67 @@
# Practical Coding benchmark chain
-This directory contains the reproducible evaluation harness for **Practical Coding v1.2**. The v1.2 runner uses a two-dimensional routing contract; committed v1.1 results remain historical evidence for the former five-way classifier.
+This directory contains the reproducible evaluation harness for the **Practical Coding v1.3 adaptive-rigor candidate**. The last committed validated baseline is v1.2 under [`results/v1.2/`](results/v1.2/); v1.0/v1.1/v1.2 result directories remain historical evidence for their tested contracts.
-The benchmark design intentionally avoids a single manufactured leaderboard. Each capability is compared with the most relevant specialist behavior, while Practical-owned routing suites test the integration layer that specialists do not provide by themselves.
+The benchmark design intentionally avoids a single manufactured leaderboard. Each specialist capability is compared with the most relevant behavior, while the Practical-owned control suite tests the integration policy itself.
-For exact commands, pinned upstream commits, evidence boundaries, and reproduction requirements, see [`REPRODUCING.md`](REPRODUCING.md). Current compact evidence lives in [`results/v1.2/`](results/v1.2/); [`results/v1.1/`](results/v1.1/) and [`results/v1.0/`](results/v1.0/) are retained as historical evidence.
+For exact commands, pinned upstream commits, evidence boundaries, and release gates, see [`REPRODUCING.md`](REPRODUCING.md) and [`NEXT_VALIDATION.md`](NEXT_VALIDATION.md).
## What is measured
| Suite | Compared arms | What it measures |
|---|---|---|
| Delivery | Practical vs Ponytail | Correctness, safety, build reachability, LOC, tokens, time, tool calls |
-| Decision | Practical vs Matt Pocock `grilling` | Whether a material unresolved choice is surfaced and converged without premature implementation |
+| Decision | Practical vs Matt Pocock `grilling` | Whether a blocking material choice is surfaced and converged without premature implementation |
| Debug | Practical vs Superpowers | Root-cause repair, sibling callers, delivered invariant, safety, efficiency |
-| Router | Practical vs expected two-dimensional contract | Reasoning (`NONE` plus Decision / Debugging / Implementation) and independent Retrieval (`NONE` / Targeted / Bounded / Structural) |
-| Native behavior | Practical only | Real Skill discovery, reasoning-reference isolation, and independent Retrieval/reference/backend behavior without prompt injection |
-| Navigation ablation | Source search vs optional graph backend | Whether AST/LSP graph navigation pays for itself on real repositories |
+| `router` *(legacy suite id)* | Practical vs adaptive-rigor contract | Decision Gate state, current execution rigor, and Retrieval minimum/maximum cost bounds |
+| Native behavior | Practical only | Real Skill discovery, selective reference loading, transition behavior, Retrieval/backend use, and context isolation without prompt injection |
+| Navigation ablation | Source search vs optional graph backend | Whether structural retrieval pays for itself on real repositories |
The Decision and Debug comparisons are controlled project comparisons; they are not official upstream benchmark claims.
+## v1.3 control contract
+
+The old v1.2 exact classifier is no longer the architecture being tested. The v1.3 control state is:
+
+```text
+DECISION = CLEAR | REQUIRED
+EXECUTION = BLOCKED | DIRECT | DEBUGGING | IMPLEMENTATION
+RETRIEVAL = minimum sufficient .. maximum reasonable
+```
+
+Required invariant:
+
+```text
+DECISION=REQUIRED => EXECUTION=BLOCKED
+DECISION=CLEAR => EXECUTION in DIRECT | DEBUGGING | IMPLEMENTATION
+```
+
+Meaning:
+
+- `DECISION=REQUIRED`: a material unresolved choice blocks or materially changes the next safe action;
+- `DIRECT`: the Core is sufficient now;
+- `DEBUGGING`: an observed failure exists and its cause is not evidenced;
+- `IMPLEMENTATION`: safe execution is blocked by an unknown contract/invariant, unresolved material risk boundary, or insufficient evidence for a risky claim;
+- Retrieval is not required to have one unique exact label when two neighboring cheap strategies are both reasonable.
+
+Four explicit transition regressions prevent the profiles from becoming task categories or a mandatory pipeline:
+
+- Decision → Direct;
+- Decision → Implementation;
+- Debugging → Direct after diagnosis;
+- Debugging → Implementation only when an unresolved material execution boundary remains.
+
+## Runner architecture
+
+The benchmark runtime is intentionally layered:
+
+- `run_benchmarks.py`: stable v2.0 execution/scoring core retained for historical interpretability;
+- `case_catalog.py`: extended public regression corpus;
+- `adaptive_rigor.py`: v1.3 state contract, retrieval-bound scoring, and transition cases;
+- `run_catalog.py`: canonical v2.1 entrypoint that installs both adapters before execution.
+
+Use `benchmarks/run.ps1`, which invokes the canonical runner. Do not run the v2.0 core directly when evaluating v1.3.
+
## Run the harness
Self-test without model calls:
@@ -27,7 +70,7 @@ Self-test without model calls:
pwsh -NoProfile -File benchmarks/run.ps1 -SelfTest
```
-Normal release-quality matrix:
+Normal candidate matrix:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 -Profile standard -Runs 3 -Workers 3 -RequireStableRanking
@@ -39,35 +82,41 @@ Complete public regression matrix:
pwsh -NoProfile -File benchmarks/run.ps1 -Profile full -Runs 3 -Workers 3 -RequireStableRanking
```
-Focused examples:
+Focused adaptive-rigor examples:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 -Profile standard -Suite router -Runs 3 -RequireStableRanking
-pwsh -NoProfile -File benchmarks/run.ps1 -Profile standard -Suite debug -Runs 3 -RequireStableRanking
-pwsh -NoProfile -File benchmarks/run.ps1 -Profile smoke -Suite router -Case direct-artifact -Arm practical-current
+pwsh -NoProfile -File benchmarks/run.ps1 -Profile standard -Suite behavior -Runs 3 -RequireStableRanking
+pwsh -NoProfile -File benchmarks/run.ps1 -Profile smoke -Suite router -Case transition-debug-to-direct -Arm practical-current
```
-Candidate before/after gate:
+Candidate against accepted v1.2:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 `
-Profile full `
-Runs 3 `
-Workers 3 `
- -BaselineRef `
+ -BaselineRef 88382d2b0c00fa278067a5933bbcacc86f46b56e `
-IncludeBaseline `
-RequireStableRanking
```
+The v1.2 and v1.3 classification schemas are not score-comparable. Before/after claims should be made only on unchanged task/scorer surfaces such as Delivery, Decision, and Debug, plus qualitative transition/reference-loading evidence.
+
## Profiles
-| Profile | Delivery | Router | Decision | Debug | Native behavior | Default runs |
+After installing `case_catalog.py` and `adaptive_rigor.py` through the canonical runner:
+
+| Profile | Delivery | Adaptive-rigor (`router`) | Decision | Debug | Native behavior | Default runs |
|---|---:|---:|---:|---:|---:|---:|
-| `smoke` | 3 | 4 | 1 | 1 | 3 | 1 |
-| `standard` | 9 | 38 | 6 | 10 | 18 | 3 |
-| `full` | 18 | 38 | 10 | 14 | 18 | 3 |
+| `smoke` | 3 | 5 | 1 | 1 | 3 | 1 |
+| `standard` | 9 | 42 | 6 | 10 | 22 | 3 |
+| `full` | 18 | 42 | 10 | 14 | 22 | 3 |
-`standard` is the normal public release gate. `full` carries the broader complete public regression matrix. Router cases cover all four reasoning outputs (`NONE` plus three reasoning routes), all four Retrieval modes, and cross-products such as `NONE+STRUCTURAL` and `IMPLEMENTATION+STRUCTURAL`. Native behavior repeats Direct/Implementation boundaries without injected Skill text, adds Decision/Debug precedence cases, and scores Navigation/backend use independently from reasoning-reference selection. The expanded Debug set covers fourteen cases across parsing, normalization, tenant isolation, pagination, units, row handling, state invariants, TTL semantics, URL handling, and the upstream transfer/amount tasks. Decision grows from six to ten two-turn decisions in `full`. A stable published ranking requires at least three determinate repetitions per selected case/arm.
+`standard` is the normal bounded candidate gate. `full` carries the complete public regression matrix. A stable published comparison requires at least three determinate repetitions per selected case/arm.
+
+The public task catalog is a **regression corpus**, not a private generalization set. Once a case has influenced Skill wording or scoring design, its future score is evidence against regression, not independent evidence that the same behavior generalizes everywhere.
## Acceptance order
@@ -75,26 +124,47 @@ Interpret results in this order:
1. correctness and safety;
2. build/reachability;
-3. only then LOC, tokens, model time, and tool calls.
+3. control correctness and missed escalation;
+4. Retrieval sufficiency before Retrieval efficiency;
+5. only then LOC, tokens, model time, and tool calls.
+
+A cheap failure cannot beat a correct safe result. Cost cannot rescue insufficient retrieval or a wrong execution state.
+
+## Why adaptive-rigor classification matters
-A cheap failure cannot beat a correct safe result. The scorecard first applies a conservative quality gate and only computes relative efficiency after that gate is satisfied. See [`../docs/evaluations/2026-08-26-quality-gated-scorecard.md`](../docs/evaluations/2026-08-26-quality-gated-scorecard.md).
+Practical Coding's architectural claim cannot be established only by Delivery vs Ponytail or Debug vs Superpowers. The control suite measures whether the system pays for stricter process only when a blocker exists:
-The public task catalog is a **regression corpus**, not a private generalization set. Once a case has influenced Skill wording, its future score is evidence against regression, not independent evidence that the same behavior generalizes everywhere.
+- settled or cheap reversible choices do not block execution;
+- an unresolved material choice blocks execution rather than competing with Debugging/Implementation as a peer route;
+- ordinary execution starts Direct;
+- an unknown-cause failure adds Debugging rigor;
+- an unresolved material execution boundary adds Implementation rigor;
+- diagnosed failures return to Direct unless a different blocker remains;
+- Retrieval is independent and bounded by sufficiency and reasonable cost;
+- broad structural exploration does not become a reasoning state;
+- only the required reasoning reference is read in a root context;
+- workers are an economic isolation mechanism, not a mandatory stage.
-## Why the router matters
+The next validation cycle also requires a **Ponytail + Superpowers combined-install arm** before any claim of experimental superiority over installing both together. See [`NEXT_VALIDATION.md`](NEXT_VALIDATION.md).
-Practical Coding's main architectural claim cannot be established by comparing only against Ponytail or only against Superpowers. The project also measures whether the integration policy itself behaves as intended:
+## Retrieval scoring
-- ordinary work selects no reasoning route;
-- an unresolved bug loads Debugging, not every engineering module;
-- a material choice loads Decision;
-- risky boundaries load Implementation;
-- Retrieval is classified independently as none, targeted, bounded, or structural;
-- broad structural exploration may load Navigation without becoming a fourth reasoning route;
-- only the selected reasoning reference is read;
-- workers are not a mandatory stage.
+The v1.3 classifier stores each case with:
-The next validation cycle also includes a **Ponytail + Superpowers combined-install arm**. That test is required before claiming that Practical is experimentally superior to installing both together. See [`NEXT_VALIDATION.md`](NEXT_VALIDATION.md).
+```text
+retrieval_min
+retrieval_max
+```
+
+A result is:
+
+- **insufficient** when actual retrieval is below `retrieval_min`;
+- **efficient enough** when actual retrieval is at or below `retrieval_max`;
+- **passed** on Retrieval only when both conditions hold.
+
+This avoids treating `TARGETED` vs a still-cheap `BOUNDED` search as necessarily semantically wrong while continuing to reject missing context and unnecessary structural exploration.
+
+The bounds are part of the benchmark contract and must be frozen before a validation cycle. Do not widen them after seeing failures simply to improve the score.
## Output artifacts
@@ -112,28 +182,29 @@ report.md
cells/
```
-The committed release directory contains only compact aggregates suitable for public review. Raw transcripts/workspaces stay local because they are large and may contain machine-specific paths.
+The committed result directories contain compact aggregates suitable for public review. Raw transcripts/workspaces stay local because they are large and may contain machine-specific paths.
## Suites and scoring
-- `delivery`: Ponytail's published agentic tasks and deterministic scorer. For frontend template cases, the runner installs the pinned lockfile dependencies before the agent starts, so the agent and the runner-owned production build use the same executable type/build environment. Reports correctness, safety, production LOC, test LOC, files, tokens, time, tool calls, setup time, and optional frontend build result.
-- `router`: exact two-field classification. `REASONING` is `NONE`, `DECISION`, `DEBUGGING`, or `IMPLEMENTATION`; only the latter three are reasoning routes. `RETRIEVAL` is independently `NONE`, `TARGETED`, `BOUNDED`, or `STRUCTURAL`. A cell passes only when both fields match. The former Verification route remains folded into Implementation, while former Exploration cases now expect `REASONING=NONE; RETRIEVAL=STRUCTURAL`; results are therefore not comparable with the v1.1 five-way classifier.
-- `decision`: Practical versus Matt Pocock `grilling`. Uses a real resumed second turn and gates on frontier questions, one recommendation per question, no premature implementation, and convergence after scripted user decisions. Trade-off language is reported diagnostically but is not a declared grilling contract gate.
-- `debug`: shared-root-cause tasks scored on the repaired invariant and sibling callers. Tests/TDD process receives no bonus. Each Practical-owned Debug seed is required to fail its deterministic scorer, and a separate oracle fixture must pass it before the case is accepted into the catalog.
-- `behavior`: installs Practical Coding into an isolated native `CODEX_HOME`, does not inject its text into the prompt, and mechanically inspects command traces for `SKILL.md` discovery. Reasoning references are scored separately from Retrieval: Direct reads no reasoning reference; Decision, Debugging, and Implementation read only their expected reasoning reference; structural Retrieval may read `navigation.md` and is separately checked against the expected source/graph backend. Infrastructure, timeout, transcript-capture, missing-runtime, and build OOM failures are reported as `indeterminate`, not Skill failures. Comparisons omit pairs containing indeterminate cells instead of rewarding the unaffected arm.
-
-`total_tokens` includes cached input because that is how Codex reports turn input. The report therefore also separates cached input, uncached input, output, and reasoning tokens. `duration_seconds` is per-cell process duration; suite elapsed time is recorded separately and is not obtained by summing concurrent cell durations.
+- `delivery`: Ponytail's published agentic tasks and deterministic scorer. Frontend cases prepare pinned dependencies before the agent starts and use the same environment for runner-owned production build evidence.
+- `router`: retained as a CLI suite name for compatibility. Under canonical runner v2.1 it scores the three-field adaptive-rigor state and Retrieval interval, not the v1.2 `REASONING/RETRIEVAL` classifier.
+- `decision`: Practical versus Matt Pocock `grilling`. Uses a real resumed second turn and gates on frontier questions, recommendations, no premature implementation, and convergence after scripted user constraints.
+- `debug`: shared-root-cause tasks scored on the repaired invariant and sibling callers. Named TDD/process rituals receive no bonus; delivered behavior and safety are what count.
+- `behavior`: installs Practical Coding into an isolated native `CODEX_HOME`, does not inject Skill text into the prompt, and inspects tool traces for Skill discovery and reference isolation. Transition cases ensure settled decisions and diagnosed failures do not reload obsolete profiles. Structural Retrieval/backend use is scored independently.
-## Acceptance
+Infrastructure, timeout, transcript-capture, missing-runtime, and build-OOM failures are `indeterminate`, not Skill failures. Comparisons omit pairs containing indeterminate cells instead of rewarding the unaffected arm.
-Use repeated paired results. A candidate is not accepted merely because its prose matches a Skill contract. Require no correctness/build regression, then compare delivered code and behavior. Treat LOC, tokens, and time as secondary within equally correct artifacts. `n=1` is a smoke result, not a stable ranking.
+`total_tokens` includes cached input because that is how Codex reports usage. Reports also separate cached input, uncached input, output, and reasoning tokens. `duration_seconds` is per-cell process duration; suite elapsed time is recorded separately.
-The machine-readable scorecard makes that order explicit. A comparison first has to stay within a 3 percentage-point suite pass-rate non-inferiority margin, with no lower suite build/safety rate and no case-level safety regression. Cost cannot rescue a failed quality gate. For a quality-qualified comparison, relative efficiency is the weighted geometric mean
+## Evidence status
-`E = exp(sum(w_i * ln(cost_comparator_i / cost_practical_i)))`
+No v1.3 model result is committed yet. The last validated evidence is v1.2:
-over uncached input tokens (0.35), output tokens (0.15), model time (0.35), and tool calls (0.15), renormalized when a metric is unavailable. The diagnostic utility is `U = ((Q_practical + 0.01) / (Q_comparator + 0.01))^2 * E`. It is a relative sensitivity summary, not an absolute leaderboard score. The report also preserves the Pareto result, so users can see when one arm dominates and when the result is a real quality/cost trade-off. `qualified` additionally requires at least three determinate repetitions in every paired case; otherwise a quality-passing score remains `provisional`.
+- reasoning classification: 114/114;
+- exact Retrieval classification: 106/114;
+- Native Behavior: 54/54;
+- Practical-only Delivery/Decision/Debug regression: 75/75.
-A published internal stable ranking must pass `benchmarks/check_stability.py` with the default minimum `n=3`. The gate checks distinct repetition IDs, complete-run metadata, and infrastructure errors. Behavioral or build failures remain valid benchmark observations and therefore do not invalidate the sample by themselves.
+Those values are historical baseline evidence, not v1.3 scores.
-The public catalog is a **regression suite**, not a hidden generalization test. Once a case has influenced Skill wording, its future 100% score should be treated as a ceiling check. A private held-out set is still required for the strongest generalization claims.
+Use repeated paired results. `n=1` is a smoke result, not a stable ranking. The strongest generalization claims still require the private held-out protocol in [`NEXT_VALIDATION.md`](NEXT_VALIDATION.md).
diff --git a/benchmarks/REPRODUCING.md b/benchmarks/REPRODUCING.md
index 90ce4e8..7c03a14 100644
--- a/benchmarks/REPRODUCING.md
+++ b/benchmarks/REPRODUCING.md
@@ -1,21 +1,46 @@
-# Reproducing the Practical Coding v1.2 Luna benchmarks
+# Reproducing the Practical Coding v1.3 adaptive-rigor benchmarks
-This document reproduces the Practical Coding v1.2 benchmark chain. Runner v2.0 invokes `gpt-5.6-luna` directly through `codex exec`, uses isolated workspaces, and records deterministic evidence where possible.
+This document reproduces the **v1.3 candidate** benchmark chain. The canonical runner is v2.1: `run_benchmarks.py` remains the stable v2.0 execution core, while `case_catalog.py` installs the public corpus and `adaptive_rigor.py` installs the v1.3 control contract before execution.
-Current compact v1.2 evidence is in [`results/v1.2/`](results/v1.2/). The committed [`results/v1.1/`](results/v1.1/) and [`results/v1.0/`](results/v1.0/) directories are historical evidence; their five-way Router scores do not validate the v1.2 two-dimensional contract.
+The last committed validated evidence is v1.2 under [`results/v1.2/`](results/v1.2/). [`results/v1.1/`](results/v1.1/) and [`results/v1.0/`](results/v1.0/) are also retained as historical evidence. Do not interpret their Router scores as v1.3 adaptive-rigor scores.
## Evidence types
| Suite | Tasks / grader | Compared arms | Evidence status |
|---|---|---|---|
| Delivery | Ponytail's published agentic tasks and deterministic scorer through a Codex adapter | Practical, Ponytail | Upstream task content with project execution adapter |
-| Router | Practical-owned exact two-dimensional reasoning + Retrieval corpus | Practical | Public project regression corpus |
+| Adaptive rigor (`router` suite id) | Practical-owned Decision / Execution / Retrieval-bound corpus | Practical | Public project regression corpus |
| Decision | Practical-owned two-turn scenarios and mechanical contract grader | Practical, Matt Pocock `grilling` | Controlled comparative benchmark |
| Debug | Upstream and Practical-owned shared-boundary cases with deterministic invariant grading | Practical, Superpowers | Controlled comparative benchmark |
-| Native behavior | Native Skill installation/discovery traces | Practical | Integration regression evidence |
+| Native behavior | Native Skill installation/discovery traces and transition cases | Practical | Integration regression evidence |
| Navigation | Real-repository paired source-vs-graph ablation | Source, optional Codebase Memory graph | Repository-specific evidence |
-Decision/Debug results must not be described as official upstream benchmark results. The benchmark scores delivered behavior and safety; it does not award points merely for following a particular planning or TDD process.
+Decision/Debug results must not be described as official upstream benchmark results. The benchmark scores delivered behavior and safety; it does not award points merely for following a named planning or TDD process.
+
+## v1.3 state contract
+
+The canonical classifier returns:
+
+```text
+DECISION=;
+EXECUTION=;
+RETRIEVAL=
+```
+
+with this state invariant:
+
+```text
+DECISION=REQUIRED => EXECUTION=BLOCKED
+DECISION=CLEAR => EXECUTION in DIRECT | DEBUGGING | IMPLEMENTATION
+```
+
+Retrieval is graded against a case-specific interval rather than one unique exact label:
+
+```text
+retrieval_min <= actual_retrieval <= retrieval_max
+```
+
+The order is `NONE < TARGETED < BOUNDED < STRUCTURAL`. Falling below the minimum is insufficient context; exceeding the maximum is unnecessary retrieval cost.
## Pinned upstream sources
@@ -34,7 +59,7 @@ Decision/Debug results must not be described as official upstream benchmark resu
- `bun` for the frontend production-build gate. Use `-NoBuilds` only for runner diagnosis; do not treat such a run as full delivery evidence.
- Enough model allowance for the selected matrix. `standard` and `full` default to three executions per cell.
-For each frontend template cell, the runner executes `bun install --frozen-lockfile` before the agent starts and keeps that `node_modules` tree for the runner-owned production build. This makes compile/type/build checks available to the agent without changing dependency declarations; setup duration is recorded separately from model duration.
+For each frontend template cell, the runner executes `bun install --frozen-lockfile` before the agent starts and keeps that `node_modules` tree for the runner-owned production build. Setup duration is recorded separately from model duration.
Confirm the local tools:
@@ -46,17 +71,22 @@ codex --version
bun --version
```
-## 1. Clone and self-test
+## 1. Clone, select the candidate, and self-test
+
+For the v1.3 candidate branch:
```powershell
git clone https://github.com/Hubujiu/practical-coding.git
Set-Location practical-coding
+git switch adaptive-rigor-v1.3
pwsh -NoProfile -File benchmarks/run.ps1 -SelfTest
```
-The self-test makes no model calls. It validates the harness, profiles, source pins, seed failures, oracle passes, and scorer mechanics.
+The self-test makes no model calls. It validates the harness, profiles, source pins, seed failures, oracle passes, adaptive-rigor schema, transition catalog, and scorer mechanics.
+
+`benchmarks/run.ps1` invokes `run_catalog.py`; do not invoke `run_benchmarks.py` directly for v1.3 evaluation because the latter intentionally remains the v2.0 core contract.
-## 2. Run the v1.2 capability suites
+## 2. Run the specialist capability suites
Delivery + Decision + Debug:
@@ -71,7 +101,11 @@ pwsh -NoProfile -File benchmarks/run.ps1 `
-RequireStableRanking
```
-Reasoning + Retrieval regression matrix (114 cells):
+These task/scorer surfaces can be compared with v1.2 where the underlying case contract is unchanged.
+
+## 3. Run the adaptive-rigor matrix
+
+The standard/full canonical profile currently contains **42 adaptive-rigor cases**. At three repetitions and one current arm this is **126 cells**:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 `
@@ -81,111 +115,129 @@ pwsh -NoProfile -File benchmarks/run.ps1 `
-RequireStableRanking
```
-Native Skill discovery, reasoning isolation, and Retrieval behavior (54 cells):
+The suite id remains `router` for CLI compatibility only. It now scores:
+
+- Decision Gate correctness;
+- current execution rigor;
+- state validity;
+- Retrieval sufficiency;
+- Retrieval efficiency relative to the declared maximum.
+
+The four explicit transition cases are:
+
+```text
+transition-decision-to-direct
+transition-decision-to-implementation
+transition-debug-to-direct
+transition-debug-to-implementation
+```
+
+A focused transition run:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 `
- -Profile standard `
- -Suite behavior `
+ -Profile full `
+ -Suite router `
+ -Case transition-decision-to-direct `
+ -Case transition-decision-to-implementation `
+ -Case transition-debug-to-direct `
+ -Case transition-debug-to-implementation `
-Runs 3 `
-RequireStableRanking
```
-Focused explicit-security comparison:
+## 4. Run Native Behavior
+
+The canonical standard/full profile contains **22 native-behavior cases**, or **66 cells** at three repetitions:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 `
-Profile standard `
- -Suite debug `
- -Case security-path-containment `
- -Case security-tenant-authorization `
+ -Suite behavior `
-Runs 3 `
-RequireStableRanking
```
-Published v1.1 aggregates are split by affected surface in [`results/v1.1/`](results/v1.1/). Delivery/Decision and Router/Behavior used runner v1.8; the final Debug rerun used v1.9. Current runner v2.0 changes the Router result schema and must publish v1.2 evidence separately rather than rescoring v1.1 transcripts as comparable results.
+Native Behavior validates real Skill discovery without prompt injection, selective reference loading, structural Retrieval/backend behavior, and the new transition boundaries. In particular, settled decisions and already-diagnosed bugs should not reload obsolete Decision/Debugging references.
-## 3. Complete public matrix
+## 5. Complete public matrices
-```powershell
-pwsh -NoProfile -File benchmarks/run.ps1 -Profile full -Runs 3 -Workers 3 -RequireStableRanking
-```
+`standard` without previous/no-Skill arms contains 342 cells:
-The current public `standard` profile runs 318 isolated cells when no previous-version or no-Skill arm is requested:
+- 9 Delivery × 2 arms × 3 = 54;
+- 42 adaptive-rigor × 1 arm × 3 = 126;
+- 6 Decision × 2 arms × 3 = 36;
+- 10 Debug × 2 arms × 3 = 60;
+- 22 Native Behavior × 1 arm × 3 = 66.
-- 9 Delivery cases × 2 arms × 3 runs = 54;
-- 38 Router cases × 1 arm × 3 runs = 114;
-- 6 Decision cases × 2 arms × 3 runs = 36;
-- 10 Debug cases × 2 arms × 3 runs = 60;
-- 18 native-behavior cases × 1 arm × 3 runs = 54.
+Run:
```powershell
-pwsh -NoProfile -File benchmarks/run.ps1 -Profile full -Runs 3 -Workers 3
+pwsh -NoProfile -File benchmarks/run.ps1 -Profile standard -Runs 3 -Workers 3 -RequireStableRanking
```
-`full` runs 420 cells without a previous-version or no-Skill arm:
+`full` without previous/no-Skill arms contains 444 cells:
-- 18 Delivery cases × 2 arms × 3 runs = 108;
-- 38 Router cases × 1 arm × 3 runs = 114;
-- 10 Decision cases × 2 arms × 3 runs = 60;
-- 14 Debug cases × 2 arms × 3 runs = 84;
-- 18 native-behavior cases × 1 arm × 3 runs = 54.
+- 18 Delivery × 2 arms × 3 = 108;
+- 42 adaptive-rigor × 1 arm × 3 = 126;
+- 10 Decision × 2 arms × 3 = 60;
+- 14 Debug × 2 arms × 3 = 84;
+- 22 Native Behavior × 1 arm × 3 = 66.
-The `smoke` profile intentionally remains small and defaults to one repetition. It is for harness/model sanity only.
-Do not publish `n=1` as a stable ranking.
-
-## 4. Run a before/after candidate gate
-
-While editing a dirty candidate, compare it with the checked-in version:
+Run:
```powershell
-pwsh -NoProfile -File benchmarks/run.ps1 `
- -Profile standard `
- -Runs 3 `
- -Workers 3 `
- -BaselineRef HEAD
+pwsh -NoProfile -File benchmarks/run.ps1 -Profile full -Runs 3 -Workers 3 -RequireStableRanking
```
-After a candidate has already been committed, use the actual previous revision or release tag instead, for example:
+The `smoke` profile intentionally remains small and defaults to one repetition. It is for harness/model sanity only. Do not publish `n=1` as a stable ranking.
+
+## 6. Run the v1.3 vs accepted-v1.2 candidate gate
+
+Use the accepted v1.2 baseline commit explicitly:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 `
- -Profile standard `
+ -Profile full `
-Runs 3 `
- -BaselineRef 75d501380bc4c9c26de8b81f8e4fc320717b0141
+ -Workers 3 `
+ -BaselineRef 88382d2b0c00fa278067a5933bbcacc86f46b56e `
+ -IncludeBaseline `
+ -RequireStableRanking
```
-The materialized baseline Skill is copied into the run directory. Both entrypoint and complete Skill-bundle hashes are recorded in `manifest.json`, preventing an ambiguous "previous version" comparison.
+The materialized baseline Skill is copied into the run directory. Entry-point and complete Skill-bundle hashes are recorded in `manifest.json`.
-For a comparison that will be published as a stable ranking, add `-RequireStableRanking`. It rejects effective `n<3`, incomplete runs, infrastructure failures, and Delivery rankings without production-build evidence. For release claims, also follow the stricter current/previous/no-Skill and claim-boundary requirements in [`NEXT_VALIDATION.md`](NEXT_VALIDATION.md).
+Do **not** compare v1.2 Router accuracy numerically with v1.3 adaptive-rigor accuracy: the classification schemas differ. Use unchanged Delivery/Decision/Debug task contracts for before/after performance claims, and use transition/reference-loading results to validate the new control architecture.
-## 5. Run a focused regression
-
-Selectors are repeatable:
+## 7. Focused explicit-security comparison
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 `
-Profile standard `
- -Suite router `
+ -Suite debug `
+ -Case security-path-containment `
+ -Case security-tenant-authorization `
-Runs 3 `
- -BaselineRef HEAD
+ -RequireStableRanking
```
-For a stable full candidate/baseline comparison:
+## 8. Combined-install matrix
+
+For the required integrated-stack comparison:
```powershell
pwsh -NoProfile -File benchmarks/run.ps1 `
-Profile full `
-Runs 3 `
-Workers 3 `
- -BaselineRef `
- -IncludeBaseline `
+ -ComboMatrix `
-RequireStableRanking
```
-The run manifest records the materialized Skill bundle and hashes so the baseline is explicit.
+Interpret this through the claim boundaries in [`NEXT_VALIDATION.md`](NEXT_VALIDATION.md). The existence of a combo result does not by itself establish universal superiority.
-## 6. Navigation ablation
+## 9. Navigation ablation
```powershell
python benchmarks/navigation_ablation.py `
@@ -197,25 +249,36 @@ python benchmarks/navigation_ablation.py `
--output benchmark-results\navigation-large-n3
```
-Publish repository commit, tracked files/bytes, the task oracle, and cold/warm graph observations. The current evidence does not establish a universal file-count threshold where graph navigation always wins.
+Publish repository commit, tracked files/bytes, the task oracle, and cold/warm graph observations. Current evidence does not establish a universal file-count threshold where graph navigation always wins.
+
+## 10. Evidence boundary
-## 7. Evidence boundary
+The committed v1.2 baseline supports statements such as:
-The v1.1 data support role-specific statements such as:
+- reasoning classification passed 114/114 cells under the v1.2 exact schema;
+- Retrieval exact classification passed 106/114;
+- Native Behavior passed 54/54;
+- the Practical-only Delivery/Decision/Debug regression passed 75/75.
-- current Practical passed 242 of 243 applicable cells across the split affected-surface reruns;
-- Delivery, Decision, Router, and Native Behavior were perfect in those reruns;
-- Debug correctness was perfect, while safety was 29/30 because one run missed a sibling caller;
-- the older 15-arm combo matrix remains useful interference evidence, but its comparator rows are cross-run rather than a new v1.9 paired scorecard.
+It does **not** validate the v1.3 adaptive-rigor classifier or its Retrieval intervals.
-They do **not** support:
+Likewise, a future public v1.3 regression does not by itself support:
- a universal ranking across every coding task/model/repository;
-- a claim that Practical is already experimentally better than **Ponytail + Superpowers installed together**;
-- a universal graph-navigation size threshold.
+- a claim that Practical is experimentally better than Ponytail + Superpowers installed together unless that combined arm is measured;
+- a universal graph-navigation size threshold;
+- broad generalization beyond the public regression corpus without held-out evidence.
+
+## 11. Reproducibility identity
-The exact combined-install comparison is a required arm in [`NEXT_VALIDATION.md`](NEXT_VALIDATION.md).
+The authoritative identity of a run is the manifest, not a marketing version label. Preserve:
-## 8. Reproducibility note about the public v1.1 label
+- candidate commit;
+- previous baseline commit when applicable;
+- canonical runner version and bundle SHA;
+- Skill entrypoint and complete bundle SHA;
+- model/reasoning setting;
+- pinned comparator commits;
+- profile/case list and repetition count.
-The committed aggregate files preserve the exact tested bundle hashes and runner versions. The v1.1 release metadata bump happened after the behavioral runs; the tested Skill text is otherwise the released routing/reference behavior. Treat the recorded hashes, runner versions, and pinned comparator commits as the authoritative identities of the measured runs.
+Historical v1.0/v1.1/v1.2 aggregate files keep their original identities and should not be rewritten to look like v1.3 results.
diff --git a/benchmarks/adaptive_rigor.py b/benchmarks/adaptive_rigor.py
new file mode 100644
index 0000000..3dccb3c
--- /dev/null
+++ b/benchmarks/adaptive_rigor.py
@@ -0,0 +1,434 @@
+"""Adaptive-rigor benchmark adapter for Practical Coding v1.3.
+
+Keeps the v2.0 execution harness intact while changing the classification contract:
+Decision is a pre-execution gate; Direct/Debugging/Implementation are execution states;
+Retrieval is scored as a minimum-sufficient / maximum-reasonable cost interval.
+"""
+
+from __future__ import annotations
+
+import datetime as dt
+import json
+import os
+import re
+from pathlib import Path
+from typing import Any
+
+
+DECISION_STATES = ("CLEAR", "REQUIRED")
+EXECUTION_STATES = ("BLOCKED", "DIRECT", "DEBUGGING", "IMPLEMENTATION")
+RETRIEVAL_ORDER = ("NONE", "TARGETED", "BOUNDED", "STRUCTURAL")
+_RETRIEVAL_INDEX = {name: index for index, name in enumerate(RETRIEVAL_ORDER)}
+
+
+TRANSITION_CASES = {
+ "transition-decision-to-direct": {
+ "decision": "CLEAR",
+ "execution": "DIRECT",
+ "retrieval_min": "TARGETED",
+ "retrieval_max": "BOUNDED",
+ "prompt": (
+ "The earlier storage choice is settled: repository policy requires Python sqlite3. "
+ "save_history() is the named function and the existing focused test is sufficient. "
+ "Classify the current state now."
+ ),
+ },
+ "transition-decision-to-implementation": {
+ "decision": "CLEAR",
+ "execution": "IMPLEMENTATION",
+ "retrieval_min": "BOUNDED",
+ "retrieval_max": "STRUCTURAL",
+ "prompt": (
+ "The earlier choice is settled: use a dual-read rolling migration. "
+ "The producers, readers, rollback surface, and mixed-version storage contract are still unmapped. "
+ "Classify the current state now."
+ ),
+ },
+ "transition-debug-to-direct": {
+ "decision": "CLEAR",
+ "execution": "DIRECT",
+ "retrieval_min": "TARGETED",
+ "retrieval_max": "BOUNDED",
+ "prompt": (
+ "The failing test and trace already establish the root cause: parse_bool() does not strip "
+ "surrounding whitespace. The shared parser and focused check are known. Classify the current state now."
+ ),
+ },
+ "transition-debug-to-implementation": {
+ "decision": "CLEAR",
+ "execution": "IMPLEMENTATION",
+ "retrieval_min": "BOUNDED",
+ "retrieval_max": "STRUCTURAL",
+ "prompt": (
+ "Diagnosis is complete: invoice leakage comes from checking owner_id without tenant ownership. "
+ "However the authoritative tenant authorization boundary, indirect callers, rejection-before-read "
+ "contract, and sufficient security evidence are still unmapped. Classify the current state now."
+ ),
+ },
+}
+
+
+TRANSITION_BEHAVIOR_CASES = {
+ "native-transition-decision-to-direct": {
+ "prompt": (
+ "The storage decision is already settled: repository policy requires Python sqlite3. "
+ "save_history() in cli.py is the only affected function and its focused test is sufficient. "
+ "Implement it without reopening the settled decision."
+ ),
+ "files": {
+ "cli.py": "import sqlite3\n\ndef save_history(connection, command):\n pass\n",
+ },
+ "reasoning_module": None,
+ "retrieval": "TARGETED",
+ },
+ "native-transition-decision-to-implementation": {
+ "prompt": (
+ "The migration strategy is already settled as dual-read rolling migration. "
+ "Before editing, the readers, writers, rollback path, mixed-version guarantees, and authoritative "
+ "storage boundary are still unmapped. Determine the safe implementation surface without reopening "
+ "the settled strategy choice."
+ ),
+ "files": {
+ "sessions.py": "def encode_session(value):\n return value\n\ndef decode_session(raw):\n return raw\n",
+ },
+ "reasoning_module": "implementation.md",
+ "retrieval": "STRUCTURAL",
+ },
+ "native-transition-debug-to-direct": {
+ "prompt": (
+ "The failing test and trace already establish the cause: parse_bool() does not strip whitespace. "
+ "Fix that known shared parser and run the focused check; do not restart diagnosis."
+ ),
+ "files": {
+ "config.py": (
+ "def parse_bool(value):\n"
+ " return value.lower() in {'1', 'true', 'yes'}\n\n"
+ "def feature_enabled(env):\n"
+ " return parse_bool(env.get('FEATURE', 'false'))\n"
+ ),
+ },
+ "reasoning_module": None,
+ "retrieval": "TARGETED",
+ },
+ "native-transition-debug-to-implementation": {
+ "prompt": (
+ "Diagnosis is complete: cross-tenant invoice access comes from checking owner_id without tenant "
+ "ownership. The authoritative permission boundary, indirect callers, rejection-before-read invariant, "
+ "and sufficient security evidence remain unmapped. Determine the safe implementation surface."
+ ),
+ "files": {
+ "authorization.py": (
+ "def owns_record(user, record):\n"
+ " return user['id'] == record['owner_id']\n\n"
+ "def can_view_invoice(user, invoice):\n"
+ " return owns_record(user, invoice)\n"
+ ),
+ },
+ "reasoning_module": "implementation.md",
+ "retrieval": "STRUCTURAL",
+ },
+}
+
+
+def retrieval_bounds(expected: str) -> tuple[str, str]:
+ """Map the old exact label to a principled cost interval.
+
+ NONE permits bounded discovery because open decisions may reasonably need repository facts.
+ TARGETED permits bounded discovery when the known concept's location is not directly available.
+ BOUNDED remains exact: selecting no context is insufficient and structural work is unnecessary.
+ STRUCTURAL accepts bounded fallback when no structural backend is available.
+ """
+ if expected == "NONE":
+ return "NONE", "BOUNDED"
+ if expected == "TARGETED":
+ return "TARGETED", "BOUNDED"
+ if expected == "BOUNDED":
+ return "BOUNDED", "BOUNDED"
+ if expected == "STRUCTURAL":
+ return "BOUNDED", "STRUCTURAL"
+ raise ValueError(expected)
+
+
+def _convert_case(old: tuple[str, str, str]) -> dict[str, str]:
+ reasoning, retrieval, prompt = old
+ if reasoning == "DECISION":
+ decision, execution = "REQUIRED", "BLOCKED"
+ elif reasoning == "NONE":
+ decision, execution = "CLEAR", "DIRECT"
+ elif reasoning == "DEBUGGING":
+ decision, execution = "CLEAR", "DEBUGGING"
+ elif reasoning == "IMPLEMENTATION":
+ decision, execution = "CLEAR", "IMPLEMENTATION"
+ else:
+ raise ValueError(reasoning)
+ minimum, maximum = retrieval_bounds(retrieval)
+ return {
+ "decision": decision,
+ "execution": execution,
+ "retrieval_min": minimum,
+ "retrieval_max": maximum,
+ "prompt": prompt,
+ }
+
+
+def parse_rigor_answer(answer: str) -> tuple[str, str, str]:
+ match = re.fullmatch(
+ r"\s*`?\s*DECISION\s*=\s*(CLEAR|REQUIRED)\s*;\s*"
+ r"EXECUTION\s*=\s*(BLOCKED|DIRECT|DEBUGGING|IMPLEMENTATION)\s*;\s*"
+ r"RETRIEVAL\s*=\s*(NONE|TARGETED|BOUNDED|STRUCTURAL)\s*`?\s*[.]?\s*",
+ answer.upper(),
+ )
+ return match.groups() if match else ("", "", "")
+
+
+def score_rigor(spec: dict[str, str], answer: str) -> dict[str, Any]:
+ actual_decision, actual_execution, actual_retrieval = parse_rigor_answer(answer)
+ decision_ok = actual_decision == spec["decision"]
+ execution_ok = actual_execution == spec["execution"]
+ state_valid = not (
+ (actual_decision == "REQUIRED" and actual_execution != "BLOCKED")
+ or (actual_decision == "CLEAR" and actual_execution == "BLOCKED")
+ )
+ if actual_retrieval in _RETRIEVAL_INDEX:
+ actual_index = _RETRIEVAL_INDEX[actual_retrieval]
+ min_index = _RETRIEVAL_INDEX[spec["retrieval_min"]]
+ max_index = _RETRIEVAL_INDEX[spec["retrieval_max"]]
+ retrieval_sufficient = actual_index >= min_index
+ retrieval_efficient = actual_index <= max_index
+ else:
+ retrieval_sufficient = retrieval_efficient = False
+ retrieval_ok = retrieval_sufficient and retrieval_efficient
+ return {
+ "expected_decision": spec["decision"],
+ "actual_decision": actual_decision,
+ "decision_ok": decision_ok,
+ "expected_execution": spec["execution"],
+ "actual_execution": actual_execution,
+ "execution_ok": execution_ok,
+ "state_valid": state_valid,
+ "retrieval_min": spec["retrieval_min"],
+ "retrieval_max": spec["retrieval_max"],
+ "actual_retrieval": actual_retrieval,
+ "retrieval_sufficient": retrieval_sufficient,
+ "retrieval_efficient": retrieval_efficient,
+ "retrieval_ok": retrieval_ok,
+ "passed": decision_ok and execution_ok and state_valid and retrieval_ok,
+ }
+
+
+def install(bench: Any) -> None:
+ if getattr(bench, "_adaptive_rigor_installed", False):
+ return
+
+ bench.DECISION_STATES = DECISION_STATES
+ bench.EXECUTION_STATES = EXECUTION_STATES
+ bench.RETRIEVAL_MODES = RETRIEVAL_ORDER
+ bench.ROUTER_CASES = {name: _convert_case(spec) for name, spec in bench.ROUTER_CASES.items()}
+ bench.ROUTER_CASES.update(TRANSITION_CASES)
+ bench.BEHAVIOR_CASES.update(TRANSITION_BEHAVIOR_CASES)
+
+ for profile in ("standard", "full"):
+ bench.PROFILE_CASES[profile]["router"] = list(bench.ROUTER_CASES)
+ bench.PROFILE_CASES[profile]["behavior"] = list(bench.BEHAVIOR_CASES)
+
+ smoke = bench.PROFILE_CASES["smoke"]
+ smoke["router"] = [
+ "direct-artifact",
+ "decision-auth",
+ "debug-named-function",
+ "verification-risk",
+ "transition-debug-to-direct",
+ ]
+
+ original_run_cell = bench.run_cell
+ original_rescore = bench.rescore_run
+
+ def run_cell(spec: tuple[str, str, str, int], args: Any, sources: dict[str, Path], previous: Path | None, ponytail: Any, eval_homes: dict[str, Path], output: Path) -> dict[str, Any]:
+ suite, case, arm, repetition = spec
+ if suite != "router":
+ return original_run_cell(spec, args, sources, previous, ponytail, eval_homes, output)
+
+ cell = output / "cells" / suite / case / arm / f"r{repetition:03d}"
+ cell.mkdir(parents=True, exist_ok=False)
+ workspace = cell / "workspace"
+ workspace.mkdir()
+
+ case_spec = bench.ROUTER_CASES[case]
+ loaded = bench.skill_text(arm, sources, previous, suite=suite)
+ prompt = (
+ "Classify the current adaptive-rigor state. Return exactly: "
+ "DECISION=; "
+ "EXECUTION=; "
+ "RETRIEVAL=. "
+ "DECISION=REQUIRED only when an unresolved material choice blocks or materially changes the next safe action; "
+ "then EXECUTION must be BLOCKED. Otherwise DECISION=CLEAR. "
+ "DIRECT means the Core is sufficient now. DEBUGGING means an observed failure exists and its cause is not evidenced. "
+ "IMPLEMENTATION means safe execution is blocked by an unknown contract/invariant, unresolved material risk boundary, "
+ "or insufficient evidence for a risky claim. These execution states are alternatives, not sequential stages. "
+ "Retrieval describes the cheapest sufficient repository-context cost for the next action: NONE, known TARGETED source, "
+ "BOUNDED/ranked discovery, or STRUCTURAL relationship mapping. Classify only; do not use tools or solve it.\n\n"
+ "Request: " + case_spec["prompt"] + "\n\n" + loaded
+ )
+ (cell / "prompt.txt").write_text(prompt, encoding="utf-8")
+
+ env = os.environ.copy()
+ env["CODEX_HOME"] = str(eval_homes["default"])
+ codex = bench.resolve_codex(args.codex)
+ first_out, first_err = cell / "round1.jsonl", cell / "round1.stderr.txt"
+ code, timed_out, forced, duration = bench.run_codex(
+ bench.codex_command(codex, workspace),
+ prompt,
+ workspace,
+ env,
+ first_out,
+ first_err,
+ args.timeout,
+ )
+ parsed = bench.parse_transcript(first_out)
+ answers = [parsed["answer"]]
+ usage = parsed["usage"]
+ record: dict[str, Any] = {
+ "suite": suite,
+ "case": case,
+ "arm": arm,
+ "repetition": repetition,
+ "exit_status": code,
+ "timed_out": timed_out,
+ "forced_after_completion": forced,
+ "duration_seconds": duration,
+ "setup_duration_seconds": 0.0,
+ "dependency_setup": None,
+ "tool_calls": parsed["tool_calls"],
+ **usage,
+ "workspace": str(workspace),
+ "answers": answers,
+ }
+ infrastructure_error = "timeout" if timed_out else (f"codex exit status {code}" if code and not forced else None)
+ if infrastructure_error:
+ record.update({"passed": None, "verdict": "indeterminate", "error": infrastructure_error})
+ else:
+ record.update(score_rigor(case_spec, answers[0]))
+ record["verdict"] = "pass" if record["passed"] else "fail"
+
+ (cell / "answer.md").write_text(answers[0] + "\n", encoding="utf-8")
+ (cell / "result.json").write_text(json.dumps(record, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
+ return record
+
+ def rescore_run(run_dir: Path, ponytail: Any) -> None:
+ results_path = run_dir / "results.json"
+ manifest_path = run_dir / "manifest.json"
+ records = json.loads(results_path.read_text(encoding="utf-8"))
+ if not any(record.get("suite") == "router" for record in records):
+ original_rescore(run_dir, ponytail)
+ return
+
+ manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+ for record in records:
+ suite, case = record["suite"], record["case"]
+ answers = record.get("answers") or []
+ if suite == "router":
+ record.update(score_rigor(bench.ROUTER_CASES[case], answers[0] if answers else ""))
+ elif suite == "decision":
+ first = bench.decision_metrics(answers[0]) if answers else bench.decision_metrics("")
+ second = bench.decision_metrics(answers[1]) if len(answers) > 1 else {
+ "questions": 999,
+ "recommendations": 0,
+ "has_tradeoff": False,
+ "attempted_implementation": False,
+ }
+ converged = second["questions"] == 0 and any(
+ term in (answers[-1].lower() if answers else "")
+ for term in bench.DECISION_CASES[case]["expected"]
+ )
+ passed = (
+ first["questions"] > 0
+ and first["recommendations"] >= first["questions"]
+ and not first["attempted_implementation"]
+ and converged
+ and not second["attempted_implementation"]
+ )
+ record.update({"first_round": first, "second_round": second, "converged": converged, "passed": passed})
+ elif suite == "behavior":
+ commands: list[str] = []
+ outputs: list[str] = []
+ for transcript in sorted(Path(record["workspace"]).parent.glob("round*.jsonl")):
+ parsed = bench.parse_transcript(transcript)
+ commands.extend(parsed["tool_commands"])
+ outputs.extend(parsed["tool_outputs"])
+ behavior = bench.BEHAVIOR_CASES[case]
+ record.update(
+ bench.behavior_score(
+ commands,
+ behavior["reasoning_module"],
+ outputs,
+ behavior["retrieval"],
+ behavior.get("backend"),
+ )
+ )
+ elif "workspace" in record:
+ workspace = Path(record["workspace"])
+ scored = (
+ ponytail.score_workspace(case, record["arm"], bench.MODEL, workspace)
+ if case in ponytail.TASKS
+ else bench.custom_debug_score(case, workspace)
+ )
+ if case not in ponytail.TASKS:
+ scored.update(ponytail.git_diff_stats(workspace))
+ record.update(scored)
+ build = record.get("build")
+ if build is not None:
+ build["infrastructure_error"] = build.get("infrastructure_error") or bench.build_infrastructure_error(
+ build.get("output_tail", "")
+ )
+ if build and build.get("infrastructure_error"):
+ record["passed"] = None
+ record["indeterminate_reason"] = build["infrastructure_error"]
+ else:
+ record["passed"] = (
+ scored.get("correct") == 1
+ and scored.get("safe") == 1
+ and (build is None or build.get("passed"))
+ )
+ if record.get("error") or record.get("timed_out"):
+ record["verdict"] = "indeterminate"
+ else:
+ record["verdict"] = (
+ "indeterminate"
+ if record.get("passed") is None
+ else ("pass" if record["passed"] else "fail")
+ )
+
+ summary = bench.aggregate(records)
+ deltas = bench.comparisons(summary)
+ rollups = bench.suite_rollups(records)
+ rollup_deltas = bench.comparisons(rollups)
+ cards = bench.scorecards(summary, rollups)
+ results_path.write_text(json.dumps(records, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
+ (run_dir / "summary.json").write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")
+ (run_dir / "comparisons.json").write_text(json.dumps(deltas, indent=2) + "\n", encoding="utf-8")
+ (run_dir / "rollups.json").write_text(json.dumps(rollups, indent=2) + "\n", encoding="utf-8")
+ (run_dir / "rollup-comparisons.json").write_text(json.dumps(rollup_deltas, indent=2) + "\n", encoding="utf-8")
+ (run_dir / "scorecards.json").write_text(json.dumps(cards, indent=2) + "\n", encoding="utf-8")
+ manifest.update({
+ "rescored_at": dt.datetime.now(dt.timezone.utc).isoformat(),
+ "runner_version": bench.VERSION,
+ "runner_sha256": bench.sha256(Path(bench.__file__)),
+ })
+ manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8")
+ bench.write_report(
+ run_dir / "report.md",
+ manifest,
+ summary,
+ deltas,
+ float(manifest.get("suite_elapsed_seconds", 0)),
+ rollups,
+ rollup_deltas,
+ cards,
+ )
+ print(f"rescored {len(records)} cells in {run_dir}")
+
+ bench.parse_router_answer = parse_rigor_answer
+ bench.score_rigor = score_rigor
+ bench.run_cell = run_cell
+ bench.rescore_run = rescore_run
+ bench._adaptive_rigor_installed = True
diff --git a/benchmarks/run_catalog.py b/benchmarks/run_catalog.py
index 97bf6b9..f33e3f4 100644
--- a/benchmarks/run_catalog.py
+++ b/benchmarks/run_catalog.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-"""Canonical benchmark entrypoint with the extended public case catalog installed."""
+"""Canonical benchmark entrypoint with the public catalog and v1.3 adaptive-rigor contract installed."""
from __future__ import annotations
@@ -12,13 +12,15 @@
sys.path.insert(0, str(HERE))
import run_benchmarks as bench
-from case_catalog import install
+from adaptive_rigor import install as install_adaptive_rigor
+from case_catalog import install as install_catalog
_CORE_SHA256 = bench.sha256
_RUNTIME_FILES = (
Path(bench.__file__).resolve(),
(HERE / "case_catalog.py").resolve(),
+ (HERE / "adaptive_rigor.py").resolve(),
Path(__file__).resolve(),
)
@@ -41,8 +43,9 @@ def catalog_aware_sha256(path: Path) -> str:
def configure() -> None:
- install(bench)
- bench.VERSION = "2.0"
+ install_catalog(bench)
+ install_adaptive_rigor(bench)
+ bench.VERSION = "2.1"
bench.sha256 = catalog_aware_sha256
diff --git a/benchmarks/test_benchmarks.py b/benchmarks/test_benchmarks.py
index 2606cd8..8f73f9b 100644
--- a/benchmarks/test_benchmarks.py
+++ b/benchmarks/test_benchmarks.py
@@ -5,6 +5,11 @@
from unittest import mock
from benchmarks import run_benchmarks as bench
+from benchmarks.adaptive_rigor import install as install_adaptive_rigor
+from benchmarks.case_catalog import install as install_catalog
+
+install_catalog(bench)
+install_adaptive_rigor(bench)
class BenchmarkHarnessTests(unittest.TestCase):
@@ -79,32 +84,39 @@ def test_snapshot_workspace_creates_checked_longpath_compatible_baseline(self):
self.assertEqual(head.returncode, 0, head.stderr)
self.assertEqual(longpaths.stdout.strip(), "true")
- def test_router_matrix_covers_three_reasoning_routes_plus_no_route(self):
- self.assertEqual(
- {reasoning for reasoning, _, _ in bench.ROUTER_CASES.values()},
- {"NONE", "DECISION", "DEBUGGING", "IMPLEMENTATION"},
- )
- self.assertEqual(len(set(bench.REASONING_ROUTES) - {"NONE"}), 3)
+ def test_rigor_matrix_covers_decision_gate_and_execution_states(self):
+ decisions = {case["decision"] for case in bench.ROUTER_CASES.values()}
+ executions = {case["execution"] for case in bench.ROUTER_CASES.values()}
+ self.assertEqual(decisions, set(bench.DECISION_STATES))
+ self.assertEqual(executions, set(bench.EXECUTION_STATES))
+ for case in bench.ROUTER_CASES.values():
+ if case["decision"] == "REQUIRED":
+ self.assertEqual(case["execution"], "BLOCKED")
+ else:
+ self.assertNotEqual(case["execution"], "BLOCKED")
- def test_router_matrix_covers_independent_retrieval_dimension(self):
- self.assertEqual(
- {retrieval for _, retrieval, _ in bench.ROUTER_CASES.values()},
- set(bench.RETRIEVAL_MODES),
- )
- self.assertIn(("NONE", "STRUCTURAL"), {(reasoning, retrieval) for reasoning, retrieval, _ in bench.ROUTER_CASES.values()})
- self.assertIn(("IMPLEMENTATION", "STRUCTURAL"), {(reasoning, retrieval) for reasoning, retrieval, _ in bench.ROUTER_CASES.values()})
+ def test_retrieval_matrix_uses_cost_intervals(self):
+ for case in bench.ROUTER_CASES.values():
+ self.assertIn(case["retrieval_min"], bench.RETRIEVAL_MODES)
+ self.assertIn(case["retrieval_max"], bench.RETRIEVAL_MODES)
+ self.assertLessEqual(
+ bench.RETRIEVAL_MODES.index(case["retrieval_min"]),
+ bench.RETRIEVAL_MODES.index(case["retrieval_max"]),
+ )
+ self.assertTrue(any(case["retrieval_min"] != case["retrieval_max"] for case in bench.ROUTER_CASES.values()))
- def test_router_answer_parser_requires_both_dimensions(self):
+ def test_rigor_answer_parser_requires_all_dimensions(self):
self.assertEqual(
- bench.parse_router_answer("REASONING=DEBUGGING; RETRIEVAL=BOUNDED"),
- ("DEBUGGING", "BOUNDED"),
+ bench.parse_router_answer("DECISION=CLEAR; EXECUTION=DEBUGGING; RETRIEVAL=BOUNDED"),
+ ("CLEAR", "DEBUGGING", "BOUNDED"),
)
- self.assertEqual(bench.parse_router_answer("DEBUGGING"), ("", ""))
+ self.assertEqual(bench.parse_router_answer("DEBUGGING"), ("", "", ""))
- def test_core_is_route_agnostic_and_router_owns_escalation(self):
+ def test_core_is_route_agnostic_and_escalation_sections_own_rigor(self):
skill = (bench.ROOT / "SKILL.md").read_text(encoding="utf-8")
- core = skill.split("## Core", 1)[1].split("## Direct Path", 1)[0]
- router = skill.split("## Event Router", 1)[1].split("## Retrieval Policy", 1)[0]
+ core = skill.split("## Core", 1)[1].split("## Decision Gate", 1)[0]
+ decision = skill.split("## Decision Gate", 1)[1].split("## Execution Escalation", 1)[0]
+ execution = skill.split("## Execution Escalation", 1)[1].split("## Retrieval Policy", 1)[0]
retrieval = skill.split("## Retrieval Policy", 1)[1].split("## Isolation Gate", 1)[0]
self.assertIn("minimum local code", core)
@@ -112,8 +124,8 @@ def test_core_is_route_agnostic_and_router_owns_escalation(self):
for module_specific in (
"references/",
"user-owned",
- "security/permissions",
- "persistence/migration",
+ "security or permissions",
+ "persistence or migration",
"Decision",
"Debugging",
"Implementation",
@@ -121,14 +133,15 @@ def test_core_is_route_agnostic_and_router_owns_escalation(self):
):
self.assertNotIn(module_specific, core)
- self.assertIn("observed failure", router)
- self.assertIn("whether or which external dependency", router)
- self.assertIn("specified and authorized", router)
- self.assertIn("security/permissions", router)
- self.assertIn("persistence/migration", router)
- self.assertNotIn("navigation.md", router.lower())
+ self.assertIn("material unresolved choice", decision)
+ self.assertIn("genuinely user-owned", decision)
+ self.assertIn("Start Direct", execution)
+ self.assertIn("observed failure", execution)
+ self.assertIn("material risk boundary", execution)
+ self.assertIn("not sequential stages", execution)
self.assertIn("structural code index", retrieval)
self.assertIn("references/navigation.md", retrieval)
+ self.assertIn("cost bounds", retrieval)
def test_decision_suite_inlines_decision_module(self):
with tempfile.TemporaryDirectory() as tmp:
@@ -174,7 +187,7 @@ def test_native_behavior_matrix_covers_direct_and_every_module(self):
{None, "decision.md", "debugging.md", "implementation.md"},
)
- def test_navigation_backend_is_scored_separately_from_reasoning_route(self):
+ def test_navigation_backend_is_scored_separately_from_execution_rigor(self):
commands = [
"Get-Content C:/eval/skills/practical-coding/SKILL.md",
"Get-Content C:/eval/skills/practical-coding/references/navigation.md",
@@ -201,7 +214,7 @@ def test_behavior_score_uses_loaded_content_not_recursive_filename_listing(self)
self.assertTrue(score["passed"])
self.assertEqual(score["module_reads"], ["decision.md"])
- def test_behavior_direct_path_rejects_reference_preload(self):
+ def test_behavior_direct_rejects_reference_preload(self):
trigger = ["Get-Content C:/eval/skills/practical-coding/SKILL.md"]
self.assertTrue(bench.behavior_score(trigger, None)["passed"])
self.assertFalse(
diff --git a/benchmarks/test_catalog.py b/benchmarks/test_catalog.py
index ba89866..3316d87 100644
--- a/benchmarks/test_catalog.py
+++ b/benchmarks/test_catalog.py
@@ -4,12 +4,17 @@
from benchmarks import run_benchmarks as bench
from benchmarks import run_catalog
+from benchmarks.adaptive_rigor import (
+ TRANSITION_BEHAVIOR_CASES,
+ TRANSITION_CASES,
+ install as install_adaptive_rigor,
+)
from benchmarks.case_catalog import (
EXTRA_BEHAVIOR_CASES,
EXTRA_DEBUG_CASES,
EXTRA_DECISION_CASES,
EXTRA_ROUTER_CASES,
- install,
+ install as install_catalog,
score_extra_debug,
)
from benchmarks.debug_oracles import DEBUG_ORACLES
@@ -18,43 +23,56 @@
class ExpandedCatalogTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
- install(bench)
+ install_catalog(bench)
+ install_adaptive_rigor(bench)
def test_public_matrix_is_materially_broader(self):
- self.assertEqual(len(bench.ROUTER_CASES), 38)
+ self.assertEqual(len(bench.ROUTER_CASES), 42)
self.assertEqual(len(bench.DECISION_CASES), 10)
self.assertEqual(len(bench.PROFILE_CASES["standard"]["decision"]), 6)
self.assertEqual(len(bench.PROFILE_CASES["full"]["decision"]), 10)
self.assertEqual(len(bench.PROFILE_CASES["standard"]["debug"]), 10)
self.assertEqual(len(bench.PROFILE_CASES["full"]["debug"]), 14)
- self.assertEqual(len(bench.PROFILE_CASES["standard"]["behavior"]), 18)
- self.assertEqual(len(bench.PROFILE_CASES["full"]["behavior"]), 18)
+ self.assertEqual(len(bench.PROFILE_CASES["standard"]["behavior"]), 22)
+ self.assertEqual(len(bench.PROFILE_CASES["full"]["behavior"]), 22)
def test_only_explicit_security_cases_get_strict_case_level_safety(self):
expected = {case for case, spec in EXTRA_DEBUG_CASES.items() if spec.get("risk") == "security"}
self.assertEqual(bench.STRICT_SAFETY_CASES, expected)
- def test_expansion_is_not_just_one_route_or_bug_shape(self):
+ def test_legacy_expansion_spans_all_old_reasoning_and_retrieval_shapes(self):
extra_reasoning = {reasoning for reasoning, _, _ in EXTRA_ROUTER_CASES.values()}
- self.assertEqual(
- extra_reasoning,
- {"NONE", "DECISION", "DEBUGGING", "IMPLEMENTATION"},
- )
+ self.assertEqual(extra_reasoning, {"NONE", "DECISION", "DEBUGGING", "IMPLEMENTATION"})
self.assertEqual({retrieval for _, retrieval, _ in EXTRA_ROUTER_CASES.values()}, set(bench.RETRIEVAL_MODES))
self.assertEqual(len({case["score"] for case in EXTRA_DEBUG_CASES.values()}), len(EXTRA_DEBUG_CASES))
self.assertGreaterEqual(len(EXTRA_DECISION_CASES), 6)
+ def test_adaptive_contract_adds_transition_boundaries(self):
+ self.assertEqual(
+ set(TRANSITION_CASES),
+ {
+ "transition-decision-to-direct",
+ "transition-decision-to-implementation",
+ "transition-debug-to-direct",
+ "transition-debug-to-implementation",
+ },
+ )
+ self.assertEqual(set(TRANSITION_BEHAVIOR_CASES), {
+ "native-transition-decision-to-direct",
+ "native-transition-decision-to-implementation",
+ "native-transition-debug-to-direct",
+ "native-transition-debug-to-implementation",
+ })
+ for case in TRANSITION_CASES:
+ self.assertIn(case, bench.ROUTER_CASES)
+ for case in TRANSITION_BEHAVIOR_CASES:
+ self.assertIn(case, bench.BEHAVIOR_CASES)
+
def test_extreme_behavior_cases_pair_direct_and_escalated_risks(self):
modules = {case["reasoning_module"] for case in EXTRA_BEHAVIOR_CASES.values()}
self.assertEqual(modules, {None, "decision.md", "debugging.md", "implementation.md"})
- self.assertEqual(
- sum(case["reasoning_module"] is None for case in EXTRA_BEHAVIOR_CASES.values()),
- 3,
- )
- self.assertEqual(
- sum(case["reasoning_module"] == "implementation.md" for case in EXTRA_BEHAVIOR_CASES.values()),
- 3,
- )
+ self.assertEqual(sum(case["reasoning_module"] is None for case in EXTRA_BEHAVIOR_CASES.values()), 3)
+ self.assertEqual(sum(case["reasoning_module"] == "implementation.md" for case in EXTRA_BEHAVIOR_CASES.values()), 3)
def test_profiles_have_no_duplicate_case_ids(self):
for profile in ("standard", "full"):
@@ -84,15 +102,14 @@ def test_decision_cases_have_two_turn_resolution_contract(self):
self.assertIn("Resolve the decision now", spec["reply"])
self.assertTrue(spec["expected"])
- def test_canonical_runner_fingerprint_includes_catalog(self):
- raw_core = bench.sha256(Path(bench.__file__))
+ def test_canonical_runner_fingerprint_includes_catalog_and_adaptive_adapter(self):
+ run_catalog.configure()
+ raw_core = run_catalog._CORE_SHA256(Path(run_catalog.bench.__file__))
bundled = run_catalog.runner_bundle_sha256()
self.assertEqual(len(bundled), 64)
self.assertNotEqual(bundled, raw_core)
- self.assertEqual(
- run_catalog.catalog_aware_sha256(Path(run_catalog.bench.__file__)),
- bundled,
- )
+ self.assertEqual(run_catalog.catalog_aware_sha256(Path(run_catalog.bench.__file__)), bundled)
+ self.assertEqual(run_catalog.bench.VERSION, "2.1")
if __name__ == "__main__":
diff --git a/references/debugging.md b/references/debugging.md
index 2035014..2382bf3 100644
--- a/references/debugging.md
+++ b/references/debugging.md
@@ -1,6 +1,8 @@
# Debugging
-Load this module only for an observed or reported failure, regression, incorrect behavior, or failed verification.
+Load this module only when an observed or reported failure, regression, incorrect behavior, or failed verification exists and its cause is not yet evidenced. Debugging is an execution escalation profile, not a task category or a mandatory stage for every bug report.
+
+If the cause is already established by fresh evidence and the safe fix is known, stay Direct with the Core.
## Evidence First
@@ -27,9 +29,9 @@ Judge a fix by the delivered code, not by whether it followed a named debugging
## Stay in Scope
-- Do not expand beyond that contract into a repo-wide search for unrelated defects.
-- Do not write tests merely because debugging occurred or because the repaired logic is non-trivial. Use the cheapest reproduction or focused check that can falsify the fix; add a durable targeted test only when regression risk, project requirements, or the evidence plan in `implementation.md` justifies its lasting value.
-- If diagnosis exposes a different material blocker such as an unresolved design/dependency choice or a previously unknown risk boundary, report that blocker to the root. Do not load another Practical Coding reference from this module; the root decides whether the Core is sufficient or whether isolated follow-up work is worth its handoff cost.
+- Do not expand beyond the diagnosed contract into a repo-wide search for unrelated defects.
+- Do not write tests merely because debugging occurred or because the repaired logic is non-trivial. Use the cheapest reproduction or focused check that can falsify the fix; add a durable targeted test only when regression risk, project requirements, or a still-unresolved evidence boundary justifies its lasting value.
+- If diagnosis exposes a material choice that changes the next action or an unresolved execution boundary that makes the fix unsafe, return that blocker to the root. Do not load another Practical Coding reference from this module.
## Exit
@@ -37,3 +39,4 @@ Judge a fix by the delivered code, not by whether it followed a named debugging
- Exercise the nearest shared caller or boundary when the root cause could affect more than the named symptom.
- Remove temporary diagnostic instrumentation unless it has durable operational value.
- Report remaining uncertainty rather than hiding it behind additional defensive code.
+- Return to the Core. Do not escalate to Implementation merely because Debugging occurred; escalate only if an unresolved material execution boundary still blocks safe action.
diff --git a/references/decision.md b/references/decision.md
index 93acfa1..c97ca38 100644
--- a/references/decision.md
+++ b/references/decision.md
@@ -1,14 +1,16 @@
# Decision
-Load this module only when a material choice about architecture, dependencies, APIs, data models, compatibility, or multiple plausible implementations remains open—including whether or which package, library, service, or mature external implementation to adopt. Its output is a resolved choice that changes the next action, not a design essay or an option dump.
+Load this module only when a material unresolved choice blocks or materially changes the next safe action. Its output is a settled choice and the constraints needed for execution, not a design essay, option dump, or mandatory interview.
-Do not load this module when the request or repository has already settled the material choice. The existence of a popular alternative is not by itself a Decision event.
+Do not load this module when the request, repository, authoritative constraint, or a cheap reversible default already settles the choice. The existence of another package, library, service, architecture, or implementation is not by itself a Decision Gate.
## Decision Frontier
-Resolve discoverable facts from the repository and authoritative sources before asking the user. Keep a compact ledger of verified facts, constraints, assumptions, decisions, and unresolved choices. Work only on the current frontier: choices whose prerequisites are already known. Do not ask about a downstream choice while an upstream answer could invalidate it.
+Resolve discoverable facts from the repository and authoritative sources before asking the user. Keep a compact ledger of verified facts, constraints, assumptions, settled choices, and unresolved choices. Work only on the current frontier: choices whose prerequisites are already known. Do not ask about a downstream choice while an upstream answer could invalidate it.
-Research is part of resolving an open Decision. Compare viable mature implementations when external evidence is necessary; do not ask the user merely for permission to research. Ask only about user-owned scope, compatibility, risk tolerance, cost, or preference when at least two plausible answers lead to materially different next actions and choosing the wrong default costs more than one interaction. Ask every independent user-owned decision on the current frontier in the same round; dependent questions wait for a later round. For every question:
+Research is part of resolving an open Decision. Compare viable mature implementations when external evidence is necessary; do not ask the user merely for permission to research. Ask only about user-owned scope, compatibility policy, risk tolerance, cost, or preference when at least two plausible answers lead to materially different next actions and choosing the wrong default costs more than one interaction.
+
+Ask every independent user-owned decision on the current frontier in the same round; dependent questions wait for a later round. For every question:
- explain briefly why the decision matters now;
- recommend one option and give the reason;
@@ -23,18 +25,24 @@ Recommendation:
Trade-off:
```
-End the round with the smallest answer format, then wait. If the task is already sufficiently specified, ask nothing. If uncertainty is cheap and reversible, choose the repository or platform default and proceed. Each reply reshapes the decision tree: record settled choices, recompute the frontier, and reopen a dependent choice if new evidence contradicts an earlier assumption. If the user named implementation options, ask the independent user-owned prerequisites that would choose among them as separate numbered questions in the same round, not as one premature option question. When the user's answers resolve the current frontier and no newly unlocked independent user-owned choice remains, state the selected option and stop; do not open a new interview round or ask confirmation of a now-determined choice.
+End the round with the smallest answer format, then wait. If the task is already sufficiently specified, ask nothing. If uncertainty is cheap and reversible, choose the repository or platform default and proceed. Each reply reshapes the decision tree: record settled choices, recompute the frontier, and reopen a dependent choice only when new evidence contradicts an earlier assumption.
+
+When the user's answers resolve the current frontier and no newly unlocked independent user-owned choice remains, state the selected option and stop. Do not ask for confirmation of a now-determined choice. Return a compact execution capsule: selected option, governing constraints, strongest trade-off, assumptions, and any concrete validation still required.
## Resolve
-1. State the exact decision and constraints that distinguish acceptable options.
+1. State the exact decision and the constraints that distinguish acceptable options.
2. Check, in order: established project pattern, standard library, platform/framework, installed dependency, then mature maintained external implementation. Research external options only when the open choice cannot be resolved from local evidence; do not install or vendor an option merely to compare it.
3. Keep at most three viable options. Compare only material fit, correctness, compatibility, operational, maintenance, and migration differences.
4. Select the smallest option that fully satisfies current requirements. Do not create an abstraction, dependency, wrapper, or extension point without a present need.
Research only when local evidence cannot resolve a lasting choice or an external dependency is being considered. Prefer official and maintained sources; verify API fit, maintenance, license, and known constraints. Integrate a fitting mature surface instead of cloning it, and isolate only a demonstrated compatibility patch.
-Converge when the goal and success condition are clear, hard constraints and non-goals are known, high-impact choices are resolved or deliberately deferred, material contradictions are gone, and remaining uncertainty is cheap to reverse or assigned to a concrete validation step. Then state the selected option, rationale, trade-off, assumptions, and any deferred validation compactly before proceeding within the user's existing authorization.
+Converge when the goal and success condition are clear, hard constraints and non-goals are known, high-impact choices are resolved or deliberately deferred, material contradictions are gone, and remaining uncertainty is cheap to reverse or assigned to a concrete validation step.
+
+## Exit to Execution
+
+Decision resolution does not imply that execution needs another rigor profile. Return to the Core first. If safe execution is already known, continue Direct. If a different substantial blocker later requires Debugging or Implementation and this reference is already resident in the root, prefer an isolated follow-up when context savings exceed handoff cost rather than accumulating a second reasoning reference.
## Durable Decisions
diff --git a/references/implementation.md b/references/implementation.md
index 66634ad..64f9221 100644
--- a/references/implementation.md
+++ b/references/implementation.md
@@ -1,12 +1,16 @@
# Implementation
-Load this module only when a change must coordinate an unmapped contract or invariant, touches a material risk boundary where direct execution would be unsafe, or when the sufficient evidence for a risky change is itself unresolved. Material risk boundaries include security or permissions, irreversible side effects, persistence or migration, concurrency or transactions, and compatibility. Produce only the change map and evidence plan the task needs; this is not a mandatory coding stage.
+Load this module only when safe execution is blocked by an unknown contract or invariant, an unresolved material risk boundary, or insufficient evidence for a risky material claim. Implementation is an execution escalation profile, not a synonym for "coding" and not a mandatory stage after Debugging.
+
+Material risk boundaries include security or permissions, irreversible side effects, persistence or migration, concurrency or transactions, and compatibility. A risk-related noun does not trigger this profile when the governing boundary, affected surface, and sufficient check are already established; in that case stay Direct with the Core.
+
+Produce only the change map and evidence plan the blocker requires.
## Work Locally
- Identify the authoritative contract or invariant and the minimum producers, consumers, adapters, data, and checks that must move together.
-- Read only those paths and their material callers/dependencies; leave nearby cleanup opportunities and unrelated code alone.
-- For a risk boundary, identify the smallest authoritative point that owns the guarantee before editing. A single-file change can still belong here when the boundary is material.
+- Read only those paths and their material callers or dependencies; leave nearby cleanup opportunities and unrelated code alone.
+- For a risk boundary, identify the smallest authoritative point that owns the guarantee before editing. A single-file change can still need Implementation rigor when the unresolved boundary is material.
- Preserve public compatibility unless the requirement authorizes a break. When migration is required, choose one authoritative internal representation and keep compatibility at the narrowest boundary.
- Match project conventions and make the smallest coherent end-to-end diff.
@@ -22,8 +26,12 @@ Put validation once at the narrowest authoritative boundary. Add retries, fallba
Map each material claim or risk to the cheapest check that can falsify it, at the lowest sufficient level: diff inspection; direct exercise or render; compile/type/lint; an existing focused test; one new focused test; a boundary integration test; the full suite only for a broad surface or a required gate. Prefer project gates and existing focused tests; add one durable test only when direct evidence is insufficient, and do not expand into open-ended bug hunting or duplicate assertions across layers.
-For persistence or concurrency, exercise restart/rollback/race behavior when relevant and close files, connections, threads, and handles before cleanup. For compatibility, exercise the materially affected old and new callers. For a security or permission boundary, include one valid case and the smallest representative rejection cases, and verify rejection happens before side effects.
+For persistence or concurrency, exercise restart, rollback, or race behavior when relevant and close files, connections, threads, and handles before cleanup. For compatibility, exercise the materially affected old and new callers. For a security or permission boundary, include one valid case and the smallest representative rejection cases, and verify rejection happens before side effects.
Claim only what fresh evidence supports, re-checked after the final edit. "Too simple to verify", "it worked before this last edit", and "the tests probably still pass" are rationalizations, not evidence: run the cheapest sufficient check, or report the limitation and remaining uncertainty when environment, credentials, or services block it.
-If implementation exposes another event, return it to the router; do not automatically load another module.
+## Exit
+
+When the governing boundary, affected surface, and sufficient evidence are established, return to the Core and execute the smallest coherent change. Do not keep expanding rigor after its blocker is resolved.
+
+If work exposes a materially different unresolved choice or unexplained failure, return that blocker to the root. Do not automatically load another Practical Coding reference from this module; isolate substantial follow-up when adding another large reference would cost more context than a handoff.
diff --git a/references/navigation.md b/references/navigation.md
index a5bc98f..1d6e6bc 100644
--- a/references/navigation.md
+++ b/references/navigation.md
@@ -1,6 +1,6 @@
# Navigation
-Navigation is the detailed runtime retrieval procedure, not an Event Router branch. Load it only when broad code discovery or structural mapping is substantial enough that the short Retrieval Policy in `SKILL.md` is insufficient. Produce the smallest bounded context that answers the current need; do not tour the repository.
+Navigation is the detailed runtime retrieval procedure, not a Decision or execution-rigor state. Load it only when broad code discovery or structural mapping is substantial enough that the short Retrieval Policy in `SKILL.md` is insufficient. Produce the smallest bounded context that answers the current need; do not tour the repository.
Use already-available capabilities only. Do not install a backend, add a persistent integration, or change project configuration solely to obtain retrieval for the current task. An already-available backend may build or refresh its normal index when that is part of using the existing integration. Missing capabilities fall back to the next cheaper available path.
@@ -51,4 +51,4 @@ A clean index or coverage result means no recorded gap, not proof of semantic co
Navigation controls what enters model context; it does not create a new reasoning state. Returning from a search does not unload anything already read.
-For routine targeted lookup, do not load this reference at all. When another reasoning reference is already resident and broad mapping would create substantial search context, prefer a read-only isolated Navigation worker if the saved context clearly exceeds handoff cost. The worker returns exact paths, symbols, relationships, constraints, gaps, and evidence limits — not raw search or graph transcripts.
+For routine targeted lookup, do not load this reference at all. When Decision, Debugging, or Implementation is already resident and broad mapping would create substantial search context, prefer a read-only isolated Navigation worker if the saved context clearly exceeds handoff cost. The worker returns exact paths, symbols, relationships, constraints, gaps, and evidence limits — not raw search or graph transcripts.