Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .claude/skills/shepherd/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,11 @@ explicit name (`opus`, `sonnet`, `haiku`) is used verbatim. Resolve `"auto"` as:
`final_reviewer` → `opus` (`sonnet` for `trivial`/`small`). `sonnet` is the floor for review —
never `haiku`. The panel never resolves entirely onto the implementer's model: keep at least one
reviewer — and one final reviewer when the panel has any — on a different model, because a judge
sharing the generator's model favors its output (self-preference bias). Pre-gate stages (verify, explorer, architect, success_criteria) resolve `"auto"`
sharing the generator's model favors its output (self-preference bias). On a fix pass —
iteration ≥2 with only `minor`/`nit` findings open and no design-level change — an auto-resolved
implementer drops one tier (`sonnet` → `haiku`); a finding that survives its fix round bumps it
back. An explicit config model never drops. Reviewer models never change mid-loop — the PASS
that ends the loop must not come from a weaker judge than the FAIL that opened it. Pre-gate stages (verify, explorer, architect, success_criteria) resolve `"auto"`
at dispatch time from this table. At step 4 (Design gate), record all picks in
`_panel.json`: the pre-gate ones as the record of what ran, the post-gate ones (implementer,
reviewers, final reviewers, fulfillment) for the human to edit before approving.
Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/shepherd/config.default.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
],
"final_reviewers": [
{ "use": "thermonuclear", "model": "auto" },
{ "use": "ponytail-review", "model": "auto" }
{ "use": "ponytail-review", "model": "sonnet" }
]
},
"oracle": {
Expand Down
11 changes: 10 additions & 1 deletion docs/shepherd-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ Core/shared or public-contract changes are `medium` at minimum regardless of lin
"reviewers": [{ "use": "staff-review", "model": "auto" }],
"final_reviewers": [
{ "use": "thermonuclear", "model": "auto" },
{ "use": "ponytail-review", "model": "auto" }
{ "use": "ponytail-review", "model": "sonnet" }
]
},
"oracle": { "commands": [] },
Expand All @@ -114,6 +114,15 @@ Single stages may be **model-only** (`{ "model": "..." }` with no `use`): built-
that model. All `"auto"` picks are resolved and shown at the design gate, where you can adjust any
before approving.

Auto is also adaptive across the loop: on a fix pass (iteration ≥2 with only minor/nit findings
open and no design-level change) an auto-resolved implementer drops one tier, bumping back if a
finding survives its round; explicitly pinned models never drop, and reviewer models never change
mid-loop, so the verdict that ends the loop is never issued by a weaker judge than the one that
raised the findings. The shipped default pins `ponytail-review` to `sonnet`: a one-line
deletion/leanness lens doesn't earn `opus`, while `sonnet` keeps the review floor — under
zero-findings convergence a hallucinated nit costs a full fix round, so review never drops to
`haiku`.

`oracle.commands` should be finite and non-mutating. Good defaults for JS/TS repos are:

```json
Expand Down
7 changes: 7 additions & 0 deletions tests/test_config_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,10 @@ def test_reviewer_without_use_reported():
bad["stages"]["reviewers"] = [{"model": "sonnet"}]
errs = validate(bad, REGISTRY)
assert any("reviewers" in e and "use" in e for e in errs)


def test_default_config_pins_ponytail_to_sonnet():
# A one-line deletion lens doesn't earn opus; sonnet stays the review floor because
# under zero-findings convergence a hallucinated nit costs a full fix round.
finals = {e["use"]: e.get("model") for e in load_json(CONFIG)["stages"]["final_reviewers"]}
assert finals["ponytail-review"] == "sonnet"
11 changes: 11 additions & 0 deletions tests/test_orchestrator_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -364,3 +364,14 @@ def test_review_panel_never_matches_implementer_model_entirely():

def test_pr_body_scales_with_diff():
assert "mechanism and known limitations" in ORCH


def test_implementer_fix_pass_is_adaptive_but_judges_are_stable():
# Iteration >=2 fix rounds are transcription-like, so auto drops the implementer a tier;
# judges keep verdict continuity — the converging PASS never comes from a weaker model.
assert "On a fix pass" in ORCH
assert "drops one tier" in ORCH
assert "bumps it" in ORCH
assert "An explicit config model never drops" in ORCH
assert "never change mid-loop" in ORCH
assert "weaker judge" in ORCH
Loading