Repository navigation
check_boundary_types: the declared boundary was never compared to the design (#21) - #22
Conversation
The before/after is the pitch — the same graph with nothing implemented, then with two implementations bound — and only the second half was ever drawn. `diagram()` was described in prose and never shown. The check that should have caught this could not. `test_the_readme_mermaid_block_is_ the_diagram_the_code_emits` asserted ONE block by name, at a time when the README had exactly one: a check narrower than its claim, and blind to a missing picture because the blocks that are present all pass. Replaced with a set match over every mermaid block in every markdown file, in both directions — a block with no generator fails, and a registered generator with no block fails too. Verified both go red. 282 tests green locally. No CI in this repo, so that is one local run.
… design (#21) A GraphSpec's `input_type` / `output_type` go straight to `GraphBuilder`, and nothing checked them against the variables the START/END edges carry: class Lying(Greeting): # every edge in Greeting carries str input_type, output_type = int, int coherence_check() -> [] run_sync(' a b ') -> 'Hello, a b!' A dead field would have been the small version. `_port_type` reads those two fields as the ORACLE for `check_subgraphs`, so a check that DOES run — whose whole job is "the child fits the node" — compared a parent node's contract against the child's unverified claim about itself. Measured: a child declaring `output_type=int` with every edge carrying `str` passed against a parent node declaring an int output, and the graph returned 'HELLO'. Assignability, not identity, and the direction differs per side: widening is legal on the way IN (parallel.py's `list[int]` into `numbers: list`), narrowing on the way OUT (stage10's `report: str` into `output_type=object`). The reverse of each is a finding, and test_the_reverse_of_each_is_NOT_legal is why the two legal cases prove anything. Two helper fixes fell out, both real: _produces called `list[int]` vs `list[int]` undecidable (`is` is False on aliases) and never compared a parameterised alias to a bare type. Both now decide — which can only turn an undecidable into a verdict, never manufacture a finding. This is why the new check is clean on all 14 designs in examples/ rather than printing a permanent NOT CHECKED on parallel.py. _type_name rendered `list[int]` and `list` IDENTICALLY. Its docstring said generic aliases have no `__name__`; since 3.10 they do, and it is the bare origin — a wrong name, not a missing one. `about=""`, not `about="input_type"`: a port name would be a fifth kind of value in a field `test_every_finding_names_something_the_caller_can_look_up` resolves. That is a vocabulary change — proposed on the issue, not slipped in here. Against the handoff's 7-for-7 pattern (every Copilot finding was a check narrower than its claim, twice shipped in the same commit as the thing it guards): the broad test runs over every GraphSpec in examples/, and one test reproduces #21 verbatim on a SHIPPED example rather than a fixture this module authored. 13 rules now, 8 needing nothing implemented. 294 tests green locally — no CI in this repo, so that is one local run.
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Type-origin handling can crash or reject compatible declarations, and example calibration silently omits designs.
Review effort: Balanced
Findings: 2
Open (2)
What changed in this PR
Adds boundary-type validation so graph declarations are checked against START/END edges before rendering or use as subgraphs.
Changes:
- Adds and exports
check_boundary_types. - Improves generic-type comparison and diagnostic names.
- Adds regression tests and updates rule documentation.
| File | Description |
|---|---|
| workflow_workbench/graph_spec.py | Runs boundary validation during coherence checks. |
| workflow_workbench/checks.py | Implements validation and updates type helpers. |
| workflow_workbench/__init__.py | Exports the new check. |
| tests/test_boundary_types.py | Tests boundaries, subgraphs, and example compatibility. |
| README.md | Updates rule counts and generated reference. |
| CHANGELOG.md | Documents validation and helper changes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
**1. `_produces` crashed on a typing wrapper, through `coherence_check()`.**
Taken, and it is worse than the review said — it reported "returns False"; it
actually raises. `get_origin` is not always a runtime class: `typing.Literal`
for `Literal['ok']`, `typing.Annotated` for `Annotated[int, 'tag']`, and
`issubclass` on either gives `TypeError: issubclass() arg 1 must be a class`.
Measured end to end before the fix:
async def decide(ctx) -> Literal["ok", "no"]: ...
coherence_check(strategy) -> ⛔ RAISED TypeError
`coherence_check()` is documented "Never raises", and a crash is not a
conservative failure — every other finding in that sweep is lost with it. The
alias branch now requires `isinstance(origin, type)`. Those wrappers stay
UNDECIDABLE rather than being unwrapped: unwrapping `Annotated` is a real
improvement that nothing has needed, and project.md says add the guard when you
have the failing case.
**2. The calibration test covered 14 of 15 designs and could have covered 0.**
Taken, and this is the eighth instance of the pattern the test's own docstring
is written against — a check narrower than its claim, in the check written to
avoid that. Two independent holes:
examples/local/ no __init__.py, so pkgutil.walk_packages never descended
into it. examples.local.extraction.Extraction was never
checked.
except: continue a module that failed to import was silently dropped, so
the sweep could go green having checked nothing.
The `>= 8` floor let either pass. Discovery is now by FILE, recursive, with no
try — a broken example is a failure. 15 designs found, all clean, and
`test_discovery_reaches_every_example_FILE_not_every_example_package` names
`examples.local.extraction` so the hole cannot reopen quietly.
`.claude/rules/checks.md`: a missing input must never read as a pass.
Regression tests added for `Literal` and `Annotated`, including that
`coherence_check()` reports NOT CHECKED rather than raising.
296 tests green locally — no CI in this repo, so that is one local run.
Both findings taken, and one was worse than labelled1.
|

Closes #21. Stacked on #20 — base is
readme-diagram-companion, so merge that first and thisretargets to
main.The failure
A
GraphSpec'sinput_type/output_typego straight toGraphBuilder. Nothing compared themto the variables the START/END edges actually carry:
A dead field would have been the small version.
_port_typereads those two fields as theORACLE for
check_subgraphs— so a check that does run, whose whole job is "the child fitsthe node", was comparing a parent node's contract against the child's unverified claim about
itself. Measured: a child declaring
output_type=intwhile every edge in it carriesstrpassedagainst a parent node declaring an int output, and the graph returned
'HELLO'.The rule existed and was carefully worded. What was missing is that nobody checked the thing
being compared against.
The check
Assignability, not identity, and the direction differs per side:
input_typeand the edge carries it onwardparallel.py—list[int]intonumbers: liststage10_no_basenode.py—report: strintooutput_type=objectThe reverse of each is a finding, and
test_the_reverse_of_each_is_NOT_legalis what makes thetwo legal cases mean anything — a check accepting both directions on both sides would pass all
four designs and prove nothing.
deliversis what's compared at END, notcarries— aTransformEdgeSpecreshapes on the wire.list[int]vslist[str]) is one aggregatedNOT CHECKEDline, non-blocking.type(None)is a claim, not an absence — it reaches the engine as the graph'sreal signature, so an undeclared boundary with a
strcrossing it is reported.Two helper fixes fell out, both real
_producescalledlist[int]vslist[int]undecidable —list[int] is list[int]isFalse— and never compared a parameterised alias to a bare type, althoughlist[int]plainlyis a
list. Both now decide. This can only turn an undecidable into a verdict, nevermanufacture a finding, and it is why the new check is clean on all 14 designs in
examples/instead of printing a permanent
NOT CHECKEDonparallel.py.check_variable_typesreadsthe same helper and gains the same decidability.
_type_namerenderedlist[int]andlistidentically. Its docstring said genericaliases have no
__name__; since 3.10 they do, and it is the bare origin — so the bug was awrong name, not a missing one, and a finding comparing those two read as a complaint that
listis notlist. Found while writing this check's message, which compares exactly that pair.Two decisions recorded rather than taken
about="", notabout="input_type". A port name would be a fifth kind of value in a fieldtest_every_finding_names_something_the_caller_can_look_upresolves. That is a vocabularychange — proposed on the issue, not slipped in. The side is in the sentence.
_port_type's refusal still says "a subgraph binding needs ONE", which will stop beingthe only caller when the step-battle work lands (Three states, not two: when a stage should become a nested graph #12). Reworded on request; not unilaterally.
Against this repo's 7-for-7 pattern
Every Copilot finding across #8, #18 and #19 was a check narrower than its claim, and twice the
defective guard shipped in the same commit as the thing it guards. A new check's own test file
is exactly that shape, so two of the twelve tests are written against it:
test_no_design_in_the_repo_is_flagged_or_silently_skipped— runs over everyGraphSpecinexamples/(14 found, 0 import-skipped), not over the fixtures this module authoredtest_the_exact_repro_from_the_issue_on_a_REAL_example— subclasses a shipped example andreproduces A GraphSpec's declared input_type/output_type is never checked — and check_subgraphs trusts it #21 verbatim, and asserts the unmodified example stays clean so the lie proves something
Also
13 rules now, 8 needing nothing implemented. Generated table regenerated; the two prose counts in
the README updated (
test_no_prose_anywhere_states_a_rule_count_the_generator_disagrees_withcovers them).
294 tests green locally — ⛔ no CI in this repo, so that is one local run.