Skip to content

check_boundary_types: the declared boundary was never compared to the design (#21) - #22

Merged
borisdev merged 3 commits into
mainfrom
boundary-types
Oct 7, 2026
Merged

borisdev merged 3 commits into
mainfrom
boundary-types

Conversation

@borisdev

@borisdev borisdev commented Oct 6, 2026

Copy link
Copy Markdown
Owner

Closes #21. Stacked on #20 — base is readme-diagram-companion, so merge that first and this
retargets to main.

The failure

A GraphSpec's input_type / output_type go straight to GraphBuilder. Nothing compared them
to the variables the START/END edges actually carry:

class Lying(Greeting):              # every edge in Greeting carries str
    input_type, output_type = int, int

Lying().coherence_check()                         # []
Lying().render(trim_only).run_sync(inputs="  a  b  ")   # 'Hello, a  b!' — a str

A dead field would have been the small version. _port_type reads those two fields as the
ORACLE for check_subgraphs — so a check that does run, whose whole job is "the child fits
the node"
, was comparing a parent node's contract against the child's unverified claim about
itself. Measured: a child declaring output_type=int while every edge in it carries str passed
against a parent node declaring an int output, and the graph returned 'HELLO'.

The rule existed and was carefully worded. What was missing is that nobody checked the thing
being compared against.

The check

Assignability, not identity, and the direction differs per side:

side legal because real example
input widened the graph receives input_type and the edge carries it onward parallel.py — list[int] into numbers: list
output narrowed the edge delivers and the graph promises stage10_no_basenode.py — report: str into output_type=object

The reverse of each is a finding, and test_the_reverse_of_each_is_NOT_legal is what makes the
two legal cases mean anything — a check accepting both directions on both sides would pass all
four designs and prove nothing.

  • delivers is what's compared at END, not carries — a TransformEdgeSpec reshapes on the wire.
  • Undecidable (list[int] vs list[str]) is one aggregated NOT CHECKED line, non-blocking.
  • The default type(None) is a claim, not an absence — it reaches the engine as the graph's
    real signature, so an undeclared boundary with a str crossing it is reported.

Two helper fixes fell out, both real

  • _produces called list[int] vs list[int] undecidable — list[int] is list[int] is
    False — and never compared a parameterised alias to a bare type, although list[int] plainly
    is a list. Both now decide. This can only turn an undecidable into a verdict, never
    manufacture a finding, and it is why the new check is clean on all 14 designs in examples/
    instead of printing a permanent NOT CHECKED on parallel.py. check_variable_types reads
    the same helper and gains the same decidability.
  • _type_name rendered list[int] and list identically. Its docstring said generic
    aliases have no __name__; since 3.10 they do, and it is the bare origin — so the bug was a
    wrong name, not a missing one, and a finding comparing those two read as a complaint that
    list is not list. Found while writing this check's message, which compares exactly that pair.

Two decisions recorded rather than taken

  • about="", not about="input_type". A port name would be a fifth kind of value in a field
    test_every_finding_names_something_the_caller_can_look_up resolves. That is a vocabulary
    change — proposed on the issue, not slipped in. The side is in the sentence.
  • _port_type's refusal still says "a subgraph binding needs ONE", which will stop being
    the only caller when the step-battle work lands (Three states, not two: when a stage should become a nested graph #12). Reworded on request; not unilaterally.

Against this repo's 7-for-7 pattern

Every Copilot finding across #8, #18 and #19 was a check narrower than its claim, and twice the
defective guard shipped in the same commit as the thing it guards
. A new check's own test file
is exactly that shape, so two of the twelve tests are written against it:

Also

13 rules now, 8 needing nothing implemented. Generated table regenerated; the two prose counts in
the README updated (test_no_prose_anywhere_states_a_rule_count_the_generator_disagrees_with
covers them).

294 tests green locally — ⛔ no CI in this repo, so that is one local run.

Boris Dev added 2 commits October 6, 2026 21:24
The before/after is the pitch — the same graph with nothing implemented, then with
two implementations bound — and only the second half was ever drawn. `diagram()`
was described in prose and never shown.

The check that should have caught this could not. `test_the_readme_mermaid_block_is_
the_diagram_the_code_emits` asserted ONE block by name, at a time when the README
had exactly one: a check narrower than its claim, and blind to a missing picture
because the blocks that are present all pass. Replaced with a set match over every
mermaid block in every markdown file, in both directions — a block with no generator
fails, and a registered generator with no block fails too. Verified both go red.

282 tests green locally. No CI in this repo, so that is one local run.
… design (#21)

A GraphSpec's `input_type` / `output_type` go straight to `GraphBuilder`, and
nothing checked them against the variables the START/END edges carry:

    class Lying(Greeting):              # every edge in Greeting carries str
        input_type, output_type = int, int

    coherence_check()  -> []            run_sync('  a  b  ') -> 'Hello, a  b!'

A dead field would have been the small version. `_port_type` reads those two
fields as the ORACLE for `check_subgraphs`, so a check that DOES run — whose
whole job is "the child fits the node" — compared a parent node's contract
against the child's unverified claim about itself. Measured: a child declaring
`output_type=int` with every edge carrying `str` passed against a parent node
declaring an int output, and the graph returned 'HELLO'.

Assignability, not identity, and the direction differs per side: widening is
legal on the way IN (parallel.py's `list[int]` into `numbers: list`), narrowing
on the way OUT (stage10's `report: str` into `output_type=object`). The reverse
of each is a finding, and test_the_reverse_of_each_is_NOT_legal is why the two
legal cases prove anything.

Two helper fixes fell out, both real:

  _produces   called `list[int]` vs `list[int]` undecidable (`is` is False on
              aliases) and never compared a parameterised alias to a bare type.
              Both now decide — which can only turn an undecidable into a
              verdict, never manufacture a finding. This is why the new check is
              clean on all 14 designs in examples/ rather than printing a
              permanent NOT CHECKED on parallel.py.
  _type_name  rendered `list[int]` and `list` IDENTICALLY. Its docstring said
              generic aliases have no `__name__`; since 3.10 they do, and it is
              the bare origin — a wrong name, not a missing one.

`about=""`, not `about="input_type"`: a port name would be a fifth kind of value
in a field `test_every_finding_names_something_the_caller_can_look_up` resolves.
That is a vocabulary change — proposed on the issue, not slipped in here.

Against the handoff's 7-for-7 pattern (every Copilot finding was a check
narrower than its claim, twice shipped in the same commit as the thing it
guards): the broad test runs over every GraphSpec in examples/, and one test
reproduces #21 verbatim on a SHIPPED example rather than a fixture this module
authored.

13 rules now, 8 needing nothing implemented. 294 tests green locally — no CI in
this repo, so that is one local run.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Type-origin handling can crash or reject compatible declarations, and example calibration silently omits designs.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
What changed in this PR

Adds boundary-type validation so graph declarations are checked against START/END edges before rendering or use as subgraphs.

Changes:

  • Adds and exports check_boundary_types.
  • Improves generic-type comparison and diagnostic names.
  • Adds regression tests and updates rule documentation.
File Description
workflow_workbench/​graph_spec.py Runs boundary validation during coherence checks.
workflow_workbench/​checks.py Implements validation and updates type helpers.
workflow_workbench/​__init__.py Exports the new check.
tests/​test_boundary_types.py Tests boundaries, subgraphs, and example compatibility.
README.md Updates rule counts and generated reference.
CHANGELOG.md Documents validation and helper changes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/test_boundary_types.py
Comment thread workflow_workbench/checks.py Outdated
**1. `_produces` crashed on a typing wrapper, through `coherence_check()`.**
Taken, and it is worse than the review said — it reported "returns False"; it
actually raises. `get_origin` is not always a runtime class: `typing.Literal`
for `Literal['ok']`, `typing.Annotated` for `Annotated[int, 'tag']`, and
`issubclass` on either gives `TypeError: issubclass() arg 1 must be a class`.
Measured end to end before the fix:

    async def decide(ctx) -> Literal["ok", "no"]: ...
    coherence_check(strategy)  ->  ⛔ RAISED TypeError

`coherence_check()` is documented "Never raises", and a crash is not a
conservative failure — every other finding in that sweep is lost with it. The
alias branch now requires `isinstance(origin, type)`. Those wrappers stay
UNDECIDABLE rather than being unwrapped: unwrapping `Annotated` is a real
improvement that nothing has needed, and project.md says add the guard when you
have the failing case.

**2. The calibration test covered 14 of 15 designs and could have covered 0.**
Taken, and this is the eighth instance of the pattern the test's own docstring
is written against — a check narrower than its claim, in the check written to
avoid that. Two independent holes:

    examples/local/     no __init__.py, so pkgutil.walk_packages never descended
                        into it. examples.local.extraction.Extraction was never
                        checked.
    except: continue    a module that failed to import was silently dropped, so
                        the sweep could go green having checked nothing.

The `>= 8` floor let either pass. Discovery is now by FILE, recursive, with no
try — a broken example is a failure. 15 designs found, all clean, and
`test_discovery_reaches_every_example_FILE_not_every_example_package` names
`examples.local.extraction` so the hole cannot reopen quietly.
`.claude/rules/checks.md`: a missing input must never read as a pass.

Regression tests added for `Literal` and `Annotated`, including that
`coherence_check()` reports NOT CHECKED rather than raising.

296 tests green locally — no CI in this repo, so that is one local run.
@borisdev

borisdev commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

Both findings taken, and one was worse than labelled

1. _produces on a non-class origin — a raise, not a wrong answer

The review says _produces(Annotated[int, 'tag'], int) "returns False". Measured: it
raises, and so does Literal.

get_origin(Literal['ok'])        -> typing.Literal
get_origin(Annotated[int,'tag']) -> typing.Annotated
issubclass(either, str)          -> TypeError: issubclass() arg 1 must be a class

And the blast radius is the part that matters — it reaches the public method:

async def decide(ctx) -> Literal["ok", "no"]: ...
G().coherence_check(strategy)      # ⛔ RAISED TypeError   (before)
                                   # NOT CHECKED — … not decidable   (after)

coherence_check() is documented "Never raises." A crash is not a conservative failure: every
other finding in that sweep is lost with it, so a design with one Literal annotation would have
reported nothing at all rather than reporting one gap.

The alias branch now requires isinstance(origin, type). Those wrappers stay undecidable
rather than being unwrapped
— unwrapping Annotated is a real improvement that nothing has
needed yet, and .claude/rules/project.md says add the guard when you have the failing case, not
when you foresee one. Regression tests for both, plus the end-to-end "does not raise" case.

2. The calibration test covered 14 of 15 designs, and could have covered 0

Taken, and this is the eighth instance of the pattern the test's own docstring is written
against
— a check narrower than its claim, in the check written to avoid that. Two independent
holes, and the review found both:

examples/local/     no __init__.py -> pkgutil.walk_packages never descended.
                    examples.local.extraction.Extraction was NEVER checked.
except: continue    an import failure was silently dropped, so the sweep could
                    go green having checked nothing at all.

The >= 8 floor let either pass, which is the bit I should have caught: I wrote a vacuity guard
and set it below the real count.

Discovery is now by file, recursive, with no try — a broken example is a failure.
15 designs found, all clean. And test_discovery_reaches_every_example_FILE_not_every_ example_package names examples.local.extraction explicitly, so the hole cannot reopen quietly.

.claude/rules/checks.md: a missing input must never read as a pass.


296 tests green locally. ⚠️ The review on record predates this push — happy to request another.

Copilot AI balanced review requested due to automatic review settings October 6, 2026 22:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The generic-origin guard incorrectly rejects Annotated types, contradicting the new regression assertion.

Review effort: Balanced
Findings: None

Resolved since last review (2)

@borisdev
borisdev deleted the branch main October 7, 2026 00:29
@borisdev borisdev closed this Oct 7, 2026
@borisdev borisdev reopened this Oct 7, 2026
@borisdev
borisdev changed the base branch from readme-diagram-companion to main October 7, 2026 00:30
@borisdev
borisdev merged commit e0c6669 into main Oct 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A GraphSpec's declared input_type/output_type is never checked — and check_subgraphs trusts it

2 participants