Both reference tables in the README, generated from the package's own docstrings

README.md

@@ -1,13 +1,38 @@
 # workflow-workbench
 
-Workflow Workbench is a high-level wrapper around Pydantic Graph Builder for developing workflows
-with an AI coding agent.
+**Let an AI coding agent build a workflow unsupervised and it produces code that works and is
+incoherent.** Not broken — that you would notice. Incoherent: a fan-out whose results are
+silently dropped, two wires crossed between values of the same type, a stage nobody implemented.
+It runs, it returns something of the right shape, and nothing downstream can tell.
 
-Specify the workflow and its data contracts, inspect its diagram, then have the agent implement
-the steps. Bind alternative implementations as strategies and compare them through simple
-evaluation battles.
+Workflow Workbench is a declaration layer over Pydantic Graph Builder. You write the workflow's
+shape and its data contracts as **data**, before any step exists — which is what makes that class
+of defect findable:
+
+```python
+spec.coherence_check()      # 11 well-formedness rules, 7 of them with nothing implemented
+spec.diagram()              # a picture of the same declaration
+spec.render(strategy)       # refuses outright if anything blocks
+```
+
+So the agent gets an acceptance test it cannot talk its way past, and you get a drawing of the
+design before you read a line of its code.
+
+Four problems, and the same declaration answers all four:
+
+| | |
+|---|---|
+| **An agent's output works and is incoherent.** Each piece is locally fine; the whole does not add up. | `coherence_check()` — 11 well-formedness rules, 7 needing nothing implemented |
+| **A reasoning strategy cannot be asserted correct — only compared.** There is no right answer to diff against, so "better" is an empirical question. | `eval_battle()` — same cases, same evaluators, plus a replicate arm as the noise floor |
+| **Complexity grows unless pieces are reused.** Two arms that differ in one stage should say so, not be two files. | the data language: declare a role once, bind it many ways; `SubgraphBinding` reuses a whole child design as one node |
+| **You cannot see what you built.** | `diagram()` and `diff_diagram()`, from the declaration alone |
+
+On that last one, honestly: Pydantic Graph **can** emit mermaid — `build_mermaid_graph` in
+`graph_builder.py`. Two differences, not a long list. It takes a BUILT graph's internals, so every
+implementation must exist first; ours reads the declaration, so the picture arrives before the
+code. And ours can draw **two strategies at once**, greying what they share and highlighting what
+differs, which is a question about a comparison rather than about a graph.
 
-The specification keeps the workflow understandable and gives the agent explicit constraints.
 Pydantic Graph executes the workflow; Pydantic Evals evaluates its results.
 
 Built on [Pydantic Graph](https://ai.pydantic.dev/graph/) and
@@ -108,6 +133,46 @@ Stages 2 and 5 are what a specification buys, and neither needs a second strateg
 implementation per step still gets a drawing before it is written and a refusal when one is
 missed.
 
+## What `coherence_check()` enforces
+
+<!-- rules:start -->
+**11 rules.** `coherence_check()` returns one finding per violation and an empty list for a clean design; `render()` refuses on any finding that blocks.
+
+**7 need no implementations at all** — runnable the moment `nodes` and `edges` are written.
+
+| check | rule |
+|---|---|
+| `check_names` | Node names must be unique — `render()` uses them as graph node ids. |
+| `check_reachable` | Every node reachable from START, and every node able to reach END. |
+| `check_variables` | Per edge: the variable it carries must be an output of its source and an input of its target. |
+| `check_step_arity` | A step body receives exactly ONE value, so a node cannot consume two inputs at once. |
+| `check_decisions` | `when` appears exactly on the edges leaving a decision, and nowhere else. |
+| `check_transform_edges` | A transform edge is fixed (`apply=`) or a variation point (bound) — exactly one. |
+| `check_fan_out_rejoins` | Everything a fan-out produces must reach a join before it reaches END. |
+
+**4 more once a strategy exists**, checking the implementations against the roles they fill.
+
+| check | rule |
+|---|---|
+| `check_bindings` | The strategy binds exactly the declared VARIATION POINTS — no missing, no extra. |
+| `check_implementations` | Each bound CALLABLE is callable and takes exactly one positional argument (`ctx`). |
+| `check_subgraphs` | Every child design used as a node implementation fits the node it is bound to. |
+| `check_variable_types` | Each implementation returns the type its role is declared to produce. |
+<!-- rules:end -->
+
+Every one of these exists because it caught something that otherwise **ran and returned a
+plausible answer**. Each check's docstring in [`checks.py`](workflow_workbench/checks.py) carries
+the measured case that produced it.
+
+**These are structural checks, not a proof of correctness.** A step that returns its input
+untouched satisfies every rule above and still does nothing — that is the boundary between what a
+specification checks and what an evaluation measures, which is why `eval_battle` exists.
+
+A finding is a `CoherenceFinding`: a `str` subclass, so it reads as the sentence it is, carrying
+`check`, `about` and `blocking` so an agent can branch on structure rather than parse English. A
+`NOT CHECKED — …` finding is a **stated gap**, not a pass, and does not block `render()`.
+
+
 ## The same example, in five stages
 
 ### 1. Declare the nodes, the named values, and the edges
@@ -134,6 +199,53 @@ checker can catch `compose` being wired to the wrong one when there is only one
 A name can. Edge fields are keyword-only and `carries` is required — four interchangeable-looking
 slots are one transposition away from a graph that is wrong and runs.
 
+#### The data language
+
+<!-- language:start -->
+A design is **data** — tuples of these, in a class body. Nothing executes, which is what lets `coherence_check()` and `diagram()` read it before a single step is written.
+
+**Values** — What flows. Named, so a mis-wiring is visible when the types are identical.
+
+| | |
+|---|---|
+| `VariableSpec` | A named, typed value that may flow along an edge. |
+
+**Boxes** — Every box the design declares. Only a step takes an implementation.
+
+| | |
+|---|---|
+| `StepSpec` | A semantic role with a typed contract. Deliberately implementation-free. |
+| `JoinSpec` | The one thing that can combine several arrivals into one value. |
+| `DecisionSpec` | A router. Sends the value down one branch, chosen by its TYPE. |
+| `NodeSpec` | `StepSpec` \| `JoinSpec` \| `DecisionSpec` |
+
+**Wires** — How values move. The kind of edge is the kind of movement.
+
+| | |
+|---|---|
+| `EdgeSpec` | One wire: `source -> target`, carrying `carries`. |
+| `MapEdgeSpec` | Fan out: `carries` is a collection, and the target runs ONCE PER `delivers`. |
+| `TransformEdgeSpec` | A cheap SYNCHRONOUS reshape that happens ON THE WIRE, creating no node. |
+
+**Endpoints** — The graph's own boundary, declared like anything else.
+
+| | |
+|---|---|
+| `START` | The graph's entry. |
+| `END` | The graph's exit. |
+
+**The design, and what fills it** — One design, many competing sets of implementations.
+
+| | |
+|---|---|
+| `GraphSpec` | Subclass it, declare `nodes` and `edges`. That is the whole interface. |
+| `StrategySpec` | A complete Bindable -> implementation mapping. One competitor. |
+| `SubgraphBinding` | A whole child design — `GraphSpec` + `StrategySpec` — used as ONE node's implementation. |
+| `Bindable` | `StepSpec` \| `TransformEdgeSpec` |
+
+The two unions are annotations, not classes you instantiate — calling either one raises `TypeError`. They exist so a signature can say *any declared box*, or *anything a strategy must bind*, and have it type-check.
+<!-- language:end -->
+
 ### 2. Check it and draw it, before implementing anything
 
 ```python

tests/test_reference.py

@@ -0,0 +1,136 @@
+"""The README's two tables are DERIVED, and this is what stops them drifting.
+
+Same shape as `test_parity.py`, for the same reason: `.claude/rules/spec-as-code.md` — an
+unchecked convention drifts back within a month, so the check IS the rule.
+
+⚠️ Three halves, and only together do they mean anything:
+  · the generated blocks match the generator
+  · the generator sees EVERY export — a table perfectly in sync with a generator that reads nine
+    of eleven checks is perfectly in sync and wrong
+  · every cell is the thing's own words, so a table cannot say what the code does not
+"""
+from __future__ import annotations
+
+import inspect
+import subprocess
+import sys
+from pathlib import Path
+
+import workflow_workbench as ww
+import workflow_workbench.checks as checks
+from workflow_workbench.reference import (
+    GROUPS,
+    NOT_THE_LANGUAGE,
+    language_markdown,
+    rules,
+    rules_markdown,
+    vocabulary,
+)
+
+ROOT = Path(__file__).resolve().parent.parent
+
+
+def test_both_readme_blocks_are_regenerated_from_reference_py() -> None:
+    """Edit a docstring, regenerate, commit both. Editing the README alone turns this red."""
+    proc = subprocess.run([sys.executable, "-m", "workflow_workbench.reference", "--check"],
+                          cwd=ROOT, capture_output=True, text=True, timeout=120)
+    assert proc.returncode == 0, proc.stdout + proc.stderr
+
+
+def test_every_export_is_either_in_the_language_or_explicitly_excluded() -> None:
+    """⛔ THE COMPLETENESS GUARD, and the one that matters most.
+
+    A new type added to `__all__` and forgotten is the silent failure here: the table stays in
+    sync with the generator and quietly under-reports the language. Excluding something is fine —
+    saying nothing about it is not, which is why `NOT_THE_LANGUAGE` carries a reason per name.
+    """
+    grouped = {w.name for _, _, words in vocabulary() for w in words}
+    assert grouped, "the vocabulary is empty — every assertion here would pass vacuously"
+    unaccounted = set(ww.__all__) - grouped - set(NOT_THE_LANGUAGE)
+    assert not unaccounted, (
+        f"exported but neither in the table nor explained: {sorted(unaccounted)}. Add it to "
+        f"GROUPS, or to NOT_THE_LANGUAGE with the reason a reader does not need it.")
+    assert not (grouped & set(NOT_THE_LANGUAGE)), "a word cannot be both shown and excluded"
+
+
+def test_the_exclusions_all_still_exist() -> None:
+    """A stale exclusion is worse than none: it names something nobody will notice is gone, and
+    the guard above goes quiet on whatever takes that name next. Copied from
+    `test_parity.py`'s exemption check, which exists for exactly this."""
+    missing = [n for n in NOT_THE_LANGUAGE if n not in ww.__all__]
+    assert not missing, f"excluded from the table but no longer exported: {missing}"
+
+
+def test_every_check_in_the_module_reaches_the_rules_table() -> None:
+    public = {n for n in dir(checks)
+              if n.startswith("check_") and callable(getattr(checks, n))}
+    assert public, "found no checks at all"
+    assert public == {r.check for r in rules()}, (
+        "a check_* function is missing from checks.__all__, so the table cannot see it")
+
+
+def test_no_cell_is_invented_prose() -> None:
+    """⛔ The rule that makes both tables trustworthy. If a cell could be hand-written, the table
+    could say something the code does not do — the whole failure a derived document removes."""
+    for r in rules():
+        first = inspect.getdoc(getattr(checks, r.check)).split("\n")[0].strip()
+        assert r.rule == first, f"{r.check}: table text is not its docstring's first line"
+        assert r.rule in rules_markdown(), f"{r.check}: its rule never reaches the table"
+
+    for _, _, words in vocabulary():
+        for w in words:
+            if w.is_union:
+                continue
+            first = inspect.getdoc(getattr(ww, w.name)).split("\n")[0].strip()
+            assert w.what == first, f"{w.name}: table text is not its docstring's first line"
+
+
+def test_every_first_docstring_line_stands_alone() -> None:
+    """⚠️ The invariant the table rests on, and it is not free — `_Start`'s first line used to
+    end mid-sentence on 'so `mypy` can narrow a', which the table would have printed verbatim.
+    A docstring whose first line wraps is a source bug once anything lifts it."""
+    for _, _, words in vocabulary():
+        for w in words:
+            if w.is_union:
+                continue
+            assert w.what.endswith((".", "!")), (
+                f"{w.name}: first docstring line does not end a sentence — it wraps, and the "
+                f"table would print the fragment. Put the summary on one line.")
+
+
+def test_a_union_renders_its_members_and_not_pythons_docstring() -> None:
+    """`inspect.getdoc` on a `X | Y` alias returns 'Represent a PEP 604 union type', which says
+    nothing about this library. And the members must be PIPE-ESCAPED or markdown reads them as
+    column separators — a bug invisible in the source string and visible only in the table."""
+    rendered = {w.name: w.what for _, _, words in vocabulary() for w in words if w.is_union}
+    assert set(rendered) == {"NodeSpec", "Bindable"}, "the two unions must both be shown"
+    assert "PEP 604" not in language_markdown()
+    for name, cell in rendered.items():
+        assert r"\|" in cell, f"{name}: unescaped pipe would break the markdown table"
+        assert f"| `{name}` | {cell} |" in language_markdown()
+
+
+def test_the_rules_table_counts_come_from_the_checks() -> None:
+    rs = rules()
+    md = rules_markdown()
+    assert f"**{len(rs)} rules.**" in md
+    assert f"**{sum(1 for r in rs if not r.needs_strategy)} need no implementations" in md
+
+
+def test_needs_strategy_is_derived_and_matches_what_actually_runs() -> None:
+    """⚠️ `check_transform_edges` takes `strategy` and tolerates `None`, so it belongs with the
+    design-only group. Getting that wrong would tell a reader a design cannot be checked until it
+    is implemented — the opposite of the pitch."""
+    design_only = {r.check for r in rules() if not r.needs_strategy}
+    assert "check_transform_edges" in design_only
+    assert {"check_bindings", "check_implementations", "check_variable_types",
+            "check_subgraphs"} == {r.check for r in rules() if r.needs_strategy}
+
+
+def test_the_authored_part_is_only_the_grouping() -> None:
+    """GROUPS carries names and headings. If it ever carried a DESCRIPTION, the table would be
+    half source and half derived — which is the state this whole module exists to avoid."""
+    for title, blurb, names in GROUPS:
+        assert isinstance(title, str) and isinstance(blurb, str)
+        assert all(isinstance(n, str) for n in names), (
+            "GROUPS must hold names only — a description here would not be derived")