README: the lineage, the learnings, and one stale table deleted (#16, #15)

[borisdev]

Closes #16. Partially addresses #15.

#16 — the A/B existed and was unpublished

examples/greeting.py is Pydantic Graph's smallest complete builder program, morphed. Its docstring says so. examples/ladder/their_hello.py keeps their original shape with no workbench in the file at all.

But grep -iE 'visualize_graph|their_hello|variation of' README.md returned nothing — so the comparison a reader wants lived only in a docstring.

	step 1	step 2
theirs — their_hello.py, no workbench in the file	step_a → 10	step_b → f'Result: {ctx.inputs}'
ours — greeting.py	normalize → a clean name	compose → f'Hello, {name}!'

Plus: theirs is fine, and that is why it is kept. What it cannot do is check or draw itself before the steps exist, or answer "and what if normalize were written differently?"

their_hello.py and contestable.py are in the More index now — both were reachable only from docs/ladder.md or one inline mention.

#15 — restructure, not a word target

⛔ The handoff's 750–900 target counted two generated tables as prose. Both go into <details> instead; the drift test matches on the markers, so nothing is deleted and nothing breaks.

3,407 → 2,902 words visible before expanding anything (−15%), with MORE content on the page.

Short of the ~1,800 I estimated. That estimate assumed cutting all three honesty sections and one of the two greeting walkthroughs; I did neither, and said why in the commit.

The one real deletion — and it had already drifted

A hand-written | check | catches | table sat ~190 lines below the generated one and listed 10 of 12. check_recursion and check_transform_edges had landed; nobody re-synced the copy. Two tables of one set of facts, one generated — spec-as-code.md, source or derived, never both.

Deleted rather than re-synced, because re-syncing leaves the next rule free to do it again. A test now refuses any table row naming a check_* outside the generated block.

Left alone on purpose

• ### ⛔ When not to use this stays at the top. A reader deciding whether to keep reading needs it in the first screen.
• The greeting walkthrough is not collapsed. That would reach ~2,000 visible words, but 43c1a85 deliberately rebuilt this README around it — undoing a decision that recent is a judgement call, not a prune. Flagged in README: collapse the generated tables, merge the three honesty sections #15.

Verification

• 280 tests (278 → 280), both generators green
• Both new guards mutation-tested: dropping the lineage → RED, re-adding a hand-written check table → RED
• Every removed line read before committing, per the standing rule: 21 lines — 13 the stale table, 6 the agent section moved (verified present), 2 headings retitled with content kept

[Copilot]

Copilot review overview

🟢 Approval recommended

The documentation-only changes have minor corrections remaining but no approval-blocking issues.

Review effort: Balanced
Findings: 2

Open (2)

• Align linked source description with the actual greeting adaptation · New
• Remove stale hardcoded rule count outside generated markers · New

What changed in this PR

Makes the Pydantic Graph comparison visible and streamlines README reference material, addressing #16 and part of #15.

Changes:

• Adds the example comparison and discovery links.
• Collapses generated tables and removes the duplicate rules table.
• Adds guards for lineage references and duplicate check tables.

File	Description
tests/​test_reference.py	Adds README documentation guards.
README.md	Publishes the comparison and reorganizes reference sections.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[borisdev]

Update — the brief changed mid-PR

Boris: "before I start manually editing the README I need to make sure you stuff it with all our learnings....then I will refactor and prune and reorg."

So this branch is no longer a prune: 3,407 → 4,425 visible words. Pruning is his next pass; the job here is to not lose anything. Retitled accordingly.

Four learnings rescued from an orphan branch

health-stack-example has 4 commits, is not in main, is two renames behind it, and its code was decided against in #14. The knowledge in it was not, and would have died with the branch:

• Same Python type, different meaning — given_graph / cited_given_graph / expanded_graph: what the user said, what the literature backs, what was added on top. The README's opening claimed that failure abstractly; it now shows it.
• ⛔ A strategy is an ALGORITHM, not an environment. The most likely misuse of this library, and documented nowhere until now. Fakes belong in ctx.deps. The test: do both arms deserve evaluation on the same cases? A fake and a real client do not.
• Two contracts that type-checked and were wrong — edges returned without the concepts they point at, so every id dangled; and subject/object reversed, which returned nothing against a real 6.9 GB index, i.e. "no evidence found", the most plausible wrong answer available.
• Recall and precision fail differently, so they are two stages. One score over both averages unrelated failures and sends you at the wrong half.

And the one about this repo

Five rounds of review on #8: every finding was a check narrower than its claim, not code that was wrong. The dead-link test skipped malformed links; "every rule" listed 10 of 12; the retired-API lint had no token for the method that release removed; the compatibility oracle omitted the three operations that broke; a UI test drove one of three render surfaces. For a library whose product is checks, that belongs on the front page rather than in a postmortem nobody reads.

Design answers that existed only in conversation

The three-state nesting test (replacing a slogan that conflated "I don't know which algorithm wins" with "I don't know what result I want"); why there is no LLM-node type; pattern-vs-example made checkable — and that this repo has zero patterns by that bar; that eval_battle does not take the noise floor for you, named as a live gap; two naming changes as proposals per domain-language.md #3; and a "where this is going" section.

280 tests, both generators green.

Branch cleanup this unblocks

health-stack-example can be deleted once this merges. The four with 0 unique commits — glossary, thesis-and-foil, coherence-finding, well-formedness-rules — can go now; they linger only because delete_branch_on_merge is off.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Both new regression guards allow documentation regressions they are intended to prevent.

Review effort: Balanced
Findings: 2

Open (2)

• Remove stale hardcoded rule count outside generated markers
• Align linked source description with the actual greeting adaptation

Previously missed (2)

In code that hasn't changed since last review

Verify the control link destination

tests/​test_reference.py:138

Issue #16 requires protecting the link to the control example, but this guard checks only its filename. Replacing both hyperlinks with inline-code labels still passes while making the control unreachable from the README. Assert the Markdown link destination as well as the lineage wording.

Detect duplicate references across entire table rows

tests/​test_reference.py:157

This guard detects only backticked check names in the first column. Both a row with check_variables in a later column and a row with an unbackticked name escape detection, allowing the duplicate reference table this test is meant to prevent. Search the whole table row without requiring backticks.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Both new regression guards miss cases they are intended to prevent.

Review effort: Balanced
Findings: 2

Open (2)

• Remove stale hardcoded rule count outside generated markers
• Align linked source description with the actual greeting adaptation

Previously missed (2)

In code that hasn't changed since last review

Validate control links, not just filenames

tests/​test_reference.py:138

Issue #16 requires preserving a link to the control example, but this checks only its filename. Replacing both control links with plain their_hello.py labels still passes, leaving no link to the runnable comparison. Check the Markdown destination as well as the lineage wording.

Detect duplicate check names anywhere in table rows

tests/​test_reference.py:157

This guard misses duplicate tables when a check name is unformatted or outside the first column. Both | check_names | duplicate names | and | duplicate names | [`check_names`](checks.py) | pass undetected. Scan check names across the row without requiring backticks so these hand-maintained copies are rejected too.

[borisdev]

Both inline comments taken. Both were mine, and the first is the worst kind of defect this branch could have shipped.

1. The lineage table was FALSE

their_hello.py does not contain step_a/step_b — it is already a greeting adaptation whose steps are pick (returns "Hello") and compose (returns f"{ctx.inputs}, {ctx.state.name}!"). step_a/step_b appear only in its docstring, describing what it was adapted from. My row pointed at a file and described something else.

Three accurate rows now, and they make the point better than two wrong ones:

upstream, unchanged   their visualize_graph.py   step_a -> 10
the control           their_hello.py             pick -> "Hello"      no workbench in the file
ours                  greeting.py                normalize -> clean   declared

⚠️ And test_the_readme_shows_the_pydantic_graph_lineage passed the entire time — it asserted the words were present, not that they were true. That is precisely the failure every finding on #8 turned out to be, committed by me in the commit that added the guard against it. It now reads the control's source, requires the row to name that file's real steps, and refuses step_a in that row specifically.

2. The numeral outside the generated markers

Correct — reference --write cannot reach it, and my prose-count test matched well-formedness rules and check functions but not 12 rules. A thirteenth check would have left the summary stale with every test green. The numeral is gone; the count lives only where it is derived, and the regex covers a bare rules now.

Both mutation-tested: restoring the false table → RED, putting a stale numeral back → RED. 280 tests, both generators green.

[Copilot]

Copilot review overview

🔵 Needs a closer look

The new guards miss relevant regressions, and the README overstates existing graph capabilities.

Review effort: Balanced
Findings: None

Resolved since last review (2)

• Remove stale hardcoded rule count outside generated markers
• Align linked source description with the actual greeting adaptation

Previously missed (3)

In code that hasn't changed since last review

Test must assert a Markdown link to the control file

tests/​test_reference.py:142

This checks for the filename, not a link to the control as required by #16. Removing both hyperlinks while keeping their labels leaves every assertion here satisfied, so the reachability regression goes undetected. Assert a Markdown link to examples/ladder/their_hello.py.

Guard should detect check names anywhere in rows

tests/​test_reference.py:175

The guard only detects backticked check names in the first column. Reintroducing the deleted table with catches | check columns, or without backticks, duplicates the same facts but passes this test. Search the entire row for check names so this guard does not depend on column order or inline-code formatting.

GraphSpec overstates bundled briefs, datasets, and evaluators

README.md:481

GraphSpec declares boundary types, but no problem brief, rubric, or evaluation dataset; eval_battle takes its dataset separately. Saying a graph “already has all three” overstates the current API and contradicts the graph-level brief listed as future work below. Describe the boundary as enabling evaluation, with the brief, dataset, and evaluators supplied separately.