Skip to content

Commit 1c764d1

Browse files
dmealingclaude
andcommitted
docs(codegen): the template-spec is discovered, and the gap notes are retracted
Documents the fix in the two commits before this, and RETRACTS the three "known gap" notes shipped earlier in this same session — they described the defect as permanent because at the time it was, and leaving them would be worse than never having written them. WHAT THE NOTES SAID AND WHY THEY GO. own-your-codegen.md, codegen-concepts.md and the metaobjects-codegen skill each carried a warning that --template-spec was accepted by `gen` only, that neither port auto-discovered a spec, and that adopters should therefore keep template output OUTSIDE the directory verify --codegen diffs. All three are now false, and the workaround they recommended is actively bad advice: output kept out of the diffed directory is output the drift gate does not cover. They are replaced by the convention rather than deleted silently, because the reason to prefer the conventional path over the flag is the same fact that made the bug: `verify --codegen` accepts no --template-spec, so discovery is the ONLY channel by which the drift gate learns your template generators exist. That is worth stating positively at every site an adopter reaches — a flag-only setup still works for `gen` and still leaves `verify` resolving a different generator list. ONE CORRECTION I OWE. The retracted notes claimed Python "convicts their committed output" unconditionally. That was true only for a spec emitting .py — the one extension the old *.py-scoped comparison could see. For every other extension it was the opposite failure, silence, and the CHANGELOG entry now states both halves with the reproduction for each. I had verified the .py case and generalised from it. Per-port pages get the worked form: the conventional path, the anchor (the metadata dir's parent, which is what .metaobjects/ already uses), the flag-overrides rule, and a two-line gen/verify pair showing both verbs resolving the same file. python.md additionally documents the declarative-config-mode refusal, and notes the flag-name asymmetry that trips people: `gen` spells the templates dir --templates, while on `verify` that name is the SUBVERB, so the directory flag there is --templates-root. cli.md's C# and Python rows say the flag is auto-discovered and why the conventional path is the one to use. The roadmap entry moves from Planned to the SP-3 shipped block — carrying the C# jurisdiction gap forward as explicitly still open, since that is the one thing here that was found and NOT fixed, and it must not disappear with the entry that named it. Fixture churn: all five agent-context-conformance corpora regenerated from tracked source. Verified: doc-example gate 0 (91 examples), site-payload fresh, site-reference fresh (16 pages), sdk 300/0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSCgs7z19w8aGXGiceCohC
1 parent 08c4cb7 commit 1c764d1

13 files changed

Lines changed: 179 additions & 78 deletions

File tree

CHANGELOG.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,76 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
77

88
## [Unreleased]
99

10+
### Fixed — `verify --codegen` convicted the output `gen` had just written (C#, Python)
11+
12+
The declarative Mustache template-spec was wired into `gen` and nowhere else, so the drift
13+
gate regenerated a **different generator list** than the generator did — and then failed the
14+
project for the difference. Reproduced against the shipped code:
15+
16+
```
17+
$ metaobjects gen ./meta --out ./out --template-spec spec.json --templates ./templates
18+
...wrote 9 file(s)
19+
$ metaobjects verify --codegen ./meta --out ./out
20+
error: generated code is out of sync with metadata.
21+
extra: OrderService.py
22+
extra: ProductService.py
23+
regenerate (metaobjects gen) and commit the result. EXIT 1
24+
```
25+
26+
The printed remedy is a **loop**: regenerating cannot produce files the regen does not know
27+
about, and `verify` accepts no `--template-spec` to be told. C# failed the same way
28+
(`committed but a fresh regen would not emit it`). It lands on exactly the two ports whose
29+
generator registries are closed — where the template-spec is the **only** consumer authoring
30+
path — so an adopter doing the one thing their port supports failed their own gate.
31+
32+
**SP-1 §4 had specified the fix and it was never built:** *"`--template-spec <path>`, with a
33+
conventional default the port auto-discovers and the flag overriding"*, and that **both**
34+
`gen` and `verify` read it. Only the flag on `gen` shipped. Both ports now resolve the spec
35+
through **one shared helper**: explicit flag wins, else `<projectRoot>/template-spec.json`,
36+
where projectRoot is the metadata dir's **parent** — not a new rule, it is the anchor
37+
`.metaobjects/.gen-state/` already uses on both ports. A flag on `verify` was rejected
38+
deliberately: it would need repeating at every CI call site, and one forgotten reproduces the
39+
bug exactly.
40+
41+
**Python was blind as well as wrong, and that half was worse.** For a spec emitting anything
42+
other than `.py`, there was no false conviction — there was silence. With one generated file
43+
**deleted** and another **overwritten with garbage**, the gate answered:
44+
45+
```
46+
metaobjects verify: in sync (7 file(s)). EXIT 0
47+
```
48+
49+
`_relative_set` globbed `*.py` under a docstring that had already called it — *"if a
50+
generator ever emits a non-`.py` artifact, broaden this glob so `verify` drift-checks it
51+
too."* One does: template-spec output is format-agnostic by design (text/markdown/csv/
52+
json/xml/html), which is the **common** case, so the usual configuration was ungated
53+
entirely. The comparison now covers every text artifact (skipping `__pycache__`/`*.pyc` and
54+
anything that will not decode).
55+
56+
**Broadening it required the jurisdiction rule, or it would have re-created the bug it
57+
fixes.** Seeing every file in `outDir` means convicting every stranger in it — precisely what
58+
failed a zero-drift project in TypeScript and produced the `0.24.3` ruling: **`outDir` is a
59+
directory, not a namespace this tool owns.** The manifest already records what we *wrote*,
60+
keyed relative to `out_dir` — the same key space the diff uses — so `extra` is now scoped to
61+
files with a write record, failing closed when there is no manifest. C# needed neither half:
62+
its comparison already enumerated `*`.
63+
64+
Also: Python's declarative-config mode **accepted `--template-spec` and silently did
65+
nothing** (`_cmd_gen_config` goes straight to `_run_gen_targets`, which has no spec pass). It
66+
now refuses with the working form, because it cannot simply be honoured — a spec entry names
67+
no `target` while config mode writes per target, so there is no non-arbitrary outDir. A
68+
*discovered* spec is ignored there rather than refused, so discovery can never hard-fail a
69+
command that did not ask for it.
70+
71+
**Behaviour change:** a project that already has `<projectRoot>/template-spec.json` and
72+
passes no flag will now run it. Exposure is near zero — the filename is only meaningful if
73+
you knew of a flag that had no default — but it is real.
74+
75+
**Known and NOT fixed:** C#'s `CodegenDrift` has no jurisdiction guard on its
76+
`inCommitted && !inFresh` branch, so it still convicts files it never wrote — the
77+
pre-`0.24.3` TypeScript behaviour. It is not introduced here and fixing it changes `verify`'s
78+
verdict for every existing C# adopter, so it is reported rather than folded in.
79+
1080
### Fixed — a reference fragment the stack no longer uses is deleted, not announced forever
1181

1282
`meta init --refresh-docs --server typescript` on a project scaffolded as python left

agent-context/skills/metaobjects-codegen/SKILL.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -361,12 +361,11 @@ reach for it here.) Use `--template-spec <json>` — plus `--templates <dir>` on
361361
`--template-root <dir>` on C# — and your entries are appended to the default suite. Worked
362362
examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`.
363363

364-
**One caveat to plan around on those two ports:** `--template-spec` is accepted by `gen`
365-
only. `verify` does not take it and neither port auto-discovers a spec file, so
366-
`verify --codegen` regenerates WITHOUT your template generators and reports their
367-
committed output as stale — Python prints `extra: <path>` and exits 1, and the remedy it
368-
prints is a loop. Emit template-spec output somewhere other than the directory
369-
`verify --codegen` diffs, and do not "fix" the failure by deleting the files.
364+
**The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both
365+
ports read `<projectRoot>/template-spec.json` — projectRoot being the metadata dir's parent.
366+
Keep it there: `verify --codegen` accepts no `--template-spec` flag, so the conventional path
367+
is how the drift gate learns your template generators exist. Put the spec somewhere else and
368+
reach it only by flag, and `verify` regenerates without it and reports its output as stale.
370369

371370
So on C#/Python, "I need a shape the built-ins do not emit" is answered by a template, not
372371
by writing generator code. Do not conclude the port cannot be customized.

docs/features/cli.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,10 @@ command surface splits in two:
3434
| **Vocabulary upgrade** (`upgrade`) | **Node `meta`** | `meta upgrade [--to <version>] [--apply]` | **any backend** — rewrites RETIRED metadata vocabulary (`@violation``@counterexample`, `@readOnly``@mutability`, dropping `@verifiedBy`) and resolves ATTRIBUTE CONTRADICTIONS (`@fields` beside `@expr` on an index key). Node-only because it edits the metadata documents themselves, which every port shares; a non-TS project runs `npx meta upgrade` against its own `metaobjects/`. **Canonical JSON and YAML alike.** Previews by default. Retirements needing a human decision are refused and the run exits non-zero, so CI cannot record a partial migration as finished |
3535
| **Vocabulary search** (`types`) | **Node `meta`** | `meta types [query]` | **any backend** — apropos/`kubectl explain` over the live metamodel registry (names + descriptions + when-to-use); the vocabulary is cross-port identical (registry-conformance) |
3636
| TS codegen | Node `meta` | `meta gen` | TS projects. **No `--template-spec` flag, deliberately**`metaobjects.config.ts` already takes generator VALUES, so a declarative template generator is declared there (`templateGenerator()`, or `templateSpecToGenerators(parseTemplateSpec(spec))` to reuse a C#/Python spec file). Keeping it in the config is what lets `meta verify --codegen` regenerate WITH it, since that gate re-runs the config's generator list; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) |
37-
| C# codegen | `dotnet meta` | `dotnet meta gen` / `verify --templates` / `verify --codegen` | a .NET tool (`ToolCommandName=dotnet-meta`); invoked `dotnet meta` so it never shadows the Node `meta`; ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2; bare `verify` = `--templates`). `gen` also accepts `--template-spec <json>` (+ `--template-root <dir>`, default `templates`) — the declarative Mustache template-codegen surface (the cross-port JSON contract shared with Python); see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) |
37+
| C# codegen | `dotnet meta` | `dotnet meta gen` / `verify --templates` / `verify --codegen` | a .NET tool (`ToolCommandName=dotnet-meta`); invoked `dotnet meta` so it never shadows the Node `meta`; ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2; bare `verify` = `--templates`). `gen` also accepts `--template-spec <json>` (+ `--template-root <dir>`, default `templates`) — the declarative Mustache template-codegen surface (the cross-port JSON contract shared with Python), **auto-discovered at `<projectRoot>/template-spec.json`** when the flag is absent. Prefer the conventional path: `verify --codegen` takes no `--template-spec`, so discovery is how the drift gate sees your template generators at all; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) |
3838
| Java/Kotlin codegen | Maven plugin | `mvn metaobjects:generate` (`metaobjects:generate`) | Kotlin generators run through the same goal — see below. **No `--template-spec` flag, deliberately**`<generator>` already loads a consumer class from the project classpath, so the declarative surface is `com.metaobjects.generator.template.TemplateScopeGenerator` wired as an ordinary `<generator>` with `<template>` / `<scope>` / `<outputPattern>` / `<templatesDir>` / `<format>` args (covers BOTH Java and Kotlin); see [declarative template scopes](codegen-concepts.md#declarative-template-scopes). The `generate`/`verify`/`docs` goals are declared `threadSafe` and support parallel multi-module reactor builds (`mvn -T`) (#233) |
3939
| Java/Kotlin verify | Maven plugin | `mvn metaobjects:verify -Dmeta.verify.mode=codegen\|templates` (`metaobjects:verify`) | parameter-driven ADR-0021 D2 modes (one goal covers BOTH Java + Kotlin): `codegen` (default, back-compat — regen + fail on drift vs committed output, generator-neutral) / `templates` (`{{field}}`↔payload drift via the render `Verify` engine). `db` rejected ("schema verify is the migrate engine, ADR-0015") |
40-
| Python codegen | console-script | `metaobjects gen` / `verify --codegen` / `verify --templates` | `[project.scripts] metaobjects`**not** `meta` (that's the Node schema CLI); ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2). `gen` also accepts `--template-spec <json>` (+ `--templates <dir>`, default `templates`) — the declarative Mustache template-codegen surface (the cross-port JSON contract shared with C#); see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) |
40+
| Python codegen | console-script | `metaobjects gen` / `verify --codegen` / `verify --templates` | `[project.scripts] metaobjects`**not** `meta` (that's the Node schema CLI); ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2). `gen` also accepts `--template-spec <json>` (+ `--templates <dir>`, default `templates`) — the declarative Mustache template-codegen surface (the cross-port JSON contract shared with C#), **auto-discovered at `<projectRoot>/template-spec.json`** when the flag is absent (and REFUSED in declarative-config mode, where a spec has no target to write into). Prefer the conventional path: `verify --codegen` takes no `--template-spec`, so discovery is how the drift gate sees your template generators at all; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) |
4141

4242
## `verify` is one verb with explicit subverbs (ADR-0021 D2)
4343

docs/features/codegen-concepts.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -242,10 +242,11 @@ overwrite a file lacking the `@generated` marker — a regenerable template must
242242
that header in its own body. The JVM's `TemplateScopeGenerator` writes directly and
243243
carries no such obligation.)
244244

245-
**Known gap on the two flag ports.** `--template-spec` is accepted by `gen` only.
246-
Neither `metaobjects verify` nor `dotnet meta verify` takes it, and neither port
247-
auto-discovers a spec file, so `verify --codegen` regenerates without your template
248-
generators and convicts their committed output as stale. The ports that have a code
249-
seam (TypeScript, JVM) do not have this problem, because there the declaration lives
250-
in the build config that the drift gate re-runs. Details and the workaround:
251-
[`own-your-codegen.md`](own-your-codegen.md#per-port).
245+
**The spec is auto-discovered.** With no `--template-spec`, both CLI ports read
246+
`<projectRoot>/template-spec.json` (projectRoot = the metadata dir's parent, the anchor
247+
`.metaobjects/` already uses); the flag overrides it. This is what lets `verify --codegen`
248+
regenerate WITH your template generators — it accepts no `--template-spec` of its own, so a
249+
flag-only setup would leave the gate resolving a different generator list from `gen` and
250+
convicting gen's own output. The ports with a code seam (TypeScript, JVM) declare template
251+
generators in the build config the gate already re-runs, reaching the same guarantee a
252+
different way.

docs/features/own-your-codegen.md

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -152,18 +152,19 @@ output *shape* is what you are iterating on, or when you want the same output ac
152152
languages; reach for a generator when the logic is gnarly or the run is hot. Full
153153
tradeoff table: [`codegen-concepts.md` §3](codegen-concepts.md).
154154

155-
> **Known gap on the two flag ports.** `--template-spec` is a flag on `gen` only —
156-
> neither `metaobjects verify` nor `dotnet meta verify` accepts it, and neither port
157-
> auto-discovers a spec file. So `verify --codegen` regenerates WITHOUT your template
158-
> generators and convicts their committed output — Python classifies each as
159-
> `extra: <path>` (*"stale committed file"*) and exits 1, and the remedy it prints
160-
> (*"regenerate and commit the result"*) is a **loop**: regenerating without the flag
161-
> cannot produce those files, and regenerating with it leaves the gate failing
162-
> identically. Until that is closed, keep template-spec output outside the directory
163-
> `verify --codegen` diffs, or run the gate only against the default suite.
164-
> (This is why the two ports that have a code seam —
165-
> TypeScript and the JVM — declare template generators in the build config instead: the
166-
> config is what the drift gate re-runs.)
155+
**The spec file is discovered, not just flagged.** With no `--template-spec`, both ports
156+
look for **`<projectRoot>/template-spec.json`** — projectRoot being the metadata dir's
157+
parent, the same anchor `.metaobjects/` already uses. The flag overrides it.
158+
159+
That matters for more than typing: `verify --codegen` takes **no** `--template-spec` flag,
160+
so discovery is how the drift gate learns about your template generators. Before it existed,
161+
`gen` honoured the flag and `verify` never looked — so verify regenerated without them and
162+
convicted their committed output, with a remedy that loops. Keep the spec at the
163+
conventional path and both verbs resolve the same one.
164+
165+
Passing `--template-spec` explicitly still works and still wins; just make sure any CI that
166+
runs `verify --codegen` can find the spec, which the conventional path guarantees and a
167+
flag-only setup does not.
167168

168169
*(Full command/flag matrix and rationale: [`docs/features/cli.md`](cli.md), locked per
169170
ADR-0015. Schema migrations are TypeScript-owned across all ports.)*

docs/ports/csharp.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,22 @@ dotnet meta gen ./metadata --out ./Generated \
260260
]}
261261
```
262262

263+
**The spec is auto-discovered.** Omit `--template-spec` and the CLI reads
264+
`<projectRoot>/template-spec.json`, where projectRoot is the metadata dir's **parent**
265+
the same anchor `.metaobjects/` uses (`GenCommand.ProjectRootFor`). The flag overrides it.
266+
267+
Prefer the conventional path over the flag, because **`dotnet meta verify --codegen`
268+
accepts no `--template-spec`**: discovery is how the drift gate learns your template
269+
generators exist. A spec reachable only by flag leaves `verify` regenerating a different
270+
generator list from `gen` and convicting your committed template output as
271+
"committed but a fresh regen would not emit it".
272+
273+
```bash
274+
dotnet meta gen ./metadata --out ./Generated --template-root ./templates
275+
dotnet meta verify --codegen ./metadata --out ./Generated --template-root ./templates
276+
# ^ both resolve <projectRoot>/template-spec.json — the gate agrees with the generator
277+
```
278+
263279
Each spec entry derives the neutral template data dict for its scope
264280
(`MetaObjects.Codegen.TemplateCodegen.TemplateData`) and names each file via the
265281
`outputPattern` placeholders (`{name}`, `{Name}`, `{package}`). The named generators

docs/ports/python.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,29 @@ metaobjects gen ./metadata --out ./generated \
194194
]}
195195
```
196196

197+
**The spec is auto-discovered.** Omit `--template-spec` and the CLI reads
198+
`<projectRoot>/template-spec.json`, where projectRoot is the metadata dir's **parent** —
199+
the same anchor `.metaobjects/` uses. The flag overrides it.
200+
201+
Prefer the conventional path over the flag, because **`verify --codegen` accepts no
202+
`--template-spec`**: discovery is how the drift gate learns your template generators exist.
203+
A spec reachable only by flag leaves `verify` regenerating a different generator list from
204+
`gen` and reporting your committed template output as stale.
205+
206+
```bash
207+
metaobjects gen ./metadata --out ./generated --templates ./templates
208+
metaobjects verify --codegen ./metadata --out ./generated --templates-root ./templates
209+
# ^ both resolve <projectRoot>/template-spec.json — the gate agrees with the generator
210+
```
211+
212+
(`gen` spells the templates dir `--templates`; on `verify` that name is the *subverb*, so
213+
the directory flag there is `--templates-root`.)
214+
215+
`--template-spec` is **refused in declarative-config mode** (`metaobjects.config.yaml`,
216+
no positional metadata dir) rather than silently ignored: a spec entry names no `target`
217+
while config mode writes per target, so there is no outDir to render into. A *discovered*
218+
spec is ignored there rather than refused.
219+
197220
Each spec entry derives the neutral template data dict for its scope and names
198221
each file via the `outputPattern` placeholders (`{name}`, `{Name}`, `{package}`).
199222
The named generators are **appended** to the default suite and gated byte-identical

fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/SKILL.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -361,12 +361,11 @@ reach for it here.) Use `--template-spec <json>` — plus `--templates <dir>` on
361361
`--template-root <dir>` on C# — and your entries are appended to the default suite. Worked
362362
examples with the full JSON: `docs/ports/python.md` and `docs/ports/csharp.md`.
363363

364-
**One caveat to plan around on those two ports:** `--template-spec` is accepted by `gen`
365-
only. `verify` does not take it and neither port auto-discovers a spec file, so
366-
`verify --codegen` regenerates WITHOUT your template generators and reports their
367-
committed output as stale — Python prints `extra: <path>` and exits 1, and the remedy it
368-
prints is a loop. Emit template-spec output somewhere other than the directory
369-
`verify --codegen` diffs, and do not "fix" the failure by deleting the files.
364+
**The spec is auto-discovered, and that is load-bearing.** With no `--template-spec`, both
365+
ports read `<projectRoot>/template-spec.json` — projectRoot being the metadata dir's parent.
366+
Keep it there: `verify --codegen` accepts no `--template-spec` flag, so the conventional path
367+
is how the drift gate learns your template generators exist. Put the spec somewhere else and
368+
reach it only by flag, and `verify` regenerates without it and reports its output as stale.
370369

371370
So on C#/Python, "I need a shape the built-ins do not emit" is answered by a template, not
372371
by writing generator code. Do not conclude the port cannot be customized.

0 commit comments

Comments
 (0)