Skip to content

Commit 91de3a8

Browse files
dmealingclaude
andcommitted
docs(adr): amend ADR-0007 per the #212 ruling — admission test + @ROLE shrink
ADR-0007 catalogued `source.event @topic @ROLE: publish`; ADR-0028 later ruled that message topics are channels, not sources, "never in source.*". Both were accepted and on the books, and nothing reconciled them. Read literally the later decision foreclosed the earlier one's catalog entry. Nothing broke only because `source.rdb` is the sole registered paradigm. Resolved in favour of ADR-0028, and generalized. Amendment 1 removes `event` and replaces the implicit admission rule with an explicit test: a source binds ADDRESSABLE STATE AT REST — readable on demand by a declared key or address, not merely observed as it flows past. The second half of that test is the load-bearing part. #212 proposed a two-prong AND (addressable AND drift-inspectable as a schema), and swept across all twelve paradigms it fails in both directions: it would ADMIT a schema-registry-backed Kafka topic and EXCLUDE source.memory, while wrongly maiming document ($jsonSchema optional), keyValue (Redis has none), objectStore (CSV), timeSeries (Prometheus) and graph. So drift-inspectability is demoted to a per-paradigm verify capability and is explicitly NOT an admission criterion. Under the single prong only event's flow kinds fall; get-by-id generalizes to get-by-declared-key/address, so keyValue composite keys, objectStore paths and timeSeries series+time all qualify, as do vector and search (both expose fetch-by-id — similarity is the query path, not the only read). The escape clause is recorded as principled: a stream becomes a source exactly when it is treated as addressable state, so a compacted changelog read by key enters as an ordinary read-only-@kind source. What stays out is only the flow itself. Amendment 2 records that @ROLE is a designation mechanism, not a routing one. The Consequences section promised codegen and runtime would "route by @ROLE (primary = CRUD; index/cache/publish = derived)". No port ever built it: across all five, every read of @ROLE is an equality test against primary — Java's OMDB has zero role usage, and Kotlin's KotlinGenUtil and Python's write-through read path are explicitly documented role-agnostic, finding the replica by @kind. So the registered vocabulary shrinks to primary | replica, with index/cache/publish/ mirror reserved-not-registered (the ADR-0040 treatment) and a stated re-entry bar: a role member enters the registry only when a shipping consumer dispatches on it. Also recorded: replica is structurally required despite no consumer reading the word, because @ROLE defaults to primary and the one-primary invariant rejects two primaries — a second source must carry an explicit non-primary role or the model fails to load. Docs only. ADR-0035 §1 excludes reserved-but-unregistered vocabulary from the compat surface, so this does not reset the quiet-period clock. The @ROLE registry change itself rides the #210 consolidation batch; the eventing surface rides FR-024 to 1.1. Synced: ADR-0028's channel clause now cites the ratification; ADR-0018's per-paradigm example no longer teaches a removed paradigm; the source-v2 design doc strikes the event row, the CDC/outbox example and the four reserved roles; FR-024 §7's channel row references a projection rather than a value, landing the symmetry — queries return projections, commands take values, events emit projections. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE
1 parent 5721696 commit 91de3a8

5 files changed

Lines changed: 120 additions & 24 deletions

docs/superpowers/specs/2026-05-23-source-v2-paradigm-subtypes-multisource-design.md

Lines changed: 21 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,13 @@
22

33
- **Date:** 2026-05-23
44
- **Status:** Design — plan-of-record. Authority for the `source` metatype going forward.
5+
**Amended 2026-08-05** ([#212](https://github.com/metaobjectsdev/metaobjects/issues/212), ADR-0007
6+
Amendments 1–2): the **`event` paradigm is removed** (a stream is a channel, not a source — a
7+
`source.*` binds **addressable state at rest**; emission moves to the surface layer as
8+
`api.eventing`/`operation.event`/`binding.messaging` with a payload→**projection** reference), and
9+
**`@role`'s registered vocabulary shrinks to `primary | replica`** (`index`/`cache`/`publish`/`mirror`
10+
become reserved-not-registered; no port ever built the anticipated role-routing). The rows below
11+
marked ~~struck~~ are retained for provenance — read ADR-0007 for the governing contract.
512
- **Decision record:** [ADR-0007](../../../spec/decisions/ADR-0007-source-v2-paradigm-subtypes-multisource.md) (the durable contract); this spec is the detailed design + migration + rollout.
613
- **Supersedes:** FR-003 / Project E `source.dbTable` / `source.dbView`.
714
- **Related:** [ADR-0006](../../../spec/decisions/ADR-0006-reserved-keywords-vs-inline-attributes.md) (reserved keywords vs `@`-attrs — resolved here for `source`), ADR-0002 (subtype behavior), ADR-0004 (per-subtype attr schemas).
@@ -13,7 +20,7 @@
1320

1421
A `source` declares **where an object's data physically lives**. Rules:
1522

16-
1. **Subtype = storage paradigm** (`rdb`, `document`, `event`, …). The paradigm selects the
23+
1. **Subtype = storage paradigm** (`rdb`, `document`, `search`, …). The paradigm selects the
1724
codegen/runtime driver, so it's the behavioral axis (ADR-0002). Each subtype owns its
1825
attribute vocabulary (ADR-0004).
1926
2. **`name` = logical name** (optional on sources), consistent with every node (ADR-0006).
@@ -31,7 +38,7 @@ A `source` declares **where an object's data physically lives**. Rules:
3138
|--|--|--|--|--|
3239
| **rdb** | Postgres, MySQL, SQLite, SQL Server, Oracle | `@table` | table* · view · materializedView · storedProc · tableFunction | `@schema`, `@refresh`(matview), `@params`(proc/fn) |
3340
| **document** | MongoDB, CouchDB, Cosmos, Firestore | `@collection` | collection* · view · gridFs | `@database`, `@viewOn`+`@pipeline`(view) |
34-
| **event** | Kafka, Pulsar, Kinesis | `@topic` | topic* · stream · eventStore · changelog | `@keySchema`, `@valueSchema`, `@partitions`, `@compaction`, `@consumerGroup` |
41+
| ~~**event**~~ **REMOVED (#212)** | ~~Kafka, Pulsar, Kinesis~~ | ~~`@topic`~~ | ~~topic* · stream · eventStore · changelog~~ | A stream is a *channel*, not a source. Emission belongs to the surface layer (`binding.messaging` on an `operation.event`, payload → projection). A compacted changelog / event store read **by key** re-enters as an ordinary read-only-`@kind` paradigm source under the escape clause — it is addressable state at rest. |
3542
| **keyValue** | DynamoDB, Redis, etcd | `@table` / `@namespace` | table* · keyspace | `@partitionKey`, `@sortKey`, `@gsi`, `@ttl` |
3643
| **wideColumn** | Cassandra, ScyllaDB, Bigtable | `@table` | table* | `@keyspace`, `@partitionKey`, `@clusteringKey`, `@columnFamily` |
3744
| **graph** | Neo4j, Neptune, ArangoDB | `@label` / `@edge` | node* · relationship | `@from`, `@to` (relationship) |
@@ -79,7 +86,7 @@ The same per-subtype principle, one level down — a field's address *within* a
7986
| paradigm | field physical attr | notes |
8087
|--|--|--|
8188
| rdb / wideColumn / objectStore | `@column` | renames the old `@dbColumn` |
82-
| document / event / search / vector | `@field` | dotted path allowed (`name.first`) |
89+
| document / search / vector | `@field` | dotted path allowed (`name.first`) |
8390
| graph | `@property` | |
8491
| keyValue | `@attribute` | |
8592
| timeSeries | `@column` / `@tag` | Influx tag vs field distinction |
@@ -101,7 +108,8 @@ An object may declare multiple `source` children:
101108
{ "object.entity": { "name": "Product", "children": [
102109
{ "source.rdb": { "@table": "products", "@schema": "catalog" } }, // primary (default), table
103110
{ "source.search": { "@index": "products_idx", "@role": "index" } }, // maintained on write
104-
{ "source.event": { "@topic": "product.changed", "@role": "publish" } },// CDC / outbox
111+
// REMOVED (#212): `source.event @role: publish` — a CDC/outbox topic is emission, not
112+
// population. The outbox TABLE is an ordinary rdb source; the topic hop is a binding.
105113
{ "source.keyValue": { "@namespace": "product:", "@role": "cache", "@ttl": 300 } }
106114
/* …fields… */
107115
]}}
@@ -111,10 +119,15 @@ An object may declare multiple `source` children:
111119
|--|--|--|
112120
| `primary`* | system of record (may be read-only for a projection) | CRUD / canonical read |
113121
| `replica` | read copy (read replica, matview) | read routing |
114-
| `index` | search/vector index derived from primary | search queries; maintained on write |
115-
| `cache` | read-/write-through cache | cache get/set |
116-
| `publish` | event/stream sink (CDC, outbox, event-sourcing) | emit-on-write |
117-
| `mirror` | dual-write (migration) | parallel write |
122+
| ~~`index`~~ **RESERVED (#212)** | ~~search/vector index derived from primary~~ | Not registered — no consumer dispatches on it |
123+
| ~~`cache`~~ **RESERVED (#212)** | ~~read-/write-through cache~~ | Not registered — no consumer dispatches on it |
124+
| ~~`publish`~~ **RESERVED, not registered (#212)** | ~~event/stream sink (CDC, outbox, event-sourcing)~~ | Emission is a surface concern; see ADR-0007 Amendment 2. `index`/`cache`/`mirror` are likewise reserved-not-registered — no consumer in any port dispatches on them. |
125+
| ~~`mirror`~~ **RESERVED (#212)** | ~~dual-write (migration)~~ | Not registered — no consumer dispatches on it |
126+
127+
**Registered vocabulary is `primary | replica` only** (ADR-0007 Amendment 2). The struck rows are
128+
reserved: documented here, not in the registry, re-entering only when a shipping consumer
129+
dispatches on them. The example above is therefore illustrative of the *paradigm* axis; its
130+
`@role: index` / `@role: cache` values are not currently loadable.
118131

119132
**Validation:** exactly one `primary` per object (`ERR_SOURCE_NO_PRIMARY` / `ERR_SOURCE_MULTIPLE_PRIMARY`). Single-source objects need no `@role` (defaults to `primary`).
120133

docs/superpowers/specs/2026-06-12-fr-024-entity-surfaces-projections-design.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -232,7 +232,7 @@ errors when it is **provably all-to-one** (every hop explicitly declares
232232
| `rdb @kind:table` | **entity** only | own, required |
233233
| `rdb @kind:view` / `materializedView` | **projection** (standalone) or **entity** (multi-source, read role) | borrowed / own |
234234
| `rdb @kind:storedProc` / `tableFunction` | result shape = **projection** (today spelled entity — migration candidate, §10); args = **value** via `@parameterRef` (shipped) | optional — result sets are often keyless |
235-
| *(future)* message topic / queue | **nobody** — a topic is a *channel*, not a population location; it belongs to the surface layer as `binding.messaging` on an operation referencing a value (AsyncAPI's model: channels reference message schemas) | n/a |
235+
| *(future)* message topic / queue | **nobody** — a topic is a *channel*, not a population location; it belongs to the surface layer as `binding.messaging` on an operation referencing a **projection** (AsyncAPI's model: channels reference message schemas). Ratified 2026-08-05 (#212 / ADR-0007 Amendment 1): an emitted event's shape is system-produced derived output, so the CQRS symmetry is *queries return projections, commands take values, **events emit projections***. A topic that is treated as addressable state (compacted changelog / event store read by key) re-enters as an ordinary read-only-`@kind` paradigm source. | n/a |
236236
| no source | **value** (embedded / payload / args) or **projection** (wire-only contract shape) | value: never; projection: optional |
237237

238238
**Assembly modes.** Whether origins are *required* depends on the source:

spec/decisions/ADR-0007-source-v2-paradigm-subtypes-multisource.md

Lines changed: 92 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
# ADR-0007 — Source metatype v2: storage-paradigm subtypes, logical names, multi-source, per-subtype physical addresses
22

3-
**Status:** Accepted (proposed 2026-05-23; shipped in all five ports)
3+
**Status:** Accepted (proposed 2026-05-23; shipped in all five ports). **Amended 2026-08-05**
4+
([#212](https://github.com/metaobjectsdev/metaobjects/issues/212)) — see Amendments 1 and 2:
5+
`event` is removed from the paradigm catalog and replaced by an explicit admission test
6+
("addressable state at rest"), and `@role`'s registered vocabulary shrinks to `primary | replica`.
47

58
**Supersedes:** the `source.dbTable` / `source.dbView` vocabulary introduced for FR-003
69
(Project E, "Source-aware entities + projections"). Builds on ADR-0002 (subtype behavior on
@@ -35,19 +38,22 @@ treats the dialect/provider as runtime config and metadata as backend-agnostic.
3538

3639
**Source v2** restructures the `source` metatype along four rules:
3740

38-
1. **Subtype = storage paradigm**, not object kind: `source.{rdb, document, event, keyValue,
41+
1. **Subtype = storage paradigm**, not object kind: `source.{rdb, document, keyValue,
3942
wideColumn, graph, search, vector, timeSeries, objectStore, api, memory}`. The paradigm is the
4043
real behavioral axis (it selects the codegen/runtime driver — JOOQ/SQLAlchemy vs a document
41-
ODM vs a Kafka client), so it belongs on the subtype per ADR-0002. Each paradigm owns its
44+
ODM vs a search client), so it belongs on the subtype per ADR-0002. Each paradigm owns its
4245
attribute vocabulary per ADR-0004.
4346

47+
**Amended 2026-08-05 (#212): `event` is removed from the catalog** — see
48+
*Amendment 1* below for the admission test that governs which paradigms belong here at all.
49+
4450
2. **`name` = logical name (optional); physical address = a per-subtype idiomatic attribute**,
4551
at **both** the source and field level (ADR-0006-compliant; the physical name is never `@name`):
4652

47-
| level | rdb | document | event | graph ||
53+
| level | rdb | document | graph | search ||
4854
|---|---|---|---|---|---|
49-
| source physical | `@table` | `@collection` | `@topic` | `@label`/`@edge` | per paradigm |
50-
| field physical | `@column` | `@field` | `@field` | `@property` | per paradigm |
55+
| source physical | `@table` | `@collection` | `@label`/`@edge` | `@index` | per paradigm |
56+
| field physical | `@column` | `@field` | `@property` | `@field` | per paradigm |
5157

5258
`@dbColumn` is renamed **`@column`**. When a physical attr is omitted it is derived from the
5359
logical `name` via the naming strategy (today's column-naming behavior, generalized). Because
@@ -61,24 +67,96 @@ treats the dialect/provider as runtime config and metadata as backend-agnostic.
6167
the dominant noun.
6268

6369
4. **Multi-source via `@role`.** An object may declare **N `source` children**; each carries
64-
`@role` (default `primary`): `primary` (system of record; may be read-only for a projection),
65-
`replica`, `index`, `cache`, `publish`, `mirror`. **Exactly one `primary`** per object. The
66-
primary drives CRUD/canonical reads; secondaries drive derived behavior (index maintenance,
67-
cache, event emission). Single-source objects are unchanged: one source, role defaults to
68-
`primary`.
70+
`@role` (default `primary`). **Exactly one `primary`** per object; the primary is the system of
71+
record and drives CRUD/canonical reads. Single-source objects are unchanged: one source, role
72+
defaults to `primary`.
73+
74+
**Amended 2026-08-05 (#212 sub-decision): the registered vocabulary is `primary | replica`**
75+
see *Amendment 2* below.
76+
77+
## Amendment 1 (2026-08-05, #212) — the admission test: addressable state at rest
78+
79+
ADR-0028 subsequently ruled that "message topics/queues are *channels*, not sources — they live at
80+
the surface layer as `binding.*` on operations, never in `source.*`", which contradicted this ADR's
81+
`source.event @topic @role: publish` catalog entry. Both were on the books. Resolved in favour of
82+
ADR-0028, and generalized into the test that governs the whole catalog:
83+
84+
> A `source.*` binds an object to **addressable state at rest** — a place where a current value of
85+
> an instance can be read on demand by a declared key or address, not merely observed as it flows
86+
> past. **Drift-inspectability is NOT an admission criterion.** It follows per paradigm from
87+
> whatever schema authority the backend exposes (`information_schema`, index mapping, schema
88+
> registry, parquet footer) and may be absent entirely; it is a `verify` capability.
89+
90+
The second sentence is load-bearing. A two-prong test requiring both addressability *and* a
91+
drift-inspectable schema fails in both directions: it would **admit** a schema-registry-backed Kafka
92+
topic (Confluent exposes key/value schemas) and **exclude** `source.memory` (nothing exists to drift
93+
against), while wrongly maiming `document` (`$jsonSchema` optional), `keyValue` (Redis has none),
94+
`objectStore` (parquet yes, CSV no), `timeSeries` (Timescale yes, Prometheus no) and `graph`.
95+
96+
Under the single-prong test only **`event`'s flow kinds (topic / stream)** fall. "Get-by-id"
97+
generalizes to "get-by-declared-key/address", so composite keys (`keyValue`), paths
98+
(`objectStore`) and series+time (`timeSeries`) all qualify; `vector` and `search` qualify too, since
99+
both expose fetch-by-id (similarity is the *query* path, not the only read).
100+
101+
**Escape clause — principled, not ad hoc:** *a stream becomes a source exactly when it is treated as
102+
addressable state.* A compacted changelog or event store read by key (KTable-style) enters as an
103+
ordinary read-only-`@kind` paradigm source. What stays out is only the **flow** itself; modeling a
104+
flow as storage is the category error. Windowed/stateful stream aggregation stays out
105+
**permanently** — it violates the determinism contract that a derivation is a pure function of the
106+
*current* entity graph, and is stream *processing*, not projection.
107+
108+
**Where emission goes instead:** the surface layer (`api.eventing` / `operation.event` /
109+
`binding.messaging`, per ADR-0030), with the event's payload referencing an `object.projection`.
110+
This lands the symmetry — **queries return projections, commands take values, events emit
111+
projections** — and the projection concept still does all the shape work; only the exposure is a
112+
channel binding rather than a source.
113+
114+
## Amendment 2 (2026-08-05, #212 sub-decision) — `@role` is a designation, not a routing mechanism
115+
116+
The Consequences below say codegen and runtime "route by `@role` (primary = CRUD; index/cache/publish
117+
= derived)". **No port ever built that dispatch.** Across all five, every read of `@role` is an
118+
equality test against `primary`: Java's OMDB has zero role usage, and Kotlin's `KotlinGenUtil` and
119+
Python's write-through read path are explicitly documented *role-agnostic* — they find the replica
120+
view by read-only `@kind`. The consumed information content is one bit.
121+
122+
Accordingly:
123+
124+
- **Registered vocabulary shrinks to `primary | replica`.** `index`, `cache`, `publish` and `mirror`
125+
are **reserved — documented here, NOT registered**, the same treatment ADR-0040 gave
126+
`index.fulltext` / `index.vector` / `index.spatial`. (`publish` is additionally dead on doctrine:
127+
emission is a surface concern per Amendment 1.)
128+
- **Re-entry bar:** *a role member enters the registry only when a shipping consumer dispatches on
129+
it.* Registration did not summon the routing feature over the project's life; reserving costs a
130+
future adopter one paragraph, and re-entry post-1.0 is additive whereas removal would be a 2.0
131+
event.
132+
- **`replica` is structurally required**, despite no consumer reading the word: `@role` defaults to
133+
`primary` when omitted and the one-primary invariant rejects two primaries, so a *second* source
134+
must carry an explicit non-primary role or the model fails to load. The enum cannot shrink below
135+
two members without redesigning the default.
136+
- **`@role` is not derivable** (so it survives ADR-0037 as a step-3 configuration attribute):
137+
primacy is a tie-breaking *designation*, and `table(primary)` + `table(replica)` — two
138+
same-writability sources — is legal and becomes inexpressible under any "the writable one is
139+
primary" derivation.
140+
141+
For consumer-meaningless author annotations about a source's purpose, the chartered home is the
142+
registered `attr.properties` bag (ADR-0023), not members of a byte-checked enum whose presence
143+
implies behavior that does not exist.
69144

70145
## Consequences
71146

72147
- **Cross-language migration** (TS / C# / Java / Python): the loader subtype set changes
73148
(`dbTable`/`dbView``rdb` + `@kind`), `@name``@table`, `@dbColumn``@column`, and
74149
`@role` + multi-source validation (exactly one primary) are added.
75150
- **Codegen/runtime** stop dispatching read-only off the *subtype* and instead off **`@kind`**,
76-
and stop assuming one source — they **route by `@role`** (primary = CRUD; index/cache/publish =
77-
derived). Degrades gracefully: no secondaries ⇒ today's behavior.
151+
and stop assuming one source — they select the primary by `@role` and everything else by
152+
`@kind`. Degrades gracefully: no secondaries ⇒ today's behavior. *(As shipped, `@role` selects
153+
only the primary; the "route by `@role`" routing this originally anticipated was never built —
154+
see Amendment 2.)*
78155
- **Conformance corpus** migrates every `source.*` fixture; this also resolves the ADR-0006
79156
`source.@name` violation (the physical name moves to the non-reserved `@table`).
80157
- **Scope is staged.** Only `source.rdb` is implemented now (it covers everything FR-003 does);
81-
the other ten paradigms are a validated roadmap, each built when a backend lands.
158+
the other paradigms are a validated roadmap, each built when a backend lands — subject to the
159+
Amendment 1 admission test.
82160
- **Back-compat:** `dbTable`/`dbView` documents stop loading. Intended — they are migrated in the
83161
shared corpus, and `source` is pre-1.0.
84162

spec/decisions/ADR-0018-per-kind-physical-name-attrs.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
## Context
99

10-
ADR-0007 introduced the source-v2 paradigm and established the rule that the physical-address attribute is *per-paradigm-idiomatic*: `@table` for `source.rdb`, `@collection` for `source.document`, `@topic` for `source.event`, `@label`/`@edge` for `source.graph`. The Alternatives Considered section rejected a single generic `@locator` / `@map` attribute on the grounds that paradigms have genuinely different vocabularies and a multi-sourced field needs paradigm-specific addresses (`@column` for the rdb side, `@field` for the document side, simultaneously).
10+
ADR-0007 introduced the source-v2 paradigm and established the rule that the physical-address attribute is *per-paradigm-idiomatic*: `@table` for `source.rdb`, `@collection` for `source.document`, `@index` for `source.search`, `@label`/`@edge` for `source.graph`. The Alternatives Considered section rejected a single generic `@locator` / `@map` attribute on the grounds that paradigms have genuinely different vocabularies and a multi-sourced field needs paradigm-specific addresses (`@column` for the rdb side, `@field` for the document side, simultaneously).
1111

1212
That per-paradigm principle is correct and is preserved here.
1313

spec/decisions/ADR-0028-object-taxonomy-projection-value-purity.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,11 @@ semantic lie.
3333
or by embedding; embedded VO storage belongs to the owning entity's field).
3434
Message topics/queues are *channels*, not sources — they live at the surface
3535
layer as `binding.*` on operations (AsyncAPI's model), never in `source.*`.
36+
*(Ratified over ADR-0007's conflicting `source.event` catalog entry on
37+
2026-08-05, [#212](https://github.com/metaobjectsdev/metaobjects/issues/212);
38+
ADR-0007 Amendment 1 removes `event` and generalizes this into the admission
39+
test — a source binds **addressable state at rest**, and a stream becomes a
40+
source exactly when it is treated as addressable state.)*
3641
4. **Derived means read-only, at two levels:** any field carrying `origin.*` is
3742
derived and therefore read-only wherever it lives (on entities: excluded from
3843
INSERT/UPDATE, write codecs, and create/update inputs); a projection is wholly

0 commit comments

Comments
 (0)