From 02c37c015b3755536def647fc62d434588c3a419 Mon Sep 17 00:00:00 2001 From: Danilo Alonso Date: Tue, 8 Sep 2026 11:00:11 -0400 Subject: [PATCH] feat(flows): stack and group DFD stores A process no longer gets one store box and one column chip per table. Its reads and writes render as stacked store boxes with one connector each, tables group into clusters (author-declared in clusters/.md and referenced as cluster:, or derived from basetype/subtype families and ERD groups), and every flow entry can carry a prose label: that shows on the chip. The flow view has a per-process view (default) and a connected view with stores/clusters/groups collapse levels, a stack dialog listing members, and a contract dialog listing which columns come from which tables. Five flow.cluster_* validation rules cover unknown or empty clusters, unknown members, and overlap. The modeling skill now authors a label on every flow entry and a cluster for stores a process treats as one thing, in every mode that writes flows, and verifies stack legibility. The key-inherited demo's Collect Payment writes a settlement cluster and is the worked example the skill and the flows guide quote. Non-breaking: existing models render unchanged; db: entries, hash routes, and the export format keep their meaning. New hash params flowview= and collapse= are additive. Design: docs/design/dfd-store-clusters.md Spec: docs/spec/dfd-store-clusters.md --- .claude/project/followups/INDEX.md | 21 +- .../followups/dfd-store-clusters-f-15.md | 14 + .claude/rules/wiki/docs.md | 2 +- .claude/rules/wiki/flow-view.md | 4 +- .claude/rules/wiki/flows.md | 4 +- .claude/rules/wiki/frontend.md | 5 +- .claude/rules/wiki/generators.md | 2 +- .claude/rules/wiki/parser.md | 2 +- .claude/rules/wiki/server.md | 2 +- .claude/rules/wiki/validate.md | 6 +- docs/design/dfd-store-clusters.md | 299 ++++ docs/glossary.md | 15 + docs/guides/flows.md | 152 +- docs/guides/folder-format.md | 21 +- docs/guides/modeling-skill.md | 2 +- docs/guides/validation.md | 5 + docs/spec/dfd-store-clusters.md | 391 +++++ docs/wiki/docs.md | 30 +- docs/wiki/feature-map.md | 1 + docs/wiki/flow-view.md | 148 +- docs/wiki/flows.md | 131 +- docs/wiki/frontend.md | 253 ++-- docs/wiki/generators.md | 53 +- docs/wiki/index.md | 34 +- docs/wiki/parser.md | 19 +- docs/wiki/scan.md | 525 +++++-- docs/wiki/server.md | 126 +- docs/wiki/skill.md | 40 +- docs/wiki/validate.md | 50 +- models/key-inherited/clusters/settlement.md | 10 + models/key-inherited/flows/index.md | 2 +- .../flows/order-to-cash/Collect-Payment.md | 24 +- .../flows/order-to-cash/Create-Sales-Order.md | 4 + .../Create-Sales-Order/Validate-Customer.md | 4 + .../order-to-cash/Create-Sales-Order/index.md | 2 +- .../flows/order-to-cash/index.md | 4 +- models/key-inherited/index.md | 2 +- .../clusters/tag-junctions.md | 12 + .../flows/tag-administration/Merge-Tag.md | 14 +- skills/ignatius-modeling/SKILL.md | 5 +- .../references/dfd-authoring.md | 112 +- .../references/discover-flow.md | 20 +- .../references/flow-templates.md | 130 +- .../references/reverse-engineering.md | 5 + .../ignatius-modeling/references/templates.md | 3 + .../references/verification.md | 12 +- src/app/App.tsx | 54 +- .../flow-node/EdgeContractDialog.tsx | 143 ++ src/app/components/flow-node/StackDialog.tsx | 216 +++ src/app/components/process/IoTable.tsx | 57 +- src/app/components/ui/FabMenu.tsx | 55 +- src/app/globals.d.ts | 5 + src/app/hash-router.ts | 41 +- src/app/hooks/useHashRoute.ts | 21 +- src/app/hooks/useModelData.ts | 12 +- src/app/styles.css | 24 + src/app/views/flow/FlowsView.tsx | 149 +- src/flow-view/FlowDiagramSvg.tsx | 708 +++++++-- src/flow-view/elk-flow-layout.ts | 54 +- src/flow-view/flow-layout.ts | 1318 ++++++++++++++++- src/flows/flow-clusters.ts | 130 ++ src/flows/flow-markdown.ts | 39 + src/flows/flow-parse.ts | 109 +- src/flows/flow-validate.ts | 134 +- src/generators/app.ts | 4 +- src/model/parse.ts | 16 +- src/model/validate.ts | 33 +- src/server/server.ts | 4 +- test/assert.ts | 10 + test/checks/test-app-gen-zero-diagrams.ts | 1 + test/checks/test-cp16-process-examples.ts | 17 +- .../test-cp4e-elk-renders-in-browser.ts | 22 +- test/checks/test-dfd-edge-hover.ts | 107 +- test/checks/test-dfd-stack-dialogs.ts | 579 ++++++++ test/checks/test-flow-chip-placement.ts | 318 ++++ test/checks/test-flow-clusters-parse.ts | 293 ++++ test/checks/test-flow-clusters-validate.ts | 172 +++ test/checks/test-flow-edge-labels.ts | 264 ++++ test/checks/test-flow-leveling.ts | 2 +- test/checks/test-flow-view-grouping.ts | 778 ++++++++++ test/checks/test-graph-search.ts | 30 +- test/checks/test-hash-router.ts | 38 +- test/checks/test-router-index.ts | 6 +- test/checks/test-validate-flows.ts | 2 +- test/checks/test-validate-index.ts | 2 +- .../clusters/role-grants-invalid.md | 9 + .../clusters/role-grants.md | 8 + .../flows/checkout/Bad-Cluster-Ref.md | 20 + .../flows/checkout/Process-Checkout.md | 4 + test/fixtures/hub-dfd/clusters/grants.md | 10 + test/fixtures/hub-dfd/data/GrantAlpha.md | 11 + test/fixtures/hub-dfd/data/GrantBeta.md | 11 + test/fixtures/hub-dfd/data/HubStoreA.md | 11 + test/fixtures/hub-dfd/data/HubStoreB.md | 11 + test/fixtures/hub-dfd/data/LogStoreOne.md | 11 + test/fixtures/hub-dfd/data/LogStoreTwo.md | 11 + .../hub-dfd/data/PrivateStore4Read.md | 11 + .../hub-dfd/data/PrivateStore4Write.md | 11 + .../hub-dfd/data/PrivateStore5Read.md | 11 + .../hub-dfd/data/PrivateStore5Write.md | 11 + .../hub-dfd/data/PrivateStore6Read.md | 11 + .../hub-dfd/data/PrivateStore6Write.md | 11 + .../hub-dfd/data/PrivateStore7Read.md | 11 + .../hub-dfd/data/PrivateStore7Write.md | 11 + test/fixtures/hub-dfd/data/RecordBase.md | 17 + test/fixtures/hub-dfd/data/RecordTypeA.md | 11 + test/fixtures/hub-dfd/data/RecordTypeB.md | 11 + test/fixtures/hub-dfd/externals/Downstream.md | 6 + .../flows/hub-diagram/Process-Eight.md | 17 + .../hub-dfd/flows/hub-diagram/Process-Five.md | 20 + .../hub-dfd/flows/hub-diagram/Process-Four.md | 20 + .../hub-dfd/flows/hub-diagram/Process-One.md | 26 + .../flows/hub-diagram/Process-Seven.md | 20 + .../hub-dfd/flows/hub-diagram/Process-Six.md | 20 + .../flows/hub-diagram/Process-Three.md | 20 + .../hub-dfd/flows/hub-diagram/Process-Two.md | 23 + test/fixtures/hub-dfd/groups/grants.md | 6 + test/fixtures/hub-dfd/groups/other.md | 6 + test/fixtures/hub-dfd/groups/records.md | 6 + test/fixtures/hub-dfd/ignatius.yml | 5 + test/fixtures/merge-tag-before-clusters.md | 99 ++ .../fixtures/subtype-no-basetype/data/Base.md | 16 + .../fixtures/subtype-no-basetype/data/SubA.md | 10 + .../fixtures/subtype-no-basetype/data/SubB.md | 10 + .../externals/Downstream.md | 5 + .../flows/diagram/Process-One.md | 14 + .../fixtures/subtype-no-basetype/ignatius.yml | 3 + test/visual/screenshot-store-clusters.ts | 224 +++ test/visual/test-cp3-dfd-url-navigability.ts | 51 +- 129 files changed, 8642 insertions(+), 918 deletions(-) create mode 100644 .claude/project/followups/dfd-store-clusters-f-15.md create mode 100644 docs/design/dfd-store-clusters.md create mode 100644 docs/spec/dfd-store-clusters.md create mode 100644 models/key-inherited/clusters/settlement.md create mode 100644 models/llm-memory-db-mssql/clusters/tag-junctions.md create mode 100644 src/app/components/flow-node/EdgeContractDialog.tsx create mode 100644 src/app/components/flow-node/StackDialog.tsx create mode 100644 src/flows/flow-clusters.ts create mode 100644 src/flows/flow-markdown.ts create mode 100644 test/checks/test-dfd-stack-dialogs.ts create mode 100644 test/checks/test-flow-chip-placement.ts create mode 100644 test/checks/test-flow-clusters-parse.ts create mode 100644 test/checks/test-flow-clusters-validate.ts create mode 100644 test/checks/test-flow-edge-labels.ts create mode 100644 test/checks/test-flow-view-grouping.ts create mode 100644 test/fixtures/broken-flows-model/clusters/role-grants-invalid.md create mode 100644 test/fixtures/broken-flows-model/clusters/role-grants.md create mode 100644 test/fixtures/broken-flows-model/flows/checkout/Bad-Cluster-Ref.md create mode 100644 test/fixtures/hub-dfd/clusters/grants.md create mode 100644 test/fixtures/hub-dfd/data/GrantAlpha.md create mode 100644 test/fixtures/hub-dfd/data/GrantBeta.md create mode 100644 test/fixtures/hub-dfd/data/HubStoreA.md create mode 100644 test/fixtures/hub-dfd/data/HubStoreB.md create mode 100644 test/fixtures/hub-dfd/data/LogStoreOne.md create mode 100644 test/fixtures/hub-dfd/data/LogStoreTwo.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore4Read.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore4Write.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore5Read.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore5Write.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore6Read.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore6Write.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore7Read.md create mode 100644 test/fixtures/hub-dfd/data/PrivateStore7Write.md create mode 100644 test/fixtures/hub-dfd/data/RecordBase.md create mode 100644 test/fixtures/hub-dfd/data/RecordTypeA.md create mode 100644 test/fixtures/hub-dfd/data/RecordTypeB.md create mode 100644 test/fixtures/hub-dfd/externals/Downstream.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-Eight.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-Five.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-Four.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-One.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-Seven.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-Six.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-Three.md create mode 100644 test/fixtures/hub-dfd/flows/hub-diagram/Process-Two.md create mode 100644 test/fixtures/hub-dfd/groups/grants.md create mode 100644 test/fixtures/hub-dfd/groups/other.md create mode 100644 test/fixtures/hub-dfd/groups/records.md create mode 100644 test/fixtures/hub-dfd/ignatius.yml create mode 100644 test/fixtures/merge-tag-before-clusters.md create mode 100644 test/fixtures/subtype-no-basetype/data/Base.md create mode 100644 test/fixtures/subtype-no-basetype/data/SubA.md create mode 100644 test/fixtures/subtype-no-basetype/data/SubB.md create mode 100644 test/fixtures/subtype-no-basetype/externals/Downstream.md create mode 100644 test/fixtures/subtype-no-basetype/flows/diagram/Process-One.md create mode 100644 test/fixtures/subtype-no-basetype/ignatius.yml create mode 100644 test/visual/screenshot-store-clusters.ts diff --git a/.claude/project/followups/INDEX.md b/.claude/project/followups/INDEX.md index d729a1f..d56520f 100644 --- a/.claude/project/followups/INDEX.md +++ b/.claude/project/followups/INDEX.md @@ -2,25 +2,26 @@ Auto-generated by `atomic followups render`. Do not edit. -Open: 9 โ€ข Stale: 0 โ€ข Last rendered: 2026-08-02 +Open: 10 โ€ข Stale: 6 โ€ข Last rendered: 2026-09-08 ## ๐Ÿ“‹ plans (1) - [usage-index-back-reference](usage-index-back-reference.md) โ€” Usage index: derived back-reference of store/entity touchpoints โ†’ docs/spec/process-flows.md -## ๐ŸŸก risks (4) +## ๐ŸŸก risks (5) -- [dfd-polish-round3-cytoscape-typing](dfd-polish-round3-cytoscape-typing.md) โ€” Type the cytoscape cy binding (retire the TS2339 Core defect) (54d) -- [dict-side-nav-scrollspy-timing](dict-side-nav-scrollspy-timing.md) โ€” Side-nav scrollspy test: scrollIntoView โ†’ waitForFunction implicit layout assumption (65d) -- [render-perf-elk-web-worker](render-perf-elk-web-worker.md) โ€” ELK web worker โ€” needs separate worker build entrypoint to activate under Bun (53d) -- [unified-app-polish-flow-modal-light-mode](unified-app-polish-flow-modal-light-mode.md) โ€” Flow-view dialogs may render dark in light mode (55d) +- [dfd-polish-round3-cytoscape-typing](dfd-polish-round3-cytoscape-typing.md) โ€” Type the cytoscape cy binding (retire the TS2339 Core defect) (91d, **stale**) +- [dfd-store-clusters-f-15](dfd-store-clusters-f-15.md) โ€” test-graph-search.ts intermittent stall in page.evaluate (0d) +- [dict-side-nav-scrollspy-timing](dict-side-nav-scrollspy-timing.md) โ€” Side-nav scrollspy test: scrollIntoView โ†’ waitForFunction implicit layout assumption (102d) +- [render-perf-elk-web-worker](render-perf-elk-web-worker.md) โ€” ELK web worker โ€” needs separate worker build entrypoint to activate under Bun (90d, **stale**) +- [unified-app-polish-flow-modal-light-mode](unified-app-polish-flow-modal-light-mode.md) โ€” Flow-view dialogs may render dark in light mode (92d, **stale**) ## ๐Ÿ”ต nits (4) -- [dfd-cold-deeplink-fallback](dfd-cold-deeplink-fallback.md) โ€” DFD cold deep-link (#view=flow&dfd=) falls back to context diagram on fresh load (48d) -- [parse-ts-preexisting-tsc-errors](parse-ts-preexisting-tsc-errors.md) โ€” Fix 6 pre-existing tsc errors in src/parse.ts (64d) -- [render-perf-flow-index](render-perf-flow-index.md) โ€” Index flow validate/parse O(n^2) lookups (findings must stay identical) (53d) -- [unified-app-polish-stack-entities-dfd](unified-app-polish-stack-entities-dfd.md) โ€” Stack entities in the DFD to reclaim visual real-estate (55d) +- [dfd-cold-deeplink-fallback](dfd-cold-deeplink-fallback.md) โ€” DFD cold deep-link (#view=flow&dfd=) falls back to context diagram on fresh load (85d, **stale**) +- [parse-ts-preexisting-tsc-errors](parse-ts-preexisting-tsc-errors.md) โ€” Fix 6 pre-existing tsc errors in src/parse.ts (101d) +- [render-perf-flow-index](render-perf-flow-index.md) โ€” Index flow validate/parse O(n^2) lookups (findings must stay identical) (90d, **stale**) +- [unified-app-polish-stack-entities-dfd](unified-app-polish-stack-entities-dfd.md) โ€” Stack entities in the DFD to reclaim visual real-estate (92d, **stale**) ## โ“ questions (0) diff --git a/.claude/project/followups/dfd-store-clusters-f-15.md b/.claude/project/followups/dfd-store-clusters-f-15.md new file mode 100644 index 0000000..921bd84 --- /dev/null +++ b/.claude/project/followups/dfd-store-clusters-f-15.md @@ -0,0 +1,14 @@ +--- +id: dfd-store-clusters-f-15 +title: test-graph-search.ts intermittent stall in page.evaluate +created: "2026-09-08" +origin: | + docs/spec/dfd-store-clusters.md, iter 9 implementer and reviewer (polish) +kind: finding +severity: risk +review_by: "2026-11-07" +status: open +file: test/checks/test-graph-search.ts +--- + +test-graph-search.ts stalled inside a page.evaluate on the graph mouseout handler in 2 of 20 instrumented back-to-back runs, after all three Enter cycles completed normally. The handler is synchronous and bounded; no code defect was found, and the stall did not reproduce in the reviewer's 8 runs. The check now carries bounded timeouts, a 55s watchdog, and a forced server stop, so it fails fast with a message instead of hanging. Left: find whether the stall is a CDP/GC hiccup under load or a real race in GraphView search, and remove the watchdog if the cause is fixed. diff --git a/.claude/rules/wiki/docs.md b/.claude/rules/wiki/docs.md index e2dfa07..3603315 100644 --- a/.claude/rules/wiki/docs.md +++ b/.claude/rules/wiki/docs.md @@ -8,7 +8,7 @@ paths: - "docs/glossary.md" --- -Domain: docs. Design docs, user guides, research notes, and implementation-contract specs โ€” 76 markdown files plus `docs/glossary.md` across `docs/design/`, `docs/guides/`, `docs/research/`, `docs/spec/` +Domain: docs. Design docs, user guides, research notes, and implementation-contract specs โ€” 78 markdown files plus `docs/glossary.md` across `docs/design/`, `docs/guides/`, `docs/research/`, `docs/spec/` Map: - docs/wiki/docs.md diff --git a/.claude/rules/wiki/flow-view.md b/.claude/rules/wiki/flow-view.md index 3599a6b..2f93f77 100644 --- a/.claude/rules/wiki/flow-view.md +++ b/.claude/rules/wiki/flow-view.md @@ -4,18 +4,20 @@ paths: - "src/flow-view/**" --- -Domain: flow-view. ELK-driven DFD layout (5-band partitioning, orthogonal edge routing); pure coord helpers for polyline rendering; SVG renderer consumes ELK positions + edgeRoutes + search-token dimming +Domain: flow-view. ELK-driven DFD layout (5-band partitioning, orthogonal edge routing) with a stack-node model for per-process/connected views and three collapse levels (stores/clusters/groups); pure coord helpers for polyline rendering; SVG renderer consumes ELK positions + edgeRoutes + search-token dimming Map: - docs/wiki/flow-view.md Contracts: - docs/spec/dfd-edge-hover-data.md - docs/spec/dfd-overhaul.md + - docs/spec/dfd-store-clusters.md - docs/spec/graph-flow-search.md - docs/spec/viewer-ux-polish.md Designs: - docs/design/dfd-edge-hover-data.md - docs/design/dfd-overhaul.md + - docs/design/dfd-store-clusters.md - docs/design/graph-flow-search.md Research: - docs/research/dfd-layout-and-leveling.md diff --git a/.claude/rules/wiki/flows.md b/.claude/rules/wiki/flows.md index bb9d929..d4581f4 100644 --- a/.claude/rules/wiki/flows.md +++ b/.claude/rules/wiki/flows.md @@ -4,18 +4,20 @@ paths: - "src/flows/**" --- -Domain: flows. SSADM data flow diagrams: `parseFlows` (recursive sub-DFDs + canonical Yourdon leveling via `deriveLevels`), `validateFlows` (12 `flow.*` rules), `buildFlowLayoutKeys`, usage indexing; role-split node model +Domain: flows. SSADM data flow diagrams: `parseFlows` (recursive sub-DFDs + canonical Yourdon leveling via `deriveLevels`), `flow-clusters.ts` `cluster:` token expansion from `clusters/.md` author files, `validateFlows` (17 `flow.*` rules), `buildFlowLayoutKeys`, usage indexing; role-split node model Map: - docs/wiki/flows.md Contracts: - docs/spec/dfd-nesting-depth.md - docs/spec/dfd-overhaul.md + - docs/spec/dfd-store-clusters.md - docs/spec/folder-model.md - docs/spec/process-flows.md Designs: - docs/design/dfd-nesting-depth.md - docs/design/dfd-overhaul.md + - docs/design/dfd-store-clusters.md - docs/design/folder-model.md - docs/design/process-flows.md Guides: diff --git a/.claude/rules/wiki/frontend.md b/.claude/rules/wiki/frontend.md index 21c2757..ba90af4 100644 --- a/.claude/rules/wiki/frontend.md +++ b/.claude/rules/wiki/frontend.md @@ -4,14 +4,17 @@ paths: - "src/app/**" --- -Domain: frontend. React 19 unified SPA (Graph/Dictionary/Flows views); shell (`App.tsx`) owns state + composition; views own cy/SVG lifecycle; components/logic/hooks/dom layered underneath +Domain: frontend. React 19 unified SPA (Graph/Dictionary/Flows views); shell (`App.tsx`) owns state + composition; `flowview=`/`collapse=` hash params drive the flow view mode and collapse level; `StackDialog`/`EdgeContractDialog` cover stack and edge-contract detail; views own cy/SVG lifecycle; components/logic/hooks/dom layered underneath Map: - docs/wiki/frontend.md +Contracts: + - docs/spec/dfd-store-clusters.md Designs: - docs/design/app-tsx-decomposition.md - docs/design/branding.md - docs/design/dd-spotlight-grid.md + - docs/design/dfd-store-clusters.md - docs/design/dict-navigation.md - docs/design/graph-flow-search.md - docs/design/graph-position-persistence.md diff --git a/.claude/rules/wiki/generators.md b/.claude/rules/wiki/generators.md index 31c9ba6..878f04c 100644 --- a/.claude/rules/wiki/generators.md +++ b/.claude/rules/wiki/generators.md @@ -4,7 +4,7 @@ paths: - "src/generators/**" --- -Domain: generators. Unified static HTML export via `generateApp` (single file โ€” graph + dict + flows); sole static generator +Domain: generators. Unified static HTML export via `generateApp` (single file โ€” graph + dict + flows), now embedding `window.__FLOW_CLUSTERS__`; sole static generator Map: - docs/wiki/generators.md diff --git a/.claude/rules/wiki/parser.md b/.claude/rules/wiki/parser.md index b15dadf..cbc7cbb 100644 --- a/.claude/rules/wiki/parser.md +++ b/.claude/rules/wiki/parser.md @@ -6,7 +6,7 @@ paths: - "src/model/model-index.ts" --- -Domain: parser. `ignatius.yml` config loading โ†’ ParseResult: {model, globalErrors}; nodes, edges, cardinality + classification derivation; wiki-link inline rule + two-pass body rendering; `buildModelIndex` โ€” 13 O(1) lookup maps built once per Model +Domain: parser. `ignatius.yml` config loading โ†’ ParseResult: {model, globalErrors}; nodes, edges, cardinality + classification derivation; wiki-link inline rule + two-pass body rendering; `ModelMeta.flowView` (`flow_view:`/`adjacency_stacks`); `buildModelIndex` โ€” 13 O(1) lookup maps built once per Model Map: - docs/wiki/parser.md diff --git a/.claude/rules/wiki/server.md b/.claude/rules/wiki/server.md index 9645aa8..08c92a0 100644 --- a/.claude/rules/wiki/server.md +++ b/.claude/rules/wiki/server.md @@ -4,7 +4,7 @@ paths: - "src/server/**" --- -Domain: server. Bun.serve with `/api/model` + `/api/flow` + `/events` SSE + fs.watch live-reload; `/dict` and `/flow` redirect to unified SPA hash routes; `/flow-dict` redirects to `/#view=dict` +Domain: server. Bun.serve with `/api/model` + `/api/flow` (now includes `clusters`) + `/events` SSE + fs.watch live-reload; `/dict` and `/flow` redirect to unified SPA hash routes; `/flow-dict` redirects to `/#view=dict` Map: - docs/wiki/server.md diff --git a/.claude/rules/wiki/validate.md b/.claude/rules/wiki/validate.md index b1451b5..3d212aa 100644 --- a/.claude/rules/wiki/validate.md +++ b/.claude/rules/wiki/validate.md @@ -4,13 +4,17 @@ paths: - "src/model/validate.ts" --- -Domain: validate. Pure model validator: 33 RuleIds across 8 prefixes (parse/config/entity/body/edge/cluster/index/flow), two severity tiers (A=warn, B=omit); `validateIndex` reuses `buildRouters` to detect router drift +Domain: validate. Pure model validator: 38 RuleIds across 8 prefixes (parse/config/entity/body/edge/cluster/index/flow), two severity tiers (A=warn, B=omit); `flow.*` now 17 ids including five `flow.cluster_*` DFD-store-cluster rules; `validateIndex` reuses `buildRouters` to detect router drift Map: - docs/wiki/validate.md Contracts: + - docs/spec/dfd-store-clusters.md + - docs/spec/model-index-routing.md - docs/spec/schema-lint-and-error-ux.md Designs: + - docs/design/dfd-store-clusters.md + - docs/design/model-index-routing.md - docs/design/schema-lint-and-error-ux.md Consult the map before changing behavior here. Behavior changes stale the pages above. Renames or removals stale mentions beyond them: grep the old name across docs/ before shipping. diff --git a/docs/design/dfd-store-clusters.md b/docs/design/dfd-store-clusters.md new file mode 100644 index 0000000..fde8de8 --- /dev/null +++ b/docs/design/dfd-store-clusters.md @@ -0,0 +1,299 @@ +# DFD store clusters + + +## Problem + + +A process that touches many tables draws one store box and one edge per table, and every edge carries a chip previewing its column list. On the Compute Loads diagram of the AutomaStructure model that is 9 processes, 22 stores, and 64 edges, each edge chip reading `project_id, Width, Eaโ€ฆ`. Half the edges sit on three hub stores: LoadInfo, BuildingInfo, and Girder. The parent diagram, design-building, has 10 processes, 79 stores, and 230 edges. The hover tooltip on one edge lists ten columns; the dictionary's process IO table lists one row per store per column. The diagram is correct and none of its surfaces are readable at that scale. + +Hand-drawn SSADM diagrams solve this three ways at once. Each process gets one stack of stores above it for reads and one below for writes, and a store used by two processes is drawn twice. Every flow carries a short prose label naming the data, never a column list. A store that fans out to many processes is repeated per process rather than wired across the page. ignatius has no way to name a flow, no way to say "these tables are one thing", and no way to draw a process's stores as one stack. + +Two earlier attempts inform this one. The ELK spike in `docs/spec/dfd-overhaul.md` tried compound nodes to cluster related stores and dropped them: every member kept its own node and edges inside the parent, and crossings went from 13 to 69. Externals already solved fan-out by contraction: `buildExternalRouting` in `src/flow-view/flow-layout.ts` merges every external into one source copy and one sink copy per diagram. Stacks apply that contraction to stores. + + +## Goals / Non-goals + + +- Goals: + - A prose `label:` on any input or output entry, shown on the edge chip in place of the column list. Clicking any chip that carries data opens a contract dialog listing which columns come from which store. + - A per-process view, the default, where each process's read set and write set render as one stack each, two processes with the identical set share a stack, and a store in two stacks is flagged as duplicated in the data without drawing the marker, since repetition is the rule in that view. The connected view, today's rendering with grouping applied, stays as the toggle and keeps drawing the duplicate marker, where a repeat is the exception. + - A collapse level, stores, clusters, or groups, that decides what one row of a stack stands for. A cluster is a subtype family from the entity model or an author-declared set in a `clusters/` file. A group is the entity's existing `group:`. + - A process file can reference a cluster directly with a `cluster:` entry, name the flow, and give each member's columns in one entry. + - Clicking a stack opens a dialog listing its rows; a row opens to its members; a member opens its entity dialog. + - In the connected view, adjacency stacking: stores wired to exactly the same processes in the same directions collapse into one honestly labelled stack. Switchable off in `ignatius.yml`. + - Validation, balancing, leveling, usage indexes, fingerprints, search, static export, and the modeling skill all understand labels, clusters, and stacks. +- Non-goals: + - Clusters of non-`db:` stores. Adjacency stacking still groups same-kind non-db stores; author clusters and the `cluster:` token are entity-only. + - Per-diagram cluster scoping. Cluster files live at the model root and apply wherever two or more members meet; a scoping key can be added later without changing the file shape. + - Per-diagram collapse level or view. Both are global view settings. + - Editing a cluster, group, or label from the viewer. + - ELK compound nodes or any change to the five-band layout contract. + - Cluster-level `examples:` rows. Examples stay keyed by member `db:` token. + - Changing the ERD, the dictionary's entity cards, subtype derivation, or the `groups/` format. + + +## Approaches + + +| # | Approach | Pros | Cons | +|---|----------|------|------| +| A | Render-time grouping only, no authoring change | Zero parser or validator work; every existing model improves on upgrade | No way to name a flow; no author control over what groups; the diagram shows something the markdown never says | +| B | `cluster:` as a first-class endpoint kind carried through every consumer | The parsed model matches the drawing exactly | Validator, balancing, leveling, usage index, fingerprint, search, and doc resolver all gain a cluster branch; balancing must compare cluster edges against member `db:` edges anyway | +| C | `cluster:` token expands to member `db:` edges at parse time, tagged with the cluster and flow label; stacks and rows are built at render time from the expanded graph | Every downstream consumer keeps seeing `db:` edges unchanged; author clusters, subtype families, groups, and adjacency share one render path; a model authored with or without the token has the same fingerprint | The parsed edge list no longer maps one-to-one onto frontmatter entries; the expansion pass is a new concept the validator must report against | +| D | ELK compound nodes | Native to the layout engine | Rejected by the overhaul spike: members keep their own edges, crossings multiply, bands collapse | + +Presentation forks, settled against the AutomaStructure model with an adjacency probe (`tmp/l1-probe.ts`) and the user's hand-drawn diagrams: + +| Fork | Options | Choice | +|------|---------|--------| +| Default view | per-process stacks ยท connected graph | per-process. On Compute Loads the connected view keeps about 48 of 64 edges after every grouping source, because 24 edges sit on hub stores no source can touch; per-process stacks are the only rendering that fits it, and they match how the user draws by hand | +| Adjacency stacking | drop ยท keep in connected view | keep, as an exploration. On the real model it lands on real families (the three MWFRS load case tables) and on coincidences (Column with OverhangInfo), so the stack label never names a member and the dialog shows the shared processes | +| Flow label scope | `cluster:` entries only ยท any entry | any entry. Hub-store edges dominate the dense diagrams and no grouping reaches them; the label is the fix for those | +| Grouping granularity | clusters only ยท clusters and groups | both, as one collapse-level dial. Groups already exist with labels and colors; a group contains clusters, so the levels nest | + +Visual decisions, recorded from the two options pages. First page: chose A2 (a store box with two offset outlines behind it, the same affordance a process with a sub-DFD uses), B2 (clicking the node opens a modal listing the members, the canvas never changes shape), C2 was chosen for the chip and then dropped: its member and column counts push the chip past the 22-character inline gate, so the chip shows the label alone and counts live in the dialog. D2 (the contract dialog is one flat table). Rejected: A1 a "C" cap, A3 a closed box with a title bar, B1 and B3 expand in place, C1 and C3 chip variants, D1 one section per table. Second page: chose B1 (externals stay aggregated in per-process view) over B2 (one external copy per process). Panel A resolved by rule: an explicit `cluster:` entry always draws its cluster, implicit grouping needs two members. + + +## Recommendation + + +Approach C, with per-process stacks as the default view. The contraction lives where the externals contraction already lives, in `buildFlowData`, and the parsed model stays a plain graph of `db:` edges that every existing rule and index already handles. + + +### Authoring + + +Any input or output entry may carry a `label:`. The chip shows it; without one the chip shows the column preview it shows today. The `data:` contract is unchanged and the contract dialog always lists the columns. + +```yaml +outputs: + - to: db:AuthEvent_AddRole + label: new auth event + data: [auth_event_id, app_user_id, role_id] +``` + +A cluster is a file in a new `clusters/` folder at the model root, beside `externals/` and `stores/`. It names a set of entities and explains why they are one thing. + +```markdown +--- +label: Role grants +entities: + - AppUser_Role + - AUR_Action + - AUR_CommunityAction +--- + +What a user holds once a role is granted: the role itself, its actions, +and its community-scoped actions. Written together, never separately. +``` + +A process file today lists every member with its columns: + +```yaml +--- +process: Grant Role to User +number: 1 +inputs: + - from: ext:App-Admin + data: granted role, community + - from: db:AppRole + data: [role_id] + - from: db:AppRole_Action + data: [role_id, action_id] + - from: db:AR_CommunityAction + data: [role_id, community_action_id] +outputs: + - to: db:AuthEvent_AddRole + data: [auth_event_id, app_user_id, role_id] + - to: db:AppUser_Role + data: [app_user_id, role_id] + - to: db:AUR_Action + data: [app_user_id, role_id, action_id] + - to: db:AUR_CommunityAction + data: [app_user_id, role_id, community_action_id] +--- +``` + +That file keeps working unchanged. Once `clusters/role-grants.md` exists and the collapse level is clusters or groups, the three output stores render as one row labelled Role grants. Rewriting the entry to the `cluster:` token is optional and buys one entry per cluster: + +```yaml +--- +process: Grant Role to User +number: 1 +inputs: + - from: ext:App-Admin + data: granted role, community + - from: cluster:role-definition + label: role id + data: + AppRole: [role_id] + AppRole_Action: [role_id, action_id] + AR_CommunityAction: [role_id, community_action_id] +outputs: + - to: db:AuthEvent_AddRole + label: new auth event + data: [auth_event_id, app_user_id, role_id] + - to: cluster:role-grants + label: added role + data: + AppUser_Role: [app_user_id, role_id] + AUR_Action: [app_user_id, role_id, action_id] + AUR_CommunityAction: [app_user_id, role_id, community_action_id] +--- +``` + +On a `cluster:` entry `data:` is a map from member entity to its column list, and every column is checked against that entity exactly as `flow.unknown_attribute` checks a `db:` entry. `label:` is the chip text; without it the chip falls back to the cluster's `label:`. A member listed in the cluster file but absent from the map is not part of this flow. A map with no members is a `flow.cluster_no_members` finding. + +A subtype family needs no file and no token. A process that reads the basetype and its subtypes: + +```yaml +inputs: + - from: db:Party + data: [party_id] + - from: db:Person + data: [party_id, first_name, last_name] + - from: db:Business + data: [party_id, legal_name] +``` + +renders at the clusters level as one row labelled Party, because `Party` owns a `subtypes:` cluster listing `Person` and `Business` in the entity model. When only subtypes are present, the row reads `BuildingPart subtypes` rather than naming a basetype that is not on the diagram. + +Groups need no new authoring either. The entity's `group:` and the `groups/.md` label and color are what the groups level uses. A group is coarser than a cluster: the identity group holds both the Party family and the Identification family, which are two clusters. `clusters/` exists beside `groups/` because a group is an ERD bucket and a cluster is a set that one process treats as one thing. + + +### Pipeline + + +The token is sugar. The parser expands it into member `db:` edges that carry a cluster tag and the flow label, so everything downstream sees the graph it sees today. + +```mermaid +flowchart LR + F[flows/*.md] --> P[parseFlows] + R[clusters/*.md] --> P + P --> X[expand cluster: entries
into db: edges tagged
cluster + label] + X --> L[deriveLevels] + L --> V[validateFlows
usage index
fingerprint
balancing] + L --> B[buildFlowData] + M[entity model
subtypeClusters, groups] --> B + B --> G[stacks per view
rows per collapse level] + G --> E[ELK layout] --> S[SVG renderer] +``` + +Two views produce different stack sets from the same expanded graph. Both are computed per diagram, per direction. + +``` +per-process view (default): + for each process P, for each direction: + members = db stores P touches in that direction + if |members| โ‰ฅ 2 โ†’ one stack node, id stack:-- + else โ†’ the plain store node, as today + two processes with the identical member set produce the same id and share the stack + a store present in two stacks in the same band is flagged duplicated; the per-process view + does not draw the marker (repetition is its rule), the connected view does + externals stay as buildExternalRouting draws them: one source copy, one sink copy + +connected view: + start from today's node model (one node per store, --read/--write split) + 0. edges from a cluster: entry group under that cluster, whatever their count + 1. author clusters: members = stores P touches โˆฉ C.entities, |members| โ‰ฅ 2 + 2. subtype families: members = stores P touches โˆฉ family, |members| โ‰ฅ 2 + 3. groups, only at the groups level: stores P touches sharing a group:, |members| โ‰ฅ 2 + 4. adjacency (when flow_view.adjacency_stacks is on): + signature(store) = (readers, writers, kind); |class| โ‰ฅ 2 โ†’ one stack + each step consumes the stores the previous one left; the rest stay plain + ids: cluster:, subtype:, group:, stack:, each --read or --write + a cluster or group node touched by several processes carries the union of the + members each process touches; the dialog says which member each process uses + a store grouped for one process and plain for another renders the plain copy + with the duplicate marker + +collapse level (stores | clusters | groups), applied to the rows of every stack: + stores โ†’ one row per table + clusters โ†’ rows: explicit cluster entries, then author clusters, then subtype + families (each needing โ‰ฅ 2 members in this stack), then loose tables + groups โ†’ rows: groups (โ‰ฅ 2 members in this stack) containing their clusters and + tables, then clusters spanning two groups, then loose tables + a lone table stays a table row at every level +``` + +The two-member threshold governs implicit grouping only; a `cluster:` entry gets its cluster row even for one member, showing the member count in parentheses, because the author asked for it. A plain `db:` entry that is the only member of a cluster or group present gets a plain row, because it asked for that. + +A stack edge aggregates its members' parsed edges. Its chip shows the members' authored labels one per line, then one column-preview line covering every unlabelled member, so a labelled cluster riding with two unlabelled tables reads `Tag junctions` over `tag_id, memory_id`; with no labelled member the chip is that single preview line. The hover tooltip lists one line per member with its columns, and the contract dialog lists every member's columns. + +Stack ids are member sets plus direction, so saved drag positions survive reloads and two processes with the same set converge on one node. Positions are stored per view, keyed by the diagram's layout fingerprint plus the view name, so toggling views never applies one view's drag to the other. Fingerprints are computed on the expanded parsed edges, so a diagram authored with `cluster:` tokens and the same diagram authored with `db:` entries share a fingerprint. + +The adjacency stack's label is `N stores`, never a member's name, and its dialog lists the reader and writer processes that produced it. A cluster or subtype row's cap reads `C` and a group row's cap reads `G`, never a D number, because those rows are not data stores; member store rows keep their D numbers. The "more inside" marker on such a row is the stacked-paper store from the hand-drawn convention: three sheets, the front one the row's own box drawn last, the two behind it offset down and right by 3px and 6px, filled the same colour as the box so they read as solid paper rather than outlines over the background, and showing their bottom edge, a short left stair segment between one sheet's bottom and the next, and the short top mark past the open right end, with no cap divider on the copies; the stack's left border and cap divider run per row, not through the strip; the stack reserves the offset plus a stroke width and two pixels after such a row so all three bottom edges stay distinct, except after the last row, where the box ends on the back sheet's bottom edge and draws no closing line of its own. + +Synthetic context and Level 1 diagrams from `deriveLevels` go through the same pipeline. Their edges carry no data, so an explicit cluster tag has nothing to survive on and the promoted stores group implicitly like any other. + +The view and the collapse level are global settings in `localStorage`, following the minimap toggle in `src/app/App.tsx`. `flow_view: { adjacency_stacks: false }` in `ignatius.yml` turns adjacency off for a model. + + +### Interaction + + +```mermaid +sequenceDiagram + actor U as User + participant C as Canvas + participant N as Stack dialog + participant K as Contract dialog + participant E as Entity dialog + U->>C: click a stack + C->>N: open (rows at the current collapse level) + U->>N: open a group or cluster row + N-->>U: its members, with D#s + U->>N: click a table + N->>E: open entity (existing open path) + U->>C: click an edge chip + C->>K: open (source โ†’ target, label, direction) + K-->>U: table: group ยท store ยท column ยท type + U->>K: click a store name + K->>E: open entity +``` + +The stack's โ“˜ badge opens the same dialog. An author cluster row renders the file's markdown body, a subtype row links the basetype, a group row shows the group's description, an adjacency stack shows its shared processes. Opening an entity from either dialog follows the existing flow-surface rule: the current dialog closes before the entity dialog opens, so Back returns to the diagram. The edge hover tooltip stays as it is; the contract dialog is a click, for every edge that carries data. On an `ext:` edge the table has the label lines and no group, store, or type. + +Search dimming keys off the base token of a node. A stack matches when any member matches. + + +### Validation + + +New `flow.*` rules, following the two-tier scheme in `src/flows/flow-validate.ts`: + +| Rule | Tier | Trigger | +|---|---|---| +| `flow.unknown_cluster` | B, strip the edge | `cluster:` with no `clusters/.md` | +| `flow.cluster_member_unknown` | B, strip that member | a `data:` map key that is not in the cluster's `entities:` | +| `flow.cluster_no_members` | A | a `cluster:` entry whose `data:` map is empty | +| `flow.cluster_entity_unknown` | A | a cluster file lists an entity that does not exist in the model | +| `flow.cluster_overlap` | A | an entity appears in two cluster files | +| `flow.unknown_attribute` | A, existing | a column in a `data:` map value not on that entity | + +The cluster registry rides in `FlowModel` beside `externals`, in the `/api/flow` payload, and in the static export payload, so the browser and the exported HTML group the same way. The `cluster:` prefix is intercepted before endpoint parsing, so the closed prefix set the guides describe stays closed for everything else, and those passages name the one exception. + + +### Proving cases + + +The AutomaStructure model is outside this repo, so the repo carries a fixture shaped like its Compute Loads diagram: at least eight processes, two hub stores every process reads, one subtype family, one author cluster, and two stores with the same adjacency. Success is measured there by edge count and by every chip being a label. A manual screenshot against the AutomaStructure model is the acceptance check. + + +### Surfaces + + +| Surface | Change | +|---|---| +| `docs/guides/flows.md` | a "Labels, stacks, clusters, and groups" section: `label:`, the two views, the collapse level, the `clusters/` folder, the token, adjacency and its switch, the two dialogs | +| `docs/guides/folder-format.md` | `clusters/` joins the recognized root folders; `groups/` notes its role in the flow view | +| `docs/guides/validation.md` | the five new rules | +| `docs/glossary.md` | stack, cluster, group, collapse level, per-process view, connected view | +| `skills/ignatius-modeling` | `dfd-authoring.md` gains a label step and a cluster step; `flow-templates.md` gains the cluster file and `label:`; `discover-flow.md` asks whether derived stores belong together; `verification.md` lists the rules | +| `docs/wiki/feature-map.md` | one row | + + +## Open questions + + +- None. diff --git a/docs/glossary.md b/docs/glossary.md index a8e71f2..c4191df 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -36,6 +36,21 @@ what they mean. When you write a spec, a commit, or a UI label, use these terms as a `db:` **DS** in a **DFD** (its role in a process). The DD describes it once; both views link back to it. +## Flow view: stacks, clusters, and groups + + +| Term | What it is | +|---|---| +| Flow label | A prose `label:` on any input or output entry. Shown on the edge chip in place of the column-list preview; `data:` is unchanged and still backs the contract dialog. | +| Cluster | A set of entities that one process treats as one thing. Two sources: an author cluster, a `clusters/.md` file at the model root naming its members; and a subtype family, the `subtypes:` a basetype entity already declares. A process can reference an author cluster directly with a `cluster:` token. | +| Group | The ERD bucket from `groups/.md`. Coarser than a cluster: one group can contain several clusters. In the flow view, the groups collapse level nests a group's clusters and stores under one row when a process touches two or more of them. | +| Stack | A rendered flow-view node standing in for two or more DS a process reads or writes together, drawn as one open-ended box with one row per store, cluster, or group. A cluster or group row carries a C or G cap and the stacked-paper marker: two sheets behind it, offset down and right, showing as lines under the row. Clicking it opens the stack dialog. | +| Collapse level | The global setting (`stores` / `clusters` / `groups`) deciding what one row of a stack represents. | +| Per-process view | The default flow-view rendering: one read stack and one write stack per process, built from the DS it touches in that direction. | +| Connected view | The flow-view toggle: one node per store, grouped per process and direction by cluster tag, author cluster, subtype family, group (groups level only), then adjacency. | +| Adjacency stack | A connected-view stack formed from stores wired to the same processes in the same directions, with no cluster or group in common. Labelled `N stores`, never a member's name. Switchable off with `flow_view: { adjacency_stacks: false }` in `ignatius.yml`. | + + ## Store kinds diff --git a/docs/guides/flows.md b/docs/guides/flows.md index e7c9ab4..0140c37 100644 --- a/docs/guides/flows.md +++ b/docs/guides/flows.md @@ -44,44 +44,75 @@ Frontmatter declares the data contract; the body explains the business. The flow --- process: Collect Payment number: 3 +description: "Settles an invoice by recording a payment and allocating it against invoice lines." inputs: - from: ext:Customer - data: payment details + label: payment details + data: card or account, amount, currency - from: db:PaymentMethod + label: stored payment method data: [party_id, payment_method_id, type, label] outputs: - - to: db:Payment - data: [party_id, payment_method_id, payment_id, amount] + - to: cluster:settlement + label: settled payment + data: + Payment: [party_id, payment_method_id, payment_id, amount] + PaymentAllocation: [party_id, payment_method_id, payment_id, sales_invoice_id, line_seq] - to: file:gateway-log + label: gateway response data: gateway transaction reference, HTTP status, raw response - to: ext:Customer - data: receipt + label: receipt + data: payment id, status, message examples: in: - from: ext:Customer label: payment details rows: - { card: "****4242", amount: 49.99, currency: GBP } + - { card: "****1234", amount: 199.00, currency: USD } + - from: db:PaymentMethod + label: stored card lookup + rows: + - { party_id: 1001, payment_method_id: 42, type: card, label: "Visa ending 4242" } out: - to: db:Payment label: settled payment record rows: - { party_id: 1001, payment_method_id: 42, payment_id: 9001, amount: 49.99 } + - { party_id: 1002, payment_method_id: 17, payment_id: 9002, amount: 199.00 } + - to: db:PaymentAllocation + label: allocation against the invoice line + rows: + - { party_id: 1001, payment_method_id: 42, payment_id: 9001, sales_invoice_id: 5001, line_seq: 1 } + - to: ext:Customer + label: receipt + rows: + - { payment_id: 9001, status: captured, message: "Payment accepted" } --- -Settles an invoice by recording a [[Payment]] and allocating it against the -invoice line it pays. A receipt is returned to the [[Customer]]. +Settles an invoice by recording a [[Payment]] and allocating it. + +Reads the customer's stored [[PaymentMethod]] (its `type` and `label`, e.g. +"Visa ending 4242"), records the `Payment` (`amount` must be positive), +then writes a [[PaymentAllocation]] linking that payment to the invoice line +it settles, both in one transaction. A receipt is returned to the [[Customer]]. + +This process is the reason `PaymentAllocation` is a five-part key: the +allocation is uniquely identified by the paying party, the method, the +payment, and the specific invoice line โ€” every column this flow writes is +part of that key. ``` | Field | Required | Meaning | |---|---|---| | `process` | yes | The human label shown on the node | | `number` | no | Local rank among sibling processes; falls back to file order. Full SSADM numbers (`1.2.1`) are composed from the folder nesting automatically | -| `inputs` / `outputs` | yes | The flows. Each names an endpoint (`from:`/`to:`) and the `data:` it carries | +| `inputs` / `outputs` | yes | The flows. Each names an endpoint (`from:`/`to:`), the `data:` it carries, and a prose `label:` the diagram shows for it | | `examples` | no | Sample in/out rows rendered as tables in the process dialog, one entry per flow | | `description` | no | One line saying what the process does and when it runs. It is the Description cell in the flow folder's generated router, so a reader can open or skip the process without reading its body | -The `data:` field is the flow's label and its contract. On a `db:` endpoint it is **always column names** โ€” a string for one column, a list for several โ€” and every name is checked against the entity's `pk` and `columns` (the `flow.unknown_attribute` rule). On any other endpoint it is an opaque label; make it enumerate everything the flow carries rather than a one-word summary. +The `data:` field is the flow's contract. On a `db:` endpoint it is **always column names** โ€” a string for one column, a list for several โ€” and every name is checked against the entity's `pk` and `columns` (the `flow.unknown_attribute` rule). On a `cluster:` endpoint it is a map from member entity to that entity's column list, checked the same way. On any other endpoint it is a phrase; make it enumerate everything the flow carries rather than a one-word summary. The `label:` is what the diagram shows on the edge chip; the contract sits one click behind it (see [Labels, stacks, clusters, and groups](#labels-stacks-clusters-and-groups)). Bodies support the same `[[Entity]]` wiki-links as entity files, and they can also link to processes, externals, and stores by name. Links open the target's dialog in place. @@ -100,6 +131,8 @@ Every flow connects a process to something. The endpoint token's prefix says wha This prefix set is closed. A store that fits none of the named kinds is authored as `other:` โ€” there is no way to invent a new prefix. A bare, unprefixed name resolves only when it is unambiguous across all namespaces; otherwise the `flow.ambiguous_endpoint` rule asks you to qualify it. +`cluster:` is the one exception: it is intercepted and expanded before endpoint parsing ever runs, so it never competes with the closed set above. See [Labels, stacks, clusters, and groups](#labels-stacks-clusters-and-groups). + ### Externals @@ -134,14 +167,111 @@ A process decomposes by placing a folder with the process's exact file name next The child diagram must be *balanced* with its parent: the data crossing the sub-DFD's boundary has to match the parent process's declared `inputs:` and `outputs:`, column for column on `db:` flows. The `flow.unbalanced_decomposition` rule checks this at every level. In the viewer, a process with a sub-DFD renders with a stacked-shadow affordance; clicking through drills down, and a breadcrumb trail leads back up. +## Labels, stacks, clusters, and groups + + +A process that touches many stores draws one edge per store, and by default each edge chip previews the store's column list. That is useful for one or two edges, unreadable across a dozen. Two features address the density: a prose `label:` that replaces the column preview on any edge, and a stacking system that draws a process's stores as one shape instead of many. + + +### Labels and the contract dialog + + +Any input or output entry may carry a `label:`, a short prose name for the flow, shown on the edge chip in place of the column-list preview. One line per `, `-separated item. An empty or whitespace-only `label:` counts as absent, and the chip falls back to today's column preview under the 22-character inline gate. `data:` is unchanged either way: it is still the flow's contract and still validated (`flow.unknown_attribute`). + +```yaml +outputs: + - to: db:AuthEvent_AddRole + label: new auth event + data: [auth_event_id, app_user_id, role_id] +``` + +Clicking any chip that carries data opens a contract dialog: a table of group, store, column, and type (group is the entity's group badge, blank when none), sorted by store then by the entity's own column order. The dialog header shows the route (source โ†’ target). An `ext:` edge has no group, store, or type to show, so its dialog is a one-column table of the label lines instead. Hovering an edge still shows the existing tooltip; on a stack edge the tooltip lists one `Store: col, col` line per member. + + +### Clusters: naming a set of stores as one thing + + +A cluster is a file in `clusters/.md` at the model root, beside `externals/` and `stores/`. It names a set of entities and explains why they belong together: + +```markdown +--- +label: Role grants +entities: + - AppUser_Role + - AUR_Action + - AUR_CommunityAction +--- + +What a user holds once a role is granted: the role itself, its actions, and its +community-scoped actions. Written together, never separately. +``` + +A process entry can reference the cluster directly with a `cluster:` token instead of separate `db:` entries. `data:` becomes a map from member entity id to its column list, plus an optional `label:` for the chip: + +```yaml +outputs: + - to: cluster:role-grants + label: added role + data: + AppUser_Role: [app_user_id, role_id] + AUR_Action: [app_user_id, role_id, action_id] + AUR_CommunityAction: [app_user_id, role_id, community_action_id] +``` + +The token is sugar: at parse time it expands into one `db:` edge per mapped member, tagged with the cluster. Fingerprints, balancing, usage indexes, and examples all see the expanded `db:` edges and are unaffected: a diagram authored with the token and the same diagram authored with plain `db:` entries share a fingerprint. `cluster:` is the one exception carved out of the closed endpoint-prefix set: it is intercepted before ordinary endpoint parsing, so the rest of that set stays closed. + +A cluster is not the only way to group stores. A subtype family declared in the entity model's `subtypes:` frontmatter needs no `clusters/` file at all: a process that reads a basetype and its subtypes groups automatically at the clusters or groups collapse level. A **group** is a different, coarser thing: the ERD bucket from `groups/.md`. A group can hold several clusters, and the entity's existing `group:` field is what the groups collapse level groups by. + + +### Two views: per-process and connected + + +The view is a single global setting, persisted in `localStorage` under `ignatius-flow-view` and deep-linked as `flowview=` in the hash beside `dfd=`; it is restored on Back/Forward. + +- **Per-process view (default).** Each process draws one read stack above it and one write stack below it, built from the stores it touches in that direction. A store set of two or more members becomes a stack; a single store stays a plain box. Two processes with the identical store set share one stack. A shared store repeats in every stack it belongs to by design here, so the duplicate marker is not drawn in this view (see the connected view below, where a duplicate is the exception). Externals are unaffected: still one source copy and one sink copy per diagram. +- **Connected view.** One node per store, grouped per process and direction, in this order: explicit `cluster:` entries (any member count), author clusters (two or more members touched by that process), subtype families from `subtypes:` (two or more), entity groups from `group:` (only at the groups collapse level, two or more), then adjacency. A store grouped for one process and touched alone by a different process renders a plain, duplicate-marked copy for that second process. + +Adjacency groups stores wired to the same processes in the same directions, on by default; switch it off with `flow_view: { adjacency_stacks: false }` in `ignatius.yml`. + + +### Collapse level: what a stack row stands for + + +The collapse level is a second global setting (`localStorage` key `ignatius-flow-collapse`, hash param `collapse=`, default `clusters`). It decides what one row of a stack represents: + +| Level | A stack row is | +|---|---| +| `stores` | one table | +| `clusters` | one author cluster or subtype family with two or more members in the stack, then loose tables | +| `groups` | one group with two or more members (containing its cluster rows and tables), then clusters spanning two groups, then loose tables | + +A lone table is a table row at every level. + + +### Stacks and the stack dialog + + +A stack draws visible rows: store rows carry their own D# cap; cluster and subtype rows cap with `C`, group rows with `G` (never a D#, since only a store row is one) alongside the label, a `(N)` count, and a peek marker meaning more sit inside. A subtype row reads the basetype name when the basetype is among the touched stores, or ` subtypes` when it is not. An adjacency stack is labelled `N stores`, never a member's name. Every stack carries one โ“˜ badge. + +Clicking a stack opens the stack dialog. Its title depends on the stack's source (a cluster's `label:`, the basetype name, the group's `label:`, `N stores` for adjacency, or `Read stack` / `Write stack` for a plain per-process stack), followed by the feeding processes and the rows at the current collapse level. Opening a row expands to its members; opening a member opens its entity or doc dialog. An author cluster row also shows the `clusters/.md` file's body; a subtype row links the basetype; a group row shows the group's description; an adjacency row lists its reader and writer processes. + +Stack edge chips show the members' authored labels one per line, then one column-preview line covering every unlabelled member; if nothing is labelled, one preview line for the whole stack. Drag positions are saved per view, so a drag in the per-process view never applies to the connected view. + + +### Controls + + +The Flows FAB menu carries three flow-specific controls: a view item labelled by its destination (`Connected view` when you're in per-process, `Per-process view` when you're in connected), a collapse item worded as an action (`Collapse to clusters`, `Collapse to groups`, `Expand to stores`), and `Copy link`. + + ## Viewing flows -`ignatius serve` shows flows in the **Flows** view (`#view=flow`); the active diagram is deep-linkable via the `dfd=` hash parameter and survives refresh. `ignatius export` includes the Flows view in the same single HTML file. Every node carries a โ“˜ badge: a `db:` store opens the rich entity dialog, everything else opens its markdown doc. The process dictionary โ€” every process, external, and store with its body and IO tables โ€” is fused into the **Dictionary** view, searchable alongside the entities. +`ignatius serve` shows flows in the **Flows** view (`#view=flow`); the active diagram is deep-linkable via the `dfd=` hash parameter and survives refresh, alongside `flowview=` and `collapse=` for the view and collapse-level settings above. `ignatius export` includes the Flows view in the same single HTML file. Every node carries a โ“˜ badge: a `db:` store opens the rich entity dialog, everything else opens its markdown doc. The process dictionary โ€” every process, external, and store with its body and IO tables โ€” is fused into the **Dictionary** view, searchable alongside the entities. -`ignatius validate` checks flows whenever a `flows/` directory exists, with eleven `flow.*` rules covering unknown references, column contracts, connection shape, numbering, and decomposition balance. See [Validation and findings](validation.md#flow-rules) for the catalog. One rule is configurable: direct process-to-process flows warn by default and can be silenced with `flow_rules: { process_to_process: false }` in `ignatius.yml`. +`ignatius validate` checks flows whenever a `flows/` directory exists, with seventeen `flow.*` rules covering unknown references, column contracts, connection shape, numbering, decomposition balance, and cluster references. See [Validation and findings](validation.md#flow-rules) for the catalog. One rule is configurable: direct process-to-process flows warn by default and can be silenced with `flow_rules: { process_to_process: false }` in `ignatius.yml`. -Hovering a data flow edge that carries data (the arrow between two nodes) reveals a styled tooltip listing the full data carried across it, under a `source โ†’ target` header. This includes the complete contents of `db:` column lists that are otherwise abbreviated on the canvas when they exceed the inline-label length limit. The tooltip is positioned fixed to the viewport and remains legible at any zoom level. Long data labels (more than 22 characters) show a truncated `โ€ฆ` preview on the canvas โ€” the first ~22 characters followed by `โ€ฆ` โ€” so you can always see at a glance which edges carry hidden data; the full contents are revealed on hover. +Hovering a data flow edge that carries data (the arrow between two nodes) reveals a styled tooltip listing the full data carried across it, under a `source โ†’ target` header. This includes the complete contents of `db:` column lists that are otherwise abbreviated on the canvas when they exceed the inline-label length limit. The tooltip is positioned fixed to the viewport and remains legible at any zoom level. Long data labels (more than 22 characters) show a truncated `โ€ฆ` preview on the canvas โ€” the first ~22 characters followed by `โ€ฆ` โ€” so you can always see at a glance which edges carry hidden data; the full contents are revealed on hover. An authored `label:` replaces this preview outright, on a plain edge and on a stack edge alike. ## Authoring with the skill diff --git a/docs/guides/folder-format.md b/docs/guides/folder-format.md index 1f41154..b268ff1 100644 --- a/docs/guides/folder-format.md +++ b/docs/guides/folder-format.md @@ -1,7 +1,7 @@ # The folder format -A folder is a model root when it contains an `ignatius.yml` file. Five top-level folders are recognized โ€” `data/`, `flows/`, `groups/`, `externals/`, and `stores/`. Everything else at the root is free-form: notes, scratch files, or any other markdown you keep nearby are never scanned. +A folder is a model root when it contains an `ignatius.yml` file. Six top-level folders are recognized โ€” `data/`, `flows/`, `groups/`, `clusters/`, `externals/`, and `stores/`. Everything else at the root is free-form: notes, scratch files, or any other markdown you keep nearby are never scanned. ``` models/ @@ -17,6 +17,8 @@ models/ groups/ identity.md transactional.md + clusters/ # optional: author-declared sets of stores for the flow view + role-grants.md externals/ # optional โ€” shared external definitions for DFDs stores/ # optional โ€” shared non-db store definitions for DFDs flows/ # optional โ€” data flow diagrams @@ -25,6 +27,21 @@ models/ Entities live under `data//.md`. The group name comes from the entity's `group:` frontmatter field โ€” the subdirectory path under `data/` is just a convenience for organization; the parser uses the frontmatter value, not the folder name. `groups/` is optional: a model with no `groups/` directory parses with zero groups and no error. `externals/` and `stores/` are the global registries for DFD authoring (see [Process flows](flows.md)). +`clusters/` is also optional and DFD-only: each file names a set of entities that a process's flows can reference as one thing with a `cluster:` token. With no token at all, a cluster still groups: whenever a process touches two or more of its members, the flow view draws them as one row, at every collapse level. At the groups collapse level, a group additionally nests its own clusters and stores under one group row. A minimal cluster file: + +```markdown +--- +label: Role grants +entities: + - AppUser_Role + - AUR_Action +--- + +What a user holds once a role is granted. +``` + +See [Labels, stacks, clusters, and groups](flows.md#labels-stacks-clusters-and-groups) for the authoring format and how clusters render. + ## ignatius.yml @@ -249,6 +266,8 @@ Party identity, classifications, and ID documents. An entity whose `group` references a name with no matching `groups/.md` file renders without a color band and is flagged with an `entity.unknown_group` warning. +In the flow view's groups collapse level, a group is also a stacking bucket: stores sharing a `group:` that a process touches (two or more) nest under one group row, containing that group's own cluster rows and loose tables. See [Labels, stacks, clusters, and groups](flows.md#labels-stacks-clusters-and-groups). + ## The flows folder diff --git a/docs/guides/modeling-skill.md b/docs/guides/modeling-skill.md index e5a3a45..2a4ce5e 100644 --- a/docs/guides/modeling-skill.md +++ b/docs/guides/modeling-skill.md @@ -47,7 +47,7 @@ This adds `ignatius-modeling` to the project's `.claude/skills/`. Add `-g` to in ## flow vs discover -`flow` and `discover` are two doors into the same artifacts. Pick `flow` when you can already name your processes โ€” it walks the structure step by step: processes as verbs, externals, the `db:`-or-`kind:` store decision, the data each flow carries, sample rows, and the business narrative. Pick `discover` when you know what the business does but have not decomposed it yet โ€” the skill interviews you in plain language, derives the entities your processes require, writes those first, then writes the flows that reference them. When a real system already exists (a database, a schema dump, a codebase, an API), `discover` reads it instead of interviewing, then walks you through the judgment calls. +`flow` and `discover` are two doors into the same artifacts. Pick `flow` when you can already name your processes โ€” it walks the structure step by step: processes as verbs, externals, the `db:`-or-`kind:` store decision, which stores travel together as a cluster, the data each flow carries and the label the diagram shows for it, sample rows, and the business narrative. Pick `discover` when you know what the business does but have not decomposed it yet โ€” the skill interviews you in plain language, derives the entities your processes require, writes those first, then writes the flows that reference them. When a real system already exists (a database, a schema dump, a codebase, an API), `discover` reads it instead of interviewing, then walks you through the judgment calls. Both modes always produce example data โ€” every entity gets sample rows and every process gets in/out example tables โ€” because concrete instances expose wrong rules that pass every structural check. diff --git a/docs/guides/validation.md b/docs/guides/validation.md index 6d3ecc4..eb1f4fe 100644 --- a/docs/guides/validation.md +++ b/docs/guides/validation.md @@ -99,6 +99,11 @@ These run whenever the model has a `flows/` directory (see [Process flows](flows | `flow.process_no_output` | A | A process has no output flows. | | `flow.duplicate_number` | A | Two sibling processes declare the same `number:`. | | `flow.unbalanced_decomposition` | A | A sub-DFD's boundary flows do not match the parent process's declared inputs and outputs. | +| `flow.unknown_cluster` | B | A `cluster:` endpoint names a slug with no `clusters/.md` file at the model root. Every edge expanded from this entry is stripped from the cleaned model. Add the cluster file or correct the slug. | +| `flow.cluster_member_unknown` | B | A `cluster:` entry's `data:` map names a member entity that is not in the cluster's `entities:` list. The edge for that member is stripped from the cleaned model. Add the entity to the cluster's `entities:` or correct the member key. | +| `flow.cluster_no_members` | A | A `cluster:` entry's `data:` map is empty โ€” no members were mapped. The entry produces no edge in the cleaned model. Add member columns to the `data:` map or remove the entry. | +| `flow.cluster_entity_unknown` | A | A `clusters/.md` file's `entities:` list names an entity that does not exist in the entity catalog. Add the entity file or correct the name in the cluster file. | +| `flow.cluster_overlap` | A | An entity appears in the `entities:` list of more than one `clusters/*.md` file. A store belongs to only one author cluster; remove it from all but one file. | ### Config rules diff --git a/docs/spec/dfd-store-clusters.md b/docs/spec/dfd-store-clusters.md new file mode 100644 index 0000000..cf622fb --- /dev/null +++ b/docs/spec/dfd-store-clusters.md @@ -0,0 +1,391 @@ +# DFD store clusters + + +## Goal + +Every input/output entry may carry a prose `label:`, shown on the edge chip in place of a column-list preview, with a contract dialog on every data-carrying edge. The default rendering becomes a per-process view โ€” one read-stack and one write-stack per process โ€” with a connected view (today's per-store rendering, plus cluster/subtype/group/adjacency grouping) as a global toggle, and a collapse level (stores/clusters/groups) deciding what a stack's rows stand for. This targets the density the AutomaStructure model exposes: its Compute Loads diagram is 9 processes, 22 stores, and 64 edges, and its parent design-building diagram is 10 processes, 79 stores, and 230 edges โ€” every edge chip a truncated column list today. + + +## Non-goals + +- Clusters of non-`db:` stores. Adjacency stacking still groups same-kind non-db stores; author clusters and the `cluster:` token are entity-only. +- Per-diagram cluster scoping. Cluster files live at the model root and apply wherever two or more members meet on any diagram. +- Per-diagram collapse level or view. Both the view (per-process/connected) and the collapse level (stores/clusters/groups) are global settings. +- Editing a cluster, a group, or a label from the viewer. +- ELK compound nodes or any change to the five-band layout contract. +- Cluster-level `examples:` rows. Examples stay keyed by member `db:` token. +- Changing the ERD, the dictionary's entity cards, subtype derivation, or the `groups/` format. +- Changing `flow-derive-levels.ts`. Synthetic context and Level 1 diagrams run through the same render-time pipeline as any leaf diagram; their edges carry no data (`data: ''`), so an explicit `cluster:` tag has nothing to survive promotion on, and promoted stores group implicitly like any other store. + + +## Success criteria + +- [ ] Chips are labels everywhere: on `test/fixtures/hub-dfd`'s `hub-diagram` (see Change tree โ€” modeled on AutomaStructure's Compute Loads: 8 processes, 17 stores, 31 edges, including two hub stores `HubStoreA`/`HubStoreB` read by every process, a 3-member subtype family, a 2-member author cluster, and a 2-member same-adjacency pair), every rendered edge chip is a `label:` string โ€” none show a truncated column-list preview โ€” and clicking any data-carrying edge opens a contract dialog listing every column with its store and type. +- [ ] Default per-process view: on `hub-diagram`, the 31 authored edges collapse to 13 rendered edges across 12 distinct store-side nodes. Each of the 8 processes renders one read-stack (`stack:--read`) containing that process's full read set, including `HubStoreA`/`HubStoreB`; the two processes (of the 8) whose read set is only the two hub stores share exactly one stack node; a process whose read set also includes a private store gets its own distinct stack; `HubStoreA` is flagged duplicated everywhere it appears in more than one distinct read-stack, and the per-process view draws no duplicate marker (the connected view still does); a process that writes only one store (below the 2-member stacking threshold) still renders that write as a plain node. Externals keep their existing one-source-copy/one-sink-copy rendering, unaffected by the view. +- [ ] Connected view + collapse level: toggling to the connected view on `hub-diagram`, at collapse level "clusters" the author cluster and the subtype family each render as one row inside whichever stack contains their members; at collapse level "groups", the same members nest under a group row when both members share an entity `group:`; at collapse level "stores" every table is its own row, with no cluster or group nesting. +- [ ] Adjacency stacking (connected view, default on): `hub-diagram`'s two same-adjacency stores โ€” written only by the same one process, read by none, same `kind` โ€” group under one `stack:--write` node labelled `2 stores` (never a member's name); its dialog lists the shared writer process (no readers). Setting `flow_view: { adjacency_stacks: false }` in `hub-dfd/ignatius.yml` and re-rendering leaves both as plain, unstacked store nodes. +- [ ] Subtype-family label rule: `hub-diagram`'s subtype-family stack row reads ` subtypes` when the basetype itself is not among the touched stores, and reads the plain basetype name when it is. +- [ ] `models/llm-memory-db-mssql`'s `tag-administration`: the edge from Merge Tag carrying the `clusters/tag-junctions.md`-tagged output entry (Project_Tag, Artifact_Tag, Milestone_Tag, Task_Tag) renders a `Tag junctions` label chip โ€” not a column-list preview โ€” and clicking it opens the contract dialog listing all four stores with their columns and types. +- [ ] `ignatius validate models/llm-memory-db-mssql` and `ignatius validate models/key-inherited` both exit 0 with 0 new findings after `clusters/tag-junctions.md` and the fixture changes land. +- [ ] On `models/key-inherited`, a process reading `db:Party` together with `db:Person` and `db:Business` renders one collapsed row labelled Party (connected view, clusters or groups level), with no `clusters/` file and no change to `Party.md`'s `subtypes:` frontmatter. +- [ ] Validation, five rules, two fixtures: `test/fixtures/broken-flows-model` gains a process file whose `cluster:` entries fire `flow.unknown_cluster` (a slug with no `clusters/.md`), `flow.cluster_member_unknown` (a `data:` map key outside the cluster's `entities:`), and `flow.cluster_no_members` (an empty `data:` map) โ€” the three rules a cluster *file* cannot fire. `test/fixtures/broken-flows-model/clusters/` gains a second cluster file that fires `flow.cluster_entity_unknown` (names an entity absent from the model) and `flow.cluster_overlap` (lists an entity already claimed by another cluster file). +- [ ] Node-id and fingerprint stability: rewriting Merge Tag's four `db:` output entries to the equivalent `cluster:tag-junctions` token yields the same `layoutFlowFingerprint` and the same node id (`cluster:tag-junctions--write`) as the unrewritten `db:` form, so a saved drag position on that diagram survives the rewrite. Adding a `label:` to any entry, with no other change, does not change `layoutFlowFingerprint`. +- [ ] Positions are stored per view: dragging a node in the per-process view and then toggling to the connected view on the same diagram does not apply the per-process drag to the connected-view layout, and vice versa โ€” the persistence key is the diagram's layout fingerprint plus the active view name. +- [ ] Dialogs follow the existing flow-surface open rule: opening `StackDialog` or the contract dialog, then clicking through to an entity, closes the currently-open dialog before the entity dialog opens (the same rule `FlowSurface`'s `open` handler already applies to the doc dialog in `src/app/views/flow/FlowsView.tsx`). +- [ ] The dictionary's process IO table (`IoTable.tsx`) shows an entry's `label:` in place of its column list whenever one is present, matching the canvas chip. +- [ ] A stack edge whose members mix labelled and unlabelled entries shows the authored labels one per line followed by exactly one column-preview line covering every unlabelled member (Merge Tag's per-process write chip on `tag-administration` reads `Tag junctions` / `tag_id, memory_id`); a stack edge with no labelled member shows one preview line; the hover tooltip on any stack edge lists one `Store: col, col` line per member. +- [ ] `bun run test` passes (all `test/checks/*.ts`, exit 0), including new checks for labels, cluster parsing/expansion/validation, per-process and connected-view grouping, and the collapse level. `bun run build:cli` succeeds. Touched files introduce zero new `tsc --noEmit` errors vs. `bun run typecheck` baseline. +- [ ] `docs/guides/flows.md`, `docs/guides/folder-format.md`, `docs/guides/validation.md`, `docs/glossary.md`, `skills/ignatius-modeling/references/{dfd-authoring,flow-templates,discover-flow,verification}.md`, and `docs/wiki/feature-map.md` all describe labels, the two views, the collapse level, clusters, groups, and adjacency โ€” including the one-sentence exception the `cluster:` prefix carves out of the otherwise-closed endpoint-prefix set. + + +## Approach + +Approach C: the `cluster:` token expands to member `db:` edges at parse time; per-process stacks are the default render, with a connected view (grouping the expanded graph by cluster tag, author cluster, subtype family, group, and adjacency) as the toggle โ€” see `docs/design/dfd-store-clusters.md`. + + +## Change tree + +``` +src/flows/ +โ”œโ”€โ”€ flow-clusters.ts .......... A (FlowCluster, parseClusters, expandClusterEdges) +โ”œโ”€โ”€ flow-markdown.ts .......... A (shared frontmatter parse, markdown renderer, label normalisation) +โ”œโ”€โ”€ flow-parse.ts ............. M (label: on any input/output entry; FlowModel.clusters; +โ”‚ FlowEdge.cluster tag and clusterIssue marker; cluster: +โ”‚ intercepted before parseEndpoint) +โ””โ”€โ”€ flow-validate.ts .......... M (flow.unknown_cluster, flow.cluster_member_unknown, + flow.cluster_no_members, flow.cluster_entity_unknown, + flow.cluster_overlap) +src/model/ +โ”œโ”€โ”€ parse.ts ................... M (ModelMeta.flowView from ignatius.yml's flow_view: block) +โ””โ”€โ”€ validate.ts ................. M (RuleId union + explanation text for the five new flow.cluster_* rules) +src/flow-view/ +โ”œโ”€โ”€ flow-layout.ts ............. M (edge label falls back to the column preview; buildFlowData +โ”‚ opts: view/collapseLevel/clusters/subtypeClusters/groups; +โ”‚ buildPerProcessStacks; buildConnectedViewGrouping; row +โ”‚ breakdown per collapse level; stack/cluster/subtype/group/ +โ”‚ adjacency node ids) +โ”œโ”€โ”€ elk-flow-layout.ts ......... M (nodeSize/bandOf handle stack nodes; computeElkLayout opts) +โ””โ”€โ”€ FlowDiagramSvg.tsx ......... M (stacked StoreNode variant; StackDialog trigger; chip shows + label only; contract dialog trigger on every data edge; + search dimming over stack membership) +src/app/ +โ”œโ”€โ”€ App.tsx .................... M (view + collapse-level state, localStorage persistence, hash writes) +โ”œโ”€โ”€ globals.d.ts ............... M (window.__FLOW_CLUSTERS__) +โ”œโ”€โ”€ hash-router.ts ............. M (flowview= and collapse= params; nextCollapseLevel) +โ”œโ”€โ”€ hooks/useHashRoute.ts ....... M (popstate reconcile for flowview= and collapse=) +โ”œโ”€โ”€ hooks/useModelData.ts ....... M (thread clusters through the live/static payload) +โ”œโ”€โ”€ components/ui/FabMenu.tsx ... M (view item, collapse item, Copy link in the flow branch) +โ”œโ”€โ”€ views/flow/FlowsView.tsx .... M (thread view/collapseLevel/clusters/subtypeClusters/groups +โ”‚ into computeElkLayout + FlowDiagramSvg; per-view position +โ”‚ key; StackDialog/contract-dialog open state, reusing the +โ”‚ existing dialog-close-before-entity-open rule) +โ””โ”€โ”€ components/ + โ”œโ”€โ”€ flow-node/ + โ”‚ โ”œโ”€โ”€ StackDialog.tsx ..... A (rows at the current collapse level; row โ†’ members โ†’ entity) + โ”‚ โ””โ”€โ”€ EdgeContractDialog.tsx A (group ยท store ยท column ยท type table) + โ””โ”€โ”€ process/IoTable.tsx ...... M (shows edge.label when present, else the column list) +src/server/server.ts ............ M (/api/flow payload carries clusters) +src/generators/app.ts ............ M (static export payload carries window.__FLOW_CLUSTERS__) +test/checks/ +โ”œโ”€โ”€ test-flow-edge-labels.ts ............... A (label: on any entry; chip fallback; fingerprint unaffected) +โ”œโ”€โ”€ test-flow-clusters-parse.ts ............ A (parseClusters + cluster: token expansion + interception) +โ”œโ”€โ”€ test-flow-clusters-validate.ts ......... A (five new flow.cluster_* rules, two fixtures) +โ”œโ”€โ”€ test-flow-view-grouping.ts ............. A (per-process stacks; connected-view sources; collapse level) +โ””โ”€โ”€ test-dfd-stack-dialogs.ts .............. A (browser: StackDialog, contract dialog, view/level toggles) +test/visual/ +โ””โ”€โ”€ screenshot-store-clusters.ts ........... A (per-process view, connected view at each collapse level, + StackDialog, contract dialog) +test/fixtures/ +โ”œโ”€โ”€ broken-flows-model/flows/checkout/Bad-Cluster-Ref.md .. A (process fixture: unknown_cluster, +โ”‚ cluster_member_unknown, cluster_no_members) +โ”œโ”€โ”€ broken-flows-model/clusters/role-grants.md ............ A (valid cluster, referenced by the above) +โ”œโ”€โ”€ broken-flows-model/clusters/role-grants-invalid.md .... A (cluster-file fixture: cluster_entity_unknown, +โ”‚ cluster_overlap against role-grants.md) +โ””โ”€โ”€ hub-dfd/ ............................................... A (model root: hub-diagram โ€” 8 processes, + 17 stores, 31 edges; two hub stores read + by every process, a 3-member subtype + family, a 2-member author cluster, a + 2-member same-adjacency pair (written by + one process, read by none), an + ignatius.yml with flow_view.adjacency_stacks) +models/llm-memory-db-mssql/ +โ””โ”€โ”€ clusters/tag-junctions.md .............. A (proving-model cluster: Project_Tag, Artifact_Tag, Milestone_Tag, Task_Tag) +models/key-inherited/flows/order-to-cash/Create-Sales-Order/ +โ””โ”€โ”€ Validate-Customer.md ................... M (adds db:Person / db:Business reads โ€” subtype-family proof) +docs/guides/ +โ”œโ”€โ”€ flows.md ................... M (Labels, stacks, clusters, and groups section) +โ”œโ”€โ”€ folder-format.md ........... M (clusters/ joins the recognized root folders; groups/ notes its flow-view role) +โ””โ”€โ”€ validation.md ............... M (five new rules in the Cluster rules table) +docs/glossary.md ................ M (stack, cluster, group, collapse level, per-process view, connected view) +skills/ignatius-modeling/references/ +โ”œโ”€โ”€ dfd-authoring.md ............ M (a label step and a cluster step) +โ”œโ”€โ”€ flow-templates.md ........... M (cluster file template; label: on the process template) +โ”œโ”€โ”€ discover-flow.md ............ M (asks whether derived stores belong together) +โ””โ”€โ”€ verification.md ............. M (five rules in the flow rule reference table) +docs/wiki/feature-map.md ........ M (one row) +``` + + +## Outline + +``` +src/flows/flow-clusters.ts + FlowCluster โ€” a parsed clusters/.md: slug, label, entities, body, bodyHtml + parseClusters โ€” reads the model root's clusters/*.md files into a lookup keyed by slug + expandClusterEdges โ€” turns one cluster: input/output entry (its data: is an entityโ†’columns map) + into one FlowEdge per mapped member, each tagged with the owning cluster's slug and label + +src/flows/flow-parse.ts + parseEndpoint / buildEdgeFromInput / buildEdgeFromOutput + intercept a cluster: token before ordinary endpoint parsing and route it through + expandClusterEdges instead; read label: off every input/output entry, cluster or not + parseFlows + calls parseClusters alongside readExternalsDir/stores scan + FlowModel + clusters: FlowCluster[] โ€” new field, sibling to externals + FlowEdge + gains a new optional label โ€” independent of data:, carrying the chip text when authored + +src/flows/flow-validate.ts + checkUnknownCluster โ€” Class B: cluster: with no clusters/.md; strips the edge + checkClusterMemberUnknown โ€” Class B: a data: map key not in the cluster's entities; strips that member's edge + checkClusterNoMembers โ€” Class A: a cluster: entry whose data: map is empty + checkClusterEntityUnknown โ€” Class A: a cluster file lists an entity absent from the model + checkClusterOverlap โ€” Class A: one entity appears in two cluster files + +src/model/parse.ts + ModelMeta + gains an optional adjacency-stacks switch, parsed from ignatius.yml's flow_view: block, + mirroring the existing flow_rules: โ†’ _meta.flowRules precedent + +src/model/validate.ts + RuleId โ€” adds the five flow.cluster_* ids + explanation text + +src/flow-view/flow-layout.ts + buildFlowData + takes an optional view/collapse-level/clusters/subtype-clusters/groups argument alongside + the diagram; omitting it keeps today's per-store, ungrouped output + buildPerProcessStacks โ€” the default view: per process, per direction, a stack of every + store it touches when the set has 2+ members; two processes with the identical member + set converge on the same stack id; a store in more than one stack in a band is duplicate-marked + buildConnectedViewGrouping โ€” the day-one node model (one node per store, read/write split), + grouped in order by explicit cluster: tag, then author clusters, then subtype families, + then groups (only when collapseLevel is "groups"), then adjacency (when flow_view.adjacency_stacks + is not disabled); each step consumes the stores the previous step left ungrouped + buildStackRows โ€” given a stack's members and the current collapse level, returns its dialog + rows: table rows only at "stores", cluster/subtype rows plus loose tables at "clusters", + group rows (containing their clusters and tables) plus cross-group clusters at "groups" + edge label resolution โ€” an edge's chip lines are its label when present (one line per ", "-separated + item, never truncated), else today's gated column preview; a stack edge shows its members' + authored labels one per line followed by one gated preview line for all unlabelled members, + and one preview line when none is labelled; lines travel as an array so nothing re-splits them + +src/flow-view/elk-flow-layout.ts + nodeSize โ€” sizes a stack node from its member count + bandOf โ€” a stack node takes its members' existing band (1 read / 3 write) + computeElkLayout + threads view/collapseLevel/clusters/subtypeClusters/groups opts into buildFlowData + +src/flow-view/FlowDiagramSvg.tsx + StoreNode + stacked variant โ€” visible rows; a cluster or subtype row caps `C`, a group row caps `G`, store + rows cap `D#`; a C or G row draws the stacked-paper marker (two filled sheets behind it offset + 3px and 6px down-right showing bottom edges, left stair segments, and the top-right mark, no + cap divider on the sheets, the stack's left border and cap divider drawn per row, a reserve + of offset plus stroke plus two pixels after the row, none after the last row, whose box ends + on the back sheet with no closing line); the duplicate marker is drawn in the connected view + only + StackDialog trigger โ€” click on a stack node opens the row-list dialog + EdgeContractDialog trigger โ€” click on any data-carrying edge chip opens the contract dialog + chip rendering โ€” label only; no member or column count suffix + baseToken / search dimming โ€” a stack node matches search when any member matches + +src/app/components/flow-node/StackDialog.tsx + StackDialog โ€” titled by source (cluster label, basetype, group label, "N stores", or + "Read stack" / "Write stack"), with the feeding processes as dotted-number links in the + header; rows at the current collapse level; opening a cluster/subtype/group row shows + its members with D#s; a member opens its entity dialog; an author-cluster row also renders + the file's markdown body, a subtype row links the basetype (or reads " subtypes" + when the basetype itself isn't a member), a group row shows the group's description, and + an adjacency row lists the shared reader and writer processes instead of any body + +src/app/components/flow-node/EdgeContractDialog.tsx + EdgeContractDialog โ€” takes the edge's member edges; one flat table: group (as the entity's group + badge), store, column, type, sorted by store then entity column order; titled by the shared + label or the store-side name; ext: edges show a one-column Item table of label lines + +src/app/components/process/IoTable.tsx + renderRow โ€” shows edge.label when present, else the existing column-list rendering + +src/app/views/flow/FlowsView.tsx + renderDiagram โ€” passes clusters (from FlowModel), entityModel.subtypeClusters, entityModel.groups, + the active view, and the collapse level into computeElkLayout and FlowDiagramSvg + layoutKeyFor โ€” appends the active view name to the fingerprint-derived key so per-process and + connected-view drag positions never collide + stack/contract dialog open state โ€” reuses the existing dialog-close-before-entity-open rule in + FlowSurface's open handler + +src/app/App.tsx + view toggle (per-process default / connected) and collapse-level setting (stores/clusters/groups) + โ€” both global, persisted under ignatius-flow-view and ignatius-flow-collapse, same pattern as + the existing minimap toggle; written to the hash as flowview= and collapse= (replaceState), + the hash winning on load and Back/Forward restoring them live like dfd=; the FAB items read + "Connected view" / "Per-process view" and "Collapse to clusters" / "Collapse to groups" / + "Expand to stores" + +src/app/hooks/useModelData.ts + FlowApiPayload / static payload โ€” clusters field added, threaded to FlowsView + +src/server/server.ts + /api/flow โ€” response gains clusters: flowModel.clusters (cleaned) + +src/generators/app.ts + static export โ€” window.__FLOW_CLUSTERS__ alongside window.__FLOW_MODEL__ +``` + + +## Flows + +**Flow: any entry's label replaces the column-list chip** + +1. A process file's `outputs:` entry carries `label: new auth event` alongside its existing `data:` column list +2. `buildFlowData` resolves the edge's chip text to the label; the `data:` contract is unchanged and still backs the contract dialog +3. `FlowDiagramSvg` renders the chip as `new auth event`, not a truncated column preview +4. Clicking the chip opens `EdgeContractDialog`: one row per column, with its store and type +5. The dictionary's `IoTable` shows the same label in place of the column list for that entry + +**Flow: default per-process view stacks a process's reads and writes** + +1. A process reads three stores, including a hub store also read by seven other processes +2. `buildPerProcessStacks` groups the process's three-store read set into one stack, id `stack:--read` +3. Another process with the identical three-store read set converges on the same stack id and shares the node +4. A third process reads only the hub store plus a different second store, so it gets its own distinct stack; the hub store is duplicate-marked in every stack it appears in +5. Externals are unaffected โ€” `buildExternalRouting` still caps each at one source copy and one sink copy + +**Flow: connected view groups by cluster tag, author cluster, subtype family, group, then adjacency** + +1. User toggles to the connected view; the collapse level is "groups" +2. `buildConnectedViewGrouping` first pulls out any edges tagged by an explicit `cluster:` entry, then groups remaining stores by author `clusters/` file membership, then by subtype family, then โ€” because the collapse level is "groups" โ€” by shared entity `group:`, then (since `flow_view.adjacency_stacks` is not disabled) collapses any remaining same-adjacency stores +3. `buildStackRows` renders each stack's dialog rows nested per the "groups" breakdown: group rows containing their clusters and tables, then clusters spanning two groups, then loose tables +4. An adjacency-only stack's row reads `2 stores`; opening it lists the shared reader and writer processes instead of a body + +**Flow: author cluster expands, groups, and opens its contract** + +1. `parseFlows` reads `clusters/tag-junctions.md` into `FlowModel.clusters` +2. Merge Tag's `cluster:tag-junctions` output entry expands into four `db:` edges (Project_Tag, Artifact_Tag, Milestone_Tag, Task_Tag), each tagged `edge.cluster = { slug: 'tag-junctions', label: 'Tag junctions' }` +3. `validateFlows` checks every mapped member against the cluster's `entities:` and every column against its entity, same as an unqualified `db:` edge; an empty `data:` map fires `flow.cluster_no_members` instead +4. In the connected view, the four expanded edges group under one stack node `cluster:tag-junctions--write`, because they carry the cluster tag +5. The edge chip reads `Tag junctions`; clicking it opens `EdgeContractDialog` with one row per store/column/type across the four members +6. Clicking the stack node opens `StackDialog`, which renders the cluster file's markdown body and lists the four members with D#s; clicking a member opens its entity dialog, closing `StackDialog` first + +**Flow: subtype family groups without authoring** + +1. A process reads `db:Party`, `db:Person`, `db:Business` +2. `buildConnectedViewGrouping` receives `entityModel.subtypeClusters` (already parsed from `Party.md`'s `subtypes:` frontmatter โ€” no new file) and groups the three stores under `subtype:Party--read` +3. Since the basetype `Party` is itself one of the touched stores, the row reads `Party`; on a diagram where only `Person`/`Business` are touched with no `Party` read, the row instead reads `Party subtypes` + + +## Checkpoints + +| # | Checkpoint | Files/areas | Agent | Est. files | Verifies | +|---|------------|-------------|-------|------------|----------| +| 1 | Labels on every entry + contract dialog for every data edge + dictionary IO table | `src/flows/flow-parse.ts`, `src/flow-view/flow-layout.ts`, `src/flow-view/FlowDiagramSvg.tsx`, `src/app/components/flow-node/EdgeContractDialog.tsx`, `src/app/components/process/IoTable.tsx`, `test/checks/test-flow-edge-labels.ts` | atomic-implementer (mode: feature) | ~6 | a plain `db:` entry's `label:` renders on the chip in place of the column preview; an entry with no `label:` keeps today's column-preview chip; every data-carrying edge opens `EdgeContractDialog`; `IoTable` shows the label; adding a `label:` does not change `layoutFlowFingerprint` | +| 2 | Clusters registry parse + `cluster:` interception + `FlowModel`/payload plumbing | `src/flows/flow-clusters.ts`, `src/flows/flow-parse.ts`, `src/server/server.ts`, `src/generators/app.ts`, `test/checks/test-flow-clusters-parse.ts`, `test/fixtures/broken-flows-model/clusters/role-grants.md` | atomic-implementer (mode: feature) | ~6 | `parseClusters` returns the fixture's cluster map; a `cluster:` process entry expands into per-member `db:` edges tagged with the cluster slug and label before `parseEndpoint` ever sees the `cluster:` prefix; `FlowModel.clusters` and the `/api/flow` payload carry the registry | +| 3 | Validation: five `flow.cluster_*` rules, two fixtures | `src/flows/flow-validate.ts`, `src/model/validate.ts`, `test/checks/test-flow-clusters-validate.ts`, `test/fixtures/broken-flows-model/flows/checkout/Bad-Cluster-Ref.md`, `test/fixtures/broken-flows-model/clusters/role-grants-invalid.md` | atomic-implementer (mode: feature) | 5 | the process fixture's bad slug, unmapped `data:` member, and empty `data:` map fire `flow.unknown_cluster`/`flow.cluster_member_unknown`/`flow.cluster_no_members`; the second cluster file fires `flow.cluster_entity_unknown` and `flow.cluster_overlap` against `role-grants.md`; `ignatius validate models/llm-memory-db-mssql` and `models/key-inherited` stay clean | +| 4 | Per-process stacks (default view) + connected-view grouping + collapse level + hub fixture | `src/flow-view/flow-layout.ts`, `src/flow-view/elk-flow-layout.ts`, `src/model/parse.ts`, `test/checks/test-flow-view-grouping.ts`, `test/fixtures/hub-dfd/`, `models/llm-memory-db-mssql/clusters/tag-junctions.md`, `models/key-inherited/flows/order-to-cash/Create-Sales-Order/Validate-Customer.md` | atomic-implementer (mode: feature) | ~8 | on `hub-diagram`, the default per-process view produces one distinct read-stack per process (converging where read sets are identical) with the duplicate marker on shared hub stores; the connected view groups by cluster tag, author cluster, subtype family, group (at the groups level), and adjacency in that order; `flow_view.adjacency_stacks: false` in `hub-dfd/ignatius.yml` leaves the adjacency pair unstacked (id `stack:--write` when enabled); on `Validate-Customer`, Party/Person/Business group into one `subtype:Party--read` row | +| 5 | SVG stack node + `StackDialog` + search dimming + drag ids | `src/flow-view/FlowDiagramSvg.tsx`, `src/app/components/flow-node/StackDialog.tsx`, `src/app/views/flow/FlowsView.tsx`, `test/checks/test-dfd-stack-dialogs.ts` (StackDialog open/close + search-dimming cases), `test/visual/screenshot-store-clusters.ts` (per-process + connected-view stacked-node screenshot) | atomic-implementer (mode: feature) | ~6 | screenshot of `hub-diagram` shows a stacked-outline node per process (per-process view) and per grouping source (connected view); clicking a stack opens `StackDialog` with rows at the current collapse level; a subtype row with no basetype member reads ` subtypes`; an adjacency row reads `2 stores` and lists shared processes; opening an entity from `StackDialog` closes it first; search for a member's name highlights its stack | +| 6 | Contract dialog upgrade + chip is label-only | `src/flow-view/FlowDiagramSvg.tsx`, `src/app/components/flow-node/EdgeContractDialog.tsx` | atomic-implementer (mode: feature) | 2-3 | on `tag-administration`, the edge into the Merge Tag cluster shows the `Tag junctions` chip with no member/column-count suffix; clicking it opens `EdgeContractDialog` with a group/store/column/type row per member; an `ext:` edge's dialog shows label lines with no group/store/type columns | +| 7 | View + collapse-level toggles, per-view position persistence | `src/app/App.tsx`, `src/app/views/flow/FlowsView.tsx`, `test/checks/test-dfd-stack-dialogs.ts` (view/collapse-level toggle cases), `test/visual/screenshot-store-clusters.ts` (both views at each collapse level) | atomic-implementer (mode: feature) | ~5 | toggling the view on `hub-diagram` switches between per-process stacks and connected-view grouping; toggling the collapse level changes `StackDialog`'s row breakdown; both settings persist across reload; a drag saved in one view does not apply in the other (fingerprint + view name key); screenshots of both views at each collapse level | +| 8 | Docs: guide, folder format, validation, glossary, skill, feature map | `docs/guides/flows.md`, `docs/guides/folder-format.md`, `docs/guides/validation.md`, `docs/glossary.md`, `skills/ignatius-modeling/references/dfd-authoring.md`, `skills/ignatius-modeling/references/flow-templates.md`, `skills/ignatius-modeling/references/discover-flow.md`, `skills/ignatius-modeling/references/verification.md`, `docs/wiki/feature-map.md` | atomic-implementer (mode: feature) | 9 | every listed surface documents labels, the two views, the collapse level, clusters, groups, and adjacency (with its `ignatius.yml` switch); the flow rule reference table lists all five new rules; the closed endpoint-prefix passages in `flows.md`/`dfd-authoring.md`/`flow-templates.md` each name the `cluster:` interception as the one exception | + + +## Risks + +| Risk | Likelihood | Mitigation | +|------|-----------|-----------| +| `buildFlowData`'s 9 existing call sites (the renderer, the ELK module, 7 test files) all assume the old single-argument signature | high | Add `{ view, collapseLevel, clusters, subtypeClusters, groups }` as an optional second argument with today's per-store output as the default โ€” every untouched call site is unaffected; only `elk-flow-layout.ts`'s `computeElkLayout` (CP4) and `FlowDiagramSvg.tsx`'s banded-fallback call (CP5) are updated to pass real data | +| `/api/flow` and the static export today ship only `flowModel.diagrams`, not `flowModel.externals` โ€” there is no existing top-level-registry precedent to copy for `clusters` | medium | CP2 adds a first-class `clusters` field to both payloads and a matching `window.__FLOW_CLUSTERS__` global, following the same shape `flowLayoutKeys` already uses (a sibling top-level key, not nested in `diagrams`) | +| `layoutFlowFingerprint` (`flow-fingerprint.ts`) hashes structural topology only; per-process and connected views would collide on one saved-position bucket if the key were fingerprint alone | medium | Per the success-criteria entry on per-view persistence, `FlowsView.tsx`'s `layoutKeyFor` appends the active view name to the fingerprint-derived key (CP7), so the two views never share a drag-position bucket | +| Adjacency stacking groups by identical (readers, writers, kind) alone, which lands on real families and on coincidental pairs equally โ€” the AutomaStructure probe found unrelated pairs sharing adjacency | low | `flow_view: { adjacency_stacks: false }` (CP4) is a model-level kill switch; the adjacency row is labelled `2 stores`, never a member's name, and its dialog names the shared processes so a reader can judge relatedness themselves | +| `key-inherited`'s only `db:Party` flow reference (`Validate-Customer.md`) never previously touched `Person`/`Business`, so subtype-family grouping had no existing case to verify against | low | CP4 adds `db:Person`/`db:Business` reads to `Validate-Customer.md` โ€” additive flow authoring, not a change to `Party.md`'s `subtypes:` frontmatter or to subtype derivation itself | +| The AutomaStructure model itself is outside this repo, so no in-repo check can run against its real Compute Loads / design-building diagrams | low | `test/fixtures/hub-dfd` is shaped like Compute Loads (8+ processes, two hub stores, one subtype family, one author cluster, two same-adjacency stores) for every automated check; a manual screenshot against the real AutomaStructure model is the acceptance check outside CI | + + +## Change log + +### 2026-09-06 โ€” explicit `cluster:` entries always group; externals untouched by per-process mode + +**What changed:** A `cluster:` entry renders its cluster node regardless of how many members its `data:` map names, marked `1 of N` for a single member. The two-member threshold applies only to implicit grouping through plain `db:` entries. Per-process mode leaves externals at one source copy and one sink copy per diagram. Success criteria, the `buildClusterGrouping` outline line, the author-cluster and per-process flows, CP3's Verifies cell, and the risks table were rewritten to this truth. + +**Why:** The user asked whether writing `cluster:` with one member would draw a plain store; the body applied the threshold after expansion to every edge, which would have discarded an explicit authoring choice. The externals question was the design's second open item; the user chose to keep externals aggregated. + +**Superseded:** author clusters applied only where two or more members were touched, with no distinction between explicit and implicit references; per-process mode said nothing about externals. + +### 2026-09-07 โ€” labels on every entry, per-process default view, collapse levels, adjacency switch + +**What changed:** Rewrote the body to the post-challenge-swarm design. A prose `label:` is legal on any input/output entry (not only `cluster:` ones) and drives the edge chip everywhere, with a contract dialog on every data-carrying edge โ€” this is now the first checkpoint, independent of the cluster pipeline. The default rendering is a per-process view (one read-stack and one write-stack per process, id `stack:--read|--write`, identical sets sharing a stack, duplicate marker on shared members); the prior single "connected + optional per-process-mode toggle" model is now the "connected view", itself the toggle. A collapse-level setting (stores/clusters/groups) decides what a stack's rows represent; groups come from the entity's existing `group:` and `groups/.md`. Connected-view grouping order is now: explicit `cluster:` tag, author clusters, subtype families, groups (groups level only), adjacency (switchable off via `flow_view: { adjacency_stacks: false }` in `ignatius.yml`); adjacency's label is always `N stores`, never a member's name. Every grouped node id now carries `--read`/`--write`; a subtype row reads ` subtypes` when the basetype is absent from the touched stores; a cluster row's cap shows the lowest member D#. The edge chip is label-only โ€” no member/column-count suffix. Positions are keyed by fingerprint plus view name. Added `flow.cluster_no_members` as a fifth validation rule. The `cluster:` prefix is now explicitly intercepted before `parseEndpoint`, and the closed-prefix guide/skill passages must name this one exception. `deriveLevels`/`flow-derive-levels.ts` needs no change โ€” synthetic diagrams carry no edge data, so they run the same render-time pipeline with nothing for an explicit tag to survive on. Renamed the rendered node's dialog from `ClusterDialog` to `StackDialog` โ€” "cluster" now names only the `clusters/` file source and the subtype family, never the rendered box. Cited the existing `FlowSurface` dialog-close-before-entity-open rule instead of re-specifying it. Replaced the motivating Problem/Goal example with the AutomaStructure model's Compute Loads (9 processes, 22 stores, 64 edges) and design-building (10 processes, 79 stores, 230 edges) diagrams; added an in-repo `test/fixtures/hub-dfd` shaped like Compute Loads for every automated success criterion, since AutomaStructure itself is outside this repo. The `tag-administration` success criterion now asserts the `Tag junctions` label only, not an edge-count drop (adjacency stacking already collapses that case on its own). Split the cluster-rule negative fixture in two: a process file for the two rules only a `cluster:` entry can fire (`flow.unknown_cluster`, `flow.cluster_member_unknown`, `flow.cluster_no_members`) and a cluster file for the two rules only a cluster file can fire (`flow.cluster_entity_unknown`, `flow.cluster_overlap`). Added `IoTable.tsx` (dictionary process IO table) to the change tree and to CP1, so it shows labels too. The hub-diagram success criteria and the `hub-dfd/` fixture carry concrete counts: 8 processes, 17 stores, 31 authored edges, collapsing to 13 rendered edges across 12 distinct store-side nodes under the default per-process view. + +**Why:** A challenge swarm and a strategist critique (`.claude/.scratchpad/dfd-store-clusters/critique.md`, evidenced by an adjacency probe against a fixture shaped like the AutomaStructure model) found that the prior design's success criteria measured an already-legible diagram, gated the highest-value fix (labels) behind the cluster pipeline, used a cluster-node id that cannot exist once a store is both read and written, and left per-process mode, adjacency precision, and the AutomaStructure motivating example unaddressed. + +**Superseded:** the three-source connected-only render (author clusters, subtype families, adjacency) with an opt-in per-process *mode* and an unsuffixed `cluster:` node id; a `label:` restricted to `cluster:` entries; an edge chip suffixed with member/column counts; a single global position key per diagram; four validation rules; `ClusterDialog` as the rendered node's dialog name; success criteria scoped to `tag-administration`'s edge-count drop and to `models/llm-memory-db-mssql`/`models/key-inherited` alone, with no in-repo hub-shaped fixture. + +### 2026-09-07 โ€” implementation deviations folded into the contract + +**What changed:** The body now states the rules the implementation settled where the earlier text left them open or named them differently. Hash parameters are `flowview=` and `collapse=` because `view=` already routes the graph, dictionary, and flow surfaces; the settings persist under `ignatius-flow-view` and `ignatius-flow-collapse` and are restored on Back/Forward like `dfd=`. A stack edge's chip carries lines as an array: authored labels one per line, then one preview line for all unlabelled members. The contract dialog takes the edge's member edges, shows the group as the entity's badge, sorts by store then entity column order, and shows an `ext:` edge as a one-column Item table. The stack dialog is titled by source and lists feeding processes as dotted-number links. The parser leaves a `clusterIssue` marker on unresolved `cluster:` edges for the validator to strip; `flow-markdown.ts` holds the shared frontmatter and markdown helpers; `FlowElementData`'s node variant is a discriminated union on `nodeType`; `buildFlowData` no longer returns a store-number map since nodes carry their own D#. `selectDiagramById` threads the leaf diagram into `onDiagramChange`, which the popstate reconcile exposed as a latent drill-depth bug. The hub fixture carries a `label:` on every entry plus three `ext:` edges so read-only processes validate; `subtype-no-basetype` is a second fixture for the ` subtypes` label. + +**Why:** Each item surfaced in a builder or reviewer round of `/subagent-implementation` and was decided there; the body must match what a fresh reader can verify in the code. + +**Superseded:** an unnamed hash and storage key scheme; a chip rule that only covered fully labelled or fully unlabelled stack edges; a contract dialog described without its group badge, sort order, or member-edge input. + +### 2026-09-07 โ€” C and G caps, stacked-paper marker, duplicate marker per view + +**What changed:** Cluster and subtype rows cap `C` and group rows cap `G` instead of a member's D number. The "more inside" marker is the hand-drawn stacked-paper construction: three sheets, the front one drawn last, the two behind offset 3px and 6px down-right and showing their bottom edges, left stair segments, and the top-right mark with no cap divider, the stack reserving the offset plus a stroke width and two pixels after such a row. The duplicate-store marker is drawn only in the connected view; the per-process view flags duplicates in the data but draws every row alike. + +**Why:** The user reviewed the rendered stacks: a D number on a row that is not a data store misreads the notation; the first marker attempt drew slivers beside the row and the second showed the copies' cap stubs as a small square; in the per-process view every shared store repeats by design, so the thick left border marked nearly every hub row and carried no information. + +**Superseded:** the cap showing the lowest member D number; two offset outlines as the marker; the duplicate marker drawn in both views. + +### 2026-09-08 โ€” filled sheets on the stacked-paper marker + +**What changed:** The two sheets behind a C or G row are filled with the box colour, drawn back to front under the row's own fill, so the marker reads as solid stacked paper. A stack whose last row is grouped ends on the back sheet's bottom edge: no clearance is reserved after it and the box draws no separate closing line, so an edge docking at the bottom meets the sheet. + +**Why:** With outline-only sheets the page background showed through them, and in the dark theme the marker looked hollow; the user asked for the fill. Once filled, the full-width closing line under a trailing grouped row floated below the sheets as a detached stroke. + +**Superseded:** outline-only sheets; the closing line and trailing clearance under a stack that ends in a grouped row. + + +## Implementation log + +### shipped on branch worktree-dfd-store-stacks โ€” 2026-09-08 + +Built across 9 iterations of /subagent-implementation (eight checkpoints plus one polish iteration of five rounds), then squashed into one commit on the branch; the per-iteration history lives in the scratchpad's `STATE.md`. Checkpoints in order: + +- CP-1 labels on every entry, contract dialog, dictionary IO table labels +- CP-2 clusters registry, `cluster:` token expansion, payload plumbing +- CP-3 five `flow.cluster_*` validation rules with disk fixtures +- CP-4a stack node model, per-process stacks, hub fixture, `flow_view` config +- CP-4b connected-view grouping, collapse levels, per-process qualification +- CP-5 stack rendering, StackDialog, search dimming, per-process default view +- CP-6 contract dialog for stack edges, mixed-chip rule, D# fix on plain per-process nodes +- CP-7 view and collapse toggles, hash deep-links, per-view position keys +- CP-8 guides, glossary, skill references, feature map +- polish: chip dedup at stack outlets, deferred SVG unmount, bounded search check, repaired navigability script, stack top alignment, dedup scope, C/G caps, stacked-paper marker with filled sheets, duplicate mark per view, stack width from the widest row label +- skill: the modeling skill authors labels and clusters by default so diagrams stay legible; the demo `models/key-inherited` Collect Payment writes `cluster:settlement` with a label on every entry, and the flows guide and the skill quote that file + +**Out-of-scope work performed during this build:** + +- `src/flows/flow-markdown.ts` extracted so the cluster parser and the flow parser share frontmatter and markdown helpers instead of duplicating them. +- `FlowElementData`'s node variant became a discriminated union on `nodeType`; a merged optional-field type made `Extract` resolve to `never` in tests. +- `selectDiagramById` now threads the leaf diagram into `onDiagramChange`; the popstate reconcile exposed a drill-depth loss on rebuild. +- Two pre-existing checks were updated for the new default view; `test-cp3-dfd-url-navigability.ts` was repaired after leveling had removed the nav card it clicked. + +**Unforeseens โ€” surprises that emerged during implementation:** + +- The hub fixture needed three `ext:` output edges so read-only processes pass `flow.process_no_output`; the 31 store-touching edges stayed exact. +- The `view=` hash key already routes surfaces, so the flow view uses `flowview=` and `collapse=`. +- The stacked-paper marker took four constructions; the final one came from a session-model implementer working from the hand-drawn reference. +- A React warning about unmounting a nested root during render predated the feature and was fixed by deferring the unmount. + +**Deferred items still open:** + +- `.claude/project/followups/dfd-store-clusters-f-15.md` โ€” the intermittent stall in `test-graph-search.ts`; bounded with a watchdog, root cause not found. diff --git a/docs/wiki/docs.md b/docs/wiki/docs.md index 773d49e..92953b5 100644 --- a/docs/wiki/docs.md +++ b/docs/wiki/docs.md @@ -10,7 +10,7 @@ tags: [design, spec, guide] ## What it does -[`docs/`](..) (excluding the generated [`docs/wiki/`](.)) is ignatius's documentation corpus: 76 markdown files plus [`docs/glossary.md`](../glossary.md), split across four directories that each answer a different question about a feature. [`docs/design/`](../design) (29 files) states why a feature exists and which approach was chosen over its alternatives. [`docs/spec/`](../spec) (35 files) is the implementation contract derived from a design: checkpoints, success criteria, and (for two specs so far) a change-tree/outline/flows triad. [`docs/guides/`](../guides) (10 files) teaches a user how to drive the built feature. [`docs/research/`](../research) (2 files) records background investigation that fed a design's option table. None of these files execute; every other domain's code and tests point back at them by name for the "why is it built this way" and "what is the contract" questions code alone can't answer. +[`docs/`](..) (excluding the generated [`docs/wiki/`](.)) is ignatius's documentation corpus, split into four directories that each answer a different question about a feature: why it exists, what its contract is, how to drive it, and what background research shaped it. A session that needs one of those answers goes to the directory that owns it, rather than searching the whole tree. [`docs/design/`](../design) states why a feature exists and which approach was chosen over its alternatives. [`docs/spec/`](../spec) is the implementation contract derived from a design: checkpoints, success criteria, and (for three specs so far) a change-tree/outline/flows triad. [`docs/guides/`](../guides) teaches a user how to drive the built feature. [`docs/research/`](../research) records background investigation that fed a design's option table. None of these files execute; every other domain's code and tests point back at them by name for the "why is it built this way" and "what is the contract" questions code alone can't answer. The corpus totals 78 markdown files plus [`docs/glossary.md`](../glossary.md): 30 in [`docs/design/`](../design), 36 in [`docs/spec/`](../spec), 10 in [`docs/guides/`](../guides), 2 in [`docs/research/`](../research). [`README.md`](../../README.md) states the design/spec relationship directly: "Conceptual designs live in [`docs/design/`](../design); the implementation contracts derived from them live in [`docs/spec/`](../spec)." @@ -48,18 +48,19 @@ stateDiagram-v2 ### The change-tree / outline / flows triad is opt-in by spec age, not by feature size -[`docs/spec/graph-flow-search.md`](../spec/graph-flow-search.md) and [`docs/spec/model-index-routing.md`](../spec/model-index-routing.md) are the only two of 35 specs carrying `## Change tree`, `## Outline`, and `## Flows` sections; the other 33 predate the rule that requires them and are not backfilled by an unrelated amendment. 28 of 35 specs also carry a `## Implementation log` (narrative build history: checkpoints landed, out-of-scope work performed, unforeseens, deferred items) โ€” a section distinct from `## Change log`, which records contract amendments, not build narrative. +[`docs/spec/graph-flow-search.md`](../spec/graph-flow-search.md), [`docs/spec/model-index-routing.md`](../spec/model-index-routing.md), and [`docs/spec/dfd-store-clusters.md`](../spec/dfd-store-clusters.md) are the only three of 36 specs carrying `## Change tree`, `## Outline`, and `## Flows` sections; the other 33 predate the rule that requires them and are not backfilled by an unrelated amendment. 29 of 36 specs also carry a `## Implementation log` (narrative build history: checkpoints landed, out-of-scope work performed, unforeseens, deferred items) โ€” a section distinct from `## Change log`, which records contract amendments, not build narrative. ## Where it lives -### [`docs/design/`](../design) โ€” conceptual/approach docs (29 files) +### [`docs/design/`](../design) โ€” conceptual/approach docs (30 files) | Path | Lines | Covers | |------|-------|--------| | [`docs/design/model-index-routing.md`](../design/model-index-routing.md) | 476 | Per-folder generated routers (`index.md`), rolled-up SHA digests, `` managed regions, `index_file:`/`harness:` config, in-folder `AGENTS.md`/[`CLAUDE.md`](../../CLAUDE.md)/`SKILL.md` agent guidance | | [`docs/design/markdown-driven-erd.md`](../design/markdown-driven-erd.md) | 333 | Canonical source for the markdown-driven entity file format; no [`docs/spec/`](../spec) counterpart | +| [`docs/design/dfd-store-clusters.md`](../design/dfd-store-clusters.md) | 299 | Per-process vs. connected DFD views, prose `label:` on any flow entry, `clusters/` author-cluster files, the `cluster:` token, subtype-family and group collapse levels, adjacency stacking, the stack/contract/subtype/group dialog rules, the stacked-paper "more inside" marker construction | | [`docs/design/process-flows.md`](../design/process-flows.md) | 218 | SSADM DFD subsystem: processes, externals, stores, sub-DFDs | | [`docs/design/schema-lint-and-error-ux.md`](../design/schema-lint-and-error-ux.md) | 205 | Schema lint + error UX | | [`docs/design/noorm-flow-discovery.md`](../design/noorm-flow-discovery.md) | 179 | `ignatius-modeling` skill's `flow` and `discover` Q&A modes | @@ -88,12 +89,13 @@ stateDiagram-v2 | [`docs/design/src-root-organization.md`](../design/src-root-organization.md) | 49 | [`src/`](../../src) top-level subdirectory split | | [`docs/design/noorm-modeling-skill.md`](../design/noorm-modeling-skill.md) | 12 | Rename stub; points to `ignatius-modeling-skill.md` | -### [`docs/spec/`](../spec) โ€” implementation contracts (35 files) +### [`docs/spec/`](../spec) โ€” implementation contracts (36 files) | Path | Lines | Covers | |------|-------|--------| | [`docs/spec/process-flows.md`](../spec/process-flows.md) | 682 | Largest spec; SSADM DFD: parse, 11 `flow.*` rules, viewer, sub-DFD drill-down, `db:` store dialog | | [`docs/spec/key-inheritance-lineage.md`](../spec/key-inheritance-lineage.md) | 372 | `buildInheritedConnections` key-edge connected-component algorithm, DG/DD lineage reveal | +| [`docs/spec/dfd-store-clusters.md`](../spec/dfd-store-clusters.md) | 391 | CP1โ€“CP8: `label:` on every flow entry + contract dialog, `flow-clusters.ts` registry + `cluster:` token expansion, five `flow.cluster_*` rules, per-process/connected view stacking, collapse levels, `StackDialog`/`EdgeContractDialog`; five `## Change log` entries plus an `## Implementation log` recording the 9-iteration shipped build and one open followup | | [`docs/spec/model-index-routing.md`](../spec/model-index-routing.md) | 270 | Router build/write, fingerprint roll-up, `index_file`/`harness` config, four `config.index_file_*`/`index.*` rules, `--agents` guidance files | | [`docs/spec/app-tsx-decomposition.md`](../spec/app-tsx-decomposition.md) | 246 | `App.tsx` decomposition | | [`docs/spec/dd-spotlight-grid.md`](../spec/dd-spotlight-grid.md) | 239 | DD browse-lens spotlight grid | @@ -136,12 +138,12 @@ All ten are linked from [`README.md`](../../README.md)'s docs table. Six were up | Path | Lines | Covers | |------|-------|--------| -| [`docs/guides/folder-format.md`](../guides/folder-format.md) | 256 | โ˜… `ignatius.yml`, the five top-level folders, entity/column/relationship authoring, `index_file:`/`harness:` config, generated routers, `description:` frontmatter | +| [`docs/guides/folder-format.md`](../guides/folder-format.md) | 275 | โ˜… `ignatius.yml`, the six top-level folders including `clusters/`, entity/column/relationship authoring, `index_file:`/`harness:` config, generated routers, `description:` frontmatter | | [`docs/guides/commands.md`](../guides/commands.md) | 174 | โ˜… The CLI subcommands including `index`/`index --agents`, `validate --index`, and the full keyboard-shortcut table | -| [`docs/guides/flows.md`](../guides/flows.md) | 150 | โ˜… DFDs: processes, externals, stores, sub-DFDs, `description:` on process/external/store | -| [`docs/guides/validation.md`](../guides/validation.md) | 137 | โ˜… The linter, severity tiers, and the new Config-rules/Index-rules tables (`config.index_file_*`, `index.stale`, `index.orphaned`, `index.unreadable_target`) | +| [`docs/guides/flows.md`](../guides/flows.md) | 280 | โ˜… DFDs: processes, externals, stores, sub-DFDs, `description:` on process/external/store, plus labels/stacks/clusters/groups โ€” the contract dialog, the closed endpoint-prefix set and its one `cluster:` exception, the per-process vs. connected view split, the stores/clusters/groups collapse level, adjacency stacking and its `ignatius.yml` switch, and the Flows FAB controls | +| [`docs/guides/validation.md`](../guides/validation.md) | 142 | โ˜… The linter, severity tiers, the Config-rules/Index-rules tables (`config.index_file_*`, `index.stale`, `index.orphaned`, `index.unreadable_target`), and the five `flow.cluster_*` rules | | [`docs/guides/getting-started.md`](../guides/getting-started.md) | 93 | โ˜… Install, build from source, serve the first model; command list now names `index` | -| [`docs/guides/modeling-skill.md`](../guides/modeling-skill.md) | 73 | โ˜… The `/ignatius-modeling` skill's Q&A modes; verification loop now runs `ignatius validate --index` | +| [`docs/guides/modeling-skill.md`](../guides/modeling-skill.md) | 73 | โ˜… The `/ignatius-modeling` skill's Q&A modes; the `flow` mode's step list now names the cluster-membership decision and the flow label alongside the `db:`-or-`kind:` store decision; verification loop now runs `ignatius validate --index` | | [`docs/guides/derivation.md`](../guides/derivation.md) | 45 | What gets derived (cardinality, classification, subtype clusters) vs authored by hand | | [`docs/guides/predicates.md`](../guides/predicates.md) | 83 | Bidirectional relationship-edge label authoring | | [`docs/guides/themes-and-branding.md`](../guides/themes-and-branding.md) | 83 | `theme`/`branding` blocks in `ignatius.yml` | @@ -154,9 +156,9 @@ All ten are linked from [`README.md`](../../README.md)'s docs table. Six were up | [`docs/research/dfd-layout-and-leveling.md`](../research/dfd-layout-and-leveling.md) | 129 | DFD layout engines and Yourdon leveling; primary source for `dfd-overhaul`'s ELK algorithm choice | | [`docs/research/ssadm-dfd-rules.md`](../research/ssadm-dfd-rules.md) | 118 | SSADM DFD rules | -### [`docs/glossary.md`](../glossary.md) (52 lines) +### [`docs/glossary.md`](../glossary.md) (67 lines) -Canonical vocabulary table: DG (Data Graph), DD (Data Dictionary), DFD (Data Flow Diagram), DE (Data Entity), DS (Data Store), EE (External Entity), Process, Data Flow, plus the DS โŠƒ DE relationship note and the `kind:` store taxonomy (`db`/`cache`/`queue`/`file`/`doc`/`manual`/`other`). +Canonical vocabulary table: DG (Data Graph), DD (Data Dictionary), DFD (Data Flow Diagram), DE (Data Entity), DS (Data Store), EE (External Entity), Process, Data Flow, plus the DS โŠƒ DE relationship note, the `kind:` store taxonomy (`db`/`cache`/`queue`/`file`/`doc`/`manual`/`other`), and a "Flow view: stacks, clusters, and groups" table (flow label, cluster, group, stack, collapse level, per-process view, connected view, adjacency stack). [`docs/wiki/`](.) also lives under [`docs/`](..) as the generated signals wiki; it is separate, self-referential infrastructure, out of scope for this domain. @@ -167,10 +169,8 @@ Canonical vocabulary table: DG (Data Graph), DD (Data Dictionary), DFD (Data Flo | Constraint | Detail | |------------|--------| | Spec body is forward-only | `docs/spec/.md` must describe only the current decision; superseded content moves to a dated `## Change log` entry with a **Superseded:** line. Leaving old text in the body instead means a subagent implementing from the spec reads a contradicted or stale contract as current truth | -| Change-tree/outline/flows apply forward only | The three required sections apply to specs drafted after the rule shipped; only 2 of 35 specs (`graph-flow-search.md`, `model-index-routing.md`) carry them. Backfilling them onto a pre-existing spec via an unrelated amendment would bundle an unrelated structural change into that amendment's `## Change log` entry, misstating what the amendment actually changed | -| Reserved router filename | `index_file:` (default `index.md`) is reserved model-wide: an entity file under `data/` sharing that basename and declaring `entity:` fails `config.index_file_entity` ([`docs/spec/model-index-routing.md`](../spec/model-index-routing.md) SC3) | -| Guidance files stay under 200 lines | `AGENTS.md`, [`CLAUDE.md`](../../CLAUDE.md), `SKILL.md` generated by `ignatius index --agents` carry only model name, description, router filename, key-style convention, and the `[[Entity]]` rule (SC11); adding entity/column/relationship content would make the file grow with the model and fail SC11 | -| Managed-region writes are byte-scoped | A generator (routers, or the `--agents` guidance files) owns only the bytes inside its own `` tag; a boundary is a tag starting at column 0 and ending its line, and mismatched/nested/orphan/unclosed tags throw with a line number rather than silently corrupting the file | +| Change-tree/outline/flows apply forward only | The three required sections apply to specs drafted after the rule shipped; only 3 of 36 specs (`graph-flow-search.md`, `model-index-routing.md`, `dfd-store-clusters.md`) carry them. Backfilling them onto a pre-existing spec via an unrelated amendment would bundle an unrelated structural change into that amendment's `## Change log` entry, misstating what the amendment actually changed | +| The `cluster:` prefix is the one exception to the closed endpoint-prefix set | [`docs/guides/flows.md`](../guides/flows.md)'s Endpoints section states the `db:`/`ext:`/`proc:`/`cache:`/`queue:`/`file:`/`doc:`/`manual:`/`other:` prefix set is closed, then names `cluster:` as intercepted and expanded before endpoint parsing runs; [`docs/spec/dfd-store-clusters.md`](../spec/dfd-store-clusters.md)'s Checkpoints table (CP8's Verifies cell) requires every closed-prefix-set passage across the guides and the `ignatius-modeling` skill references to name this one exception. A guide or skill passage that repeats the closed-set list without this exception either teaches an author that `cluster:` is unsupported, or contradicts the other doc stating the same closed set | ## Coupling @@ -185,4 +185,6 @@ Canonical vocabulary table: DG (Data Graph), DD (Data Dictionary), DFD (Data Flo | [`docs/spec/example-instance-tables.md`](../spec/example-instance-tables.md) | **skill** | Names [`skills/ignatius-modeling/references/entity-flow.md`](../../skills/ignatius-modeling/references/entity-flow.md) and directs it to add Step E7b โ€” Examples, between E7 (Columns) and E8 (Reference table) | | [`docs/spec/process-flows.md`](../spec/process-flows.md) | **skill** | Its `flow.*` frontmatter/token grammar is matched by [`skills/ignatius-modeling/references/flow-templates.md`](../../skills/ignatius-modeling/references/flow-templates.md) | | [`docs/guides/themes-and-branding.md`](../guides/themes-and-branding.md) | **theme**, **skill** | Its worked example is pointed to by [`skills/ignatius-modeling/references/model-flow.md`](../../skills/ignatius-modeling/references/model-flow.md) | +| [`docs/design/dfd-store-clusters.md`](../design/dfd-store-clusters.md) + [`docs/spec/dfd-store-clusters.md`](../spec/dfd-store-clusters.md) | **flows**, **flow-view**, **frontend**, **skill** | The pair's Change tree names a new [`src/flows/flow-clusters.ts`](../../src/flows/flow-clusters.ts) plus edits to `flow-parse.ts`/`flow-validate.ts` (**flows**); `flow-layout.ts`/`elk-flow-layout.ts`/`FlowDiagramSvg.tsx` for per-process/connected-view stacking and collapse levels (**flow-view**); `App.tsx`/`FlowsView.tsx` and new `StackDialog.tsx`/`EdgeContractDialog.tsx` for the view/collapse toggles and dialogs (**frontend**); and `dfd-authoring.md`/`flow-templates.md`/`discover-flow.md`/`verification.md` for the new authoring and validation surface (**skill**) โ€” [`docs/guides/modeling-skill.md`](../guides/modeling-skill.md) documents that same skill-side coupling from the user's side, naming the cluster-membership decision and the flow label as steps in the `flow` mode's Q&A | | [`docs/wiki/feature-map.md`](feature-map.md) | all domains | Hand-authored feature-to-doc-to-skill cross-reference table; not generated by this signals pipeline, maintained separately | + diff --git a/docs/wiki/feature-map.md b/docs/wiki/feature-map.md index f75a37f..ef73178 100644 --- a/docs/wiki/feature-map.md +++ b/docs/wiki/feature-map.md @@ -53,6 +53,7 @@ Paths are relative to [`docs/design/`](../design), [`docs/spec/`](../spec), [`do | DD spotlight inherited 1:1 key-inheritance connections (#9, CP7; SUPERSEDED by `key-inheritance-lineage` CP-A โ€” kept for history; bounded to subtype clusters, single-level; pure [`src/app/logic/spotlight-inherited.ts`](../../src/app/logic/spotlight-inherited.ts) `buildInheritedConnections(index, entityId)` โ†’ `InheritedConnection[]` (`{otherId, direction, via}`, `INHERITED_IDENTITY='identity'`); member โ†’ basetype + sibling identity links + basetype's direct rels (via=basetype); basetype โ†’ members + each member's direct rels (via=member); transitive rels de-dup against the active's own direct edges, identity links exempt; general identifying-1:1 dependent tables were a noted non-goal here โ€” now generalized below; `SpotlightOverlay` draws a THIRD line category DOTTED in `--spotlight-line-inherited` green via the CP6 `separateSpotlightLines` path with "via <basetype>"/"shared key" pills + off-screen `spotlight-chip--inherited` chips; `DictionaryView` folds inherited ids into `spotlitIds`+`focusSet`; `buildSpotlightConnections` unchanged) | viewer-ux-polish | viewer-ux-polish | โ€” | โ€” | | Key-inheritance lineage (GENERALIZES #9/CP7 above; corrected to the key-edge connected-component model โ€” [`src/app/logic/spotlight-inherited.ts`](../../src/app/logic/spotlight-inherited.ts) `buildInheritedConnections` keeps its export name + `InheritedConnection {otherId,direction,via}` shape + `INHERITED_IDENTITY='identity'` so `DictionaryView`/`SpotlightOverlay`/`GraphView` are unchanged; **lineage follows ONLY key edges** โ€” an edge whose child-side FK cols (`Object.keys(edge.on)`) are ALL โŠ† the child PK (`pkByNode.get(edge.source)`), a SUBSET test (FK โŠ† PK, non-empty), NEVER a secondary/non-key FK; this one predicate catches identifying-1:many (FK a PROPER subset of the PK, e.g. `SalesInvoiceโ†’Party` on `party_no`) AND subtype memberโ†’basetype (FK==full PK) โ€” empirically `edge.identifying`==FKโŠ†PK on `key-inherited`; **lineage** = transitive connected component over key edges in BOTH directions (undirected), cycle-safe visited map; inherited = lineage โˆ’ self โˆ’ direct real-edge neighbours (those render solid); `direction='out'` (DD draws ONE source-out line โ€” single arrowhead at the far/member end, pointing FROM the active card OUT to the member; was `'both'`), `via`=nearest key-edge predecessor on the path (or `INHERITED_IDENTITY`); bundle one per otherId, sort by otherId, singleton lineage โ†’ []; **DD inherited lines are SHIFT-GATED (mirrors DG):** in the browse lens the dotted inherited lines appear ONLY while Shift is held over an active (hover/pin) card โ€” `DictionaryView` carries a `shiftHeld` state driven by a document `keydown`/`keyup` pair on `Shift` + a `window` blur reset; the `inheritedConnections` useMemo returns `[]` unless `shiftHeld && activeId`, and the inherited-id foldings into `spotlitIds`/`focusSet` are gated on `shiftHeld` too (no lit/extra-focused inherited cards or off-screen inherited chips without Shift); FK (solid) + flow (dashed) lines are UNCHANGED (plain hover/pin); `SpotlightOverlay` unchanged (renders `inheritedConnections=[]` as zero lines/chips); no longer walks subtype-cluster maps (memberโ†’basetype IS a key edge) and no longer calls `buildSpotlightConnections` for de-dup; SUPERSEDES the old subtype-cluster + dependent-identifying-1:1 (FK==full PK + 1:1) + per-member secondary-FK expansion, which over-connected via secondary FKs (`SI_Lineโ†’Product`/`โ†’LineItemType`) and missed identifying-1:many lineage; proven on [`models/key-inherited`](../../models/key-inherited): `SSN` reaches the party-keyed family (`SalesInvoice`/`SI_Line`/`SalesOrder`/`SO_Line`/`PaymentAllocation`โ€ฆ) and EXCLUDES `Product`/`Subscription`/`LineItemType`/`PartyType`, `SI_Line` no longer over-connects, ORM surrogate-PK models have zero lineage. **CP-B (DG dotted lines), SHIFT+HOVER trigger:** lineage is revealed by SHIFT+HOVER, NOT click/select. While Shift is held and the pointer is over a node, `GraphView` `enterLineageHover` calls `drawInheritedEdges` to add EPHEMERAL dotted cytoscape edges (class `inherited`, id `_inherited___`) from the hovered node to each inherited `otherId` present in cy, then `applyFocusTiers`; styled in `styles.ts` (`edge.inherited`: dotted, arrowless, thin) using the new `SPOTLIGHT_LINE_INHERITED[mode]` constant exported from `theme-css-vars.ts` (single source of truth shared with the DD CSS var so DG==DD); endpoints folded into the focus-fade `keep` set so they stay lit. Trigger wiring: `mouseover` branches on `evt.originalEvent?.shiftKey` (shift โ†’ lineage, no-shift โ†’ plain direct-neighbour fade); `mouseout` exits (`exitLineageHover`); a document-level `Shift` keydown/keyup pair toggles lineage on the live hovered node (`hoveredNodeIdRef`) so holding/releasing Shift while already hovering works; all state in refs (stale-closure-safe), listeners removed in cy-init cleanup. A plain click now SELECTS + opens the modal only (no lineage); select/navigate/panel/hash-restore paths no longer draw lineage. `clearInheritedEdges` (`cy.remove('edge.inherited')`) on mouseout/shift-release/deselect/reselect/reset/applyLayoutMode-before-ELK/no-entity-restore/teardown; never enters model/`layoutFingerprint`/`layout-store`/static-export/ELK โ€” added after layout, removed before re-layout; no-leak unit check (`test-inherited-edges-no-leak.ts`, trigger-agnostic) + Playwright DG check (`test-graph-inherited-edges.ts`: plain click โ†’ 0 inherited; shift+hover Identity โ†’ dotted rays; mouseout โ†’ 0; shift+hover ITIN strictly larger transitive set; plain hover โ†’ 0; deselect โ†’ 0) + visual screenshot ([`test/visual/test-graph-inherited-lines.ts`](../../test/visual/test-graph-inherited-lines.ts): SSN + SI_Line owner cases via shift+hover). **3-tier focus opacity:** `GraphView` `applyFocusTiers(focusNode)` (run on shift+hover lineage + plain hover; cleared on mouseout/shift-release/deselect/reselect/relayout/teardown) splits the focused state into DIRECT (focused node + its REAL graph neighbors via `connectedEdges().not('.inherited')` + identifying lineage/descendants + joiners โ†’ opacity **1.0**), INHERITED/ancestral (the `edge.inherited` rays + their target nodes minus direct, `inherited-dim` โ†’ **0.5**), UNRELATED (everything else, `faded` โ†’ **0.2**); `styles.ts` `.faded` 0.3โ†’0.2, new `.inherited-dim` 0.5, `edge.inherited` opacity 0.85โ†’0.5; direct wins de-dup (`.difference(direct)` + `buildInheritedConnections` already de-dups); visual harness reads per-tier opacity off the live cy and asserts `direct>inherited>unrelated`) | key-inheritance-lineage | key-inheritance-lineage | โ€” | โ€” | | Glossary of app terms (DG/DD/DFD/DE/DS/EE; DSโŠƒDE) | โ€” | โ€” | `../glossary.md` | โ€” | +| Store clusters (flow labels on every entry + contract dialog; per-process view default with connected view as toggle; collapse level stores/clusters/groups; `clusters/.md` author clusters + `cluster:` token, the one exception to the closed endpoint-prefix set; five `flow.cluster_*` validation rules; adjacency stacking, `flow_view: { adjacency_stacks: false }`) | dfd-store-clusters | dfd-store-clusters | flows (Labels, stacks, clusters, and groups), folder-format (`clusters/`), validation (cluster rules) | SKILL core rule (labels and clusters always), dfd-authoring (How the diagram reads, F4a cluster step + stack row count, F5 label rules), flow-templates (`cluster:` entry in the process template, labelled worked example + `clusters/settlement.md`, cluster template), templates (`flow_view:` in `ignatius.yml`), discover-flow (Gate 4 names the data and the set), reverse-engineering (R2 labels + transaction-written clusters), verification (cluster rule table + flow self-check 5-7: labels, cluster hygiene, legibility) | | Model index routing (generated per-folder `index.md` routers with name/kind/description/link tables; rolled-up SHA digests; `` and `` managed regions beside hand-authored ``; `index_file:` and `harness:` keys in `ignatius.yml`; `ignatius index [--agents]` writes routers and in-folder `AGENTS.md` / [`CLAUDE.md`](../../CLAUDE.md) shim / `SKILL.md` guidance; `ignatius validate --index` reports drift as `index.stale`; `description:` on all five file kinds is the router payload; [`models/llm-memory-db-mssql`](../../models/llm-memory-db-mssql) is the fully realized exemplar) | model-index-routing | model-index-routing | folder-format | SKILL core rule (description-always, reserved `index_file`), entity-flow E1b, dfd-authoring F6a, conventions (reserved filename), verification (`validate --index` in the loop + `config.index_file_*` / `index.*` rule rows) | โ—† **Process flows โ€” implemented and first-class, now an in-app view.** The `ignatius flow` CLI subcommand has been removed; flows are the **Flows** view inside the unified SPA (`serve`) and are included in the single `export -o model.html` file. The process dictionary is fused into the unified **Dictionary** view (no separate `/flow-dict` route). The **flow viewer is a custom SVG renderer** ([`src/flow-view/`](../../src/flow-view), separate from the ERD's Cytoscape): Gane-Sarson notation (open-ended `D#` stores, numbered process hubs, green externals), banded DFD layout (hub-and-spoke to stores/externals, no process-to-process, shared store bridging). Every node carries a โ“˜ badge โ€” a **`db:` store** opens the rich `SelectedEntityModal` (attributes, relationships, examples) shared with graph nodes; a process / external / non-`db` store opens the plain markdown doc dialog. Flow bodies parse `[[wiki-links]]` that route in-dialog across both flow nodes and ERD entities. [`models/key-inherited`](../../models/key-inherited) carries demo DFDs (`order-to-cash` with a sub-DFD + `refund`). Skill coverage: the `ignatius-modeling` skill's `flow` mode authors flow markdown (see the skill-modes row above). Guide coverage: [`docs/guides/flows.md`](../guides/flows.md) (folder format, process frontmatter, endpoints, sub-DFDs, viewing) plus the flow rule catalog in [`docs/guides/validation.md`](../guides/validation.md). See [`docs/spec/process-flows.md`](../spec/process-flows.md) Non-goals + the design's Open questions for the deferred set (queue-payload validation, usage index). diff --git a/docs/wiki/flow-view.md b/docs/wiki/flow-view.md index cb2ae98..f317fa0 100644 --- a/docs/wiki/flow-view.md +++ b/docs/wiki/flow-view.md @@ -1,40 +1,142 @@ --- type: Domain -description: ELK-based layout and SVG rendering for DFD diagrams โ€” positions, edge routing, zoom math, and diagram chrome. +description: ELK/banded layout, stack-node grouping, and SVG rendering for DFD diagrams โ€” positions, edge routes, chips, chrome. +tags: [flow-view, flows, frontend] --- # flow-view ## What it does -Computes node positions and edge routes for data flow diagrams (DFDs) via elkjs, falls back to a hand-rolled banded positioner when ELK fails or hasn't resolved yet, and renders the result as a custom SVG with pan/zoom/drag, minimap, and edge-hover tooltips. Renderer-agnostic layout helpers are separated from the SVG rendering so the DFD chrome (breadcrumbs, minimap, nav card) can be swapped or tested independently of the layout math. +[`src/flow-view/`](../../src/flow-view) turns a parsed `FlowDiagram` into on-screen positions, routed edges, and a rendered SVG. Without this domain the app has no DFD canvas: `flows` has parsed data with nowhere to draw, and the frontend's Flows tab is empty. Layout runs two ways โ€” an async elkjs pass with 5-band partitioning and orthogonal edge routing, or a synchronous hand-rolled banded fallback used while ELK resolves or after it fails โ€” and both feed the same custom SVG renderer, which also owns pan/zoom/drag, the minimap, breadcrumbs, edge-hover tooltips, and search-driven dimming. -## CLI code +A store is not always drawn one-to-one with an entity. When a process touches two or more stores in one direction, or (in the connected view) two or more stores share a cluster tag, author cluster, subtype family, group, or read/write/kind signature, they collapse into one **stack node** โ€” a single box whose rows can represent individual stores or nested groupings, at a collapse level the caller controls. Two view modes decide how this grouping runs: **per-process** (the default) stacks each process's own reads and writes; **connected** keeps one node per store and only groups where clusters/subtypes/groups/adjacency say to. Both are global settings owned by the frontend shell, not this domain. -- [`src/flow-view/elk-flow-layout.ts`](../../src/flow-view/elk-flow-layout.ts) (353L) โ€” async ELK layout: `computeElkLayout(diagram, opts?): Promise` runs elkjs Layered with 5-band partitioning (source-ext=0, input-store=1, process-row=2, output-store=3, sink-ext=4) and `ORTHOGONAL` edge routing; matches the band contract documented in [`docs/spec/dfd-overhaul.md`](../spec/dfd-overhaul.md) C4. Returns `{ positions, edgeRoutes }` where `positions` are node centers (ELK top-left + half size) and `edgeRoutes` are full routed polylines. Also exports `terminateQuietly(elk)` (swallows the browser main-thread shim's missing `terminate()`), `isInlineLabel(label)` / `SHORT_LABEL_MAX = 22` (inline-chip length gate), `nodeSize(n)`, and `bandOf(n, srcSet)`. No Bun/Node-only APIs at module top level โ€” browser-safe. -- [`src/flow-view/flow-layout.ts`](../../src/flow-view/flow-layout.ts) (603L) โ€” renderer-agnostic layout helpers, extracted from `App.tsx` so the SVG renderer doesn't pull in Cytoscape. `buildFlowData(diagram): FlowRenderData` builds the role-split node model: `buildExternalRouting` caps each external at two aggregated copies max (`ext:--src` source-role aggregate in band 0, `ext:--snk` sink-role aggregate in band 4); stores both read and written are split into `--read` / `--write` copies. Also exports `assignStoreNumbers`, `normalizeEdgeData(data): string[]` (edge data โ†’ hover-tooltip lines), `computeFlowLayout` (the synchronous banded-preset fallback layout), and `processNodeSize(label): { lines, width, height }` โ€” pure label-wrapping/box-sizing helper consumed by both `elk-flow-layout.ts`'s `nodeSize` and the SVG renderer, so ELK and the drawn box always agree. -- [`src/flow-view/FlowDiagramSvg.tsx`](../../src/flow-view/FlowDiagramSvg.tsx) (1763L) โ€” the SVG renderer. `FlowDiagramSvg(props: FlowDiagramSvgProps)` consumes `elkPositions?` / `elkEdgeRoutes?` (from `computeElkLayout`); position priority is `savedPositions` (drag overrides) โ†’ `elkPositions` โ†’ `bandedPositions` (`computeFlowLayout`), and an edge only draws the ELK polyline when neither endpoint has been dragged off its ELK base position. Owns pan/zoom/drag interaction, the edge-hover tooltip (`edgeTooltip` state, `tooltipClearTimer` flicker guard), inline-vs-truncated label chips (`truncateLabel`, `isInlineLabel`), and search-driven dimming (`searchTokens` prop, `nodeOpacity`/`edgeOpacity`, `baseToken(id)` which strips the `--src`/`--snk`/`--read`/`--write` layout suffixes). Exports `DARK_PALETTE` / `LIGHT_PALETTE` (`FlowPalette` type), `ElkPositionMap`, `MinimapData`, `FlowDiagramSvgProps`. -- [`src/flow-view/FlowChrome.tsx`](../../src/flow-view/FlowChrome.tsx) (449L) โ€” floating chrome around the SVG: breadcrumb chips, DFD nav card (shown when more than one top-level diagram exists), and the bottom-left minimap (`FlowMinimap`, reads live `MinimapData` off an imperative `FlowChromeHandle` ref: `setStack`, `setDiagrams`, `setMinimap`, `setMinimapPanTo`). `.flow-minimap-wrapper`'s left offset is set inline per `showNav`: `16px` when the nav card is hidden (matching the DG `.minimap` left offset) or `228px` when it's visible, to clear the nav card. A `breadcrumbRef` + `ResizeObserver` writes the breadcrumb row's measured bottom edge into the `--flow-search-bar-top` CSS custom property on `document.documentElement` so the flow search bar (mounted in `App.tsx`) always clears the breadcrumb row regardless of drill depth. -- [`src/flow-view/zoom-scale.ts`](../../src/flow-view/zoom-scale.ts) (74L) โ€” pure zoom/fit math, no DOM/React/Bun imports. Implements the native-1:1 zoom model: `100%` means one diagram world-unit renders as one CSS pixel, not "fits the container." Exports `computeFitScale(content, container)` (viewBoxโ†’container scale under `preserveAspectRatio="xMidYMid meet"`, returns 1 for degenerate dimensions), `screenScaleToPercent(internalScale, fitScale)`, and `percentToScreenScale(pct, fitScale)`. +## How it works -## Docs +### Position priority -- [`docs/spec/dfd-overhaul.md`](../spec/dfd-overhaul.md) โ€” directly cited in `elk-flow-layout.ts`'s `bandOf` comment as the source of the 5-band layout contract (C4); companion design doc [`docs/design/dfd-overhaul.md`](../design/dfd-overhaul.md) covers the leveling + ELK-layout redesign that introduced this domain's ELK pipeline. -- [`docs/research/dfd-layout-and-leveling.md`](../research/dfd-layout-and-leveling.md) โ€” evidence base feeding the ELK-layout design. -- [`docs/design/dfd-edge-hover-data.md`](../design/dfd-edge-hover-data.md) and [`docs/spec/dfd-edge-hover-data.md`](../spec/dfd-edge-hover-data.md) โ€” design/spec for the edge-hover tooltip (CP1 `dataLines`, CP2 the styled HTML tooltip), both implemented in `FlowDiagramSvg.tsx` and `flow-layout.ts`. -- [`docs/spec/viewer-ux-polish.md`](../spec/viewer-ux-polish.md) โ€” source of the native-1:1 zoom model implemented in `zoom-scale.ts` and wired through `FlowDiagramSvg.tsx`'s `onZoomChange`/`onRegisterZoomControl` (comments cite it by name as "#3 viewer-ux-polish" and "#4"). -- [`docs/design/graph-flow-search.md`](../design/graph-flow-search.md) and [`docs/spec/graph-flow-search.md`](../spec/graph-flow-search.md) โ€” source of the `searchTokens` dimming feature (CP3/CP5) in `FlowDiagramSvg.tsx` and `FlowChrome.tsx`. +A node's on-screen position resolves from whichever source is authoritative for it, checked in order โ€” a user's own drag always wins. -## Coupling +```mermaid +flowchart TD + A[node position requested] --> B{"savedPositions has this id?"} + B -->|yes| C["use the saved (dragged) position"] + B -->|no| D{"elkPositions has this id?"} + D -->|yes| E[use the ELK position] + D -->|no| F["use computeFlowLayout's banded position"] +``` -- `flows` ([`src/flows/`](../../src/flows)): flow-view imports `FlowDiagram` and related types from [`src/flows/flow-parse.ts`](../../src/flows/flow-parse.ts) โ€” every layout/render entry point takes a parsed `FlowDiagram` as input. A change to the parsed shape (node/edge/store/external fields) forces changes here; flow-view never parses, validates, or levels diagrams itself. -- `frontend` ([`src/app/`](../../src/app)): [`src/app/views/flow/FlowsView.tsx`](../../src/app/views/flow/FlowsView.tsx) is the sole orchestrator โ€” it calls `computeElkLayout`, catches ELK failures so `elkPositions`/`elkEdgeRoutes` stay `undefined` and the renderer falls back to the banded layout, and passes `FlowDiagramSvg`/`FlowChrome` their props. [`src/app/views/flow/LegendModal.tsx`](../../src/app/views/flow/LegendModal.tsx) imports `DARK_PALETTE`/`LIGHT_PALETTE` from `FlowDiagramSvg.tsx` directly. `FlowDiagramSvg.tsx` also imports `PositionMap` from `src/app/views/graph/layout-store` for the drag-persistence shape. -- `theme` ([`src/theme/`](../../src/theme)): `flow-layout.ts` and `FlowDiagramSvg.tsx` import `FlowKindKey` / `FlowKindEntry` from `src/theme/theme-defaults` for kind-colored store/external fills. -- [`src/app/logic/search.ts`](../../src/app/logic/search.ts)'s `searchFlowDiagrams` produces the same base tokens (role-split suffixes stripped) that `FlowDiagramSvg.tsx`'s `baseToken` strips to match against โ€” a change to either suffix scheme has to stay in sync with the other. +`FlowDiagramSvg`'s `nodeBounds` and `elk-flow-layout.ts`'s `nodeSize` share their process and stack sizing (`processNodeSize`, `stackNodeSize`), so those two node types never disagree about where a box's edges are. External sizing is not shared: `nodeSize` computes an external's width as `estW(n.label, 6.6, 28, 110)` (label-length-dependent, 110px floor) with a fixed 52px height, while `nodeBounds` draws the renderer's own fixed `EXT_W = 120`, `EXT_H = 50` โ€” ELK lays out an external against a box the SVG does not actually draw. For a stack node, `nodeBounds` centers the box on `stackNodeSize`'s official height (the same top ELK/`StackNode` use) but reports `stackRowLayout`'s full drawn height for `h` โ€” the box's own peek-reserve padding is included in what chip placement and the viewBox treat as occupied space, even though ELK never sees it. + +### Edge routing fallback + +An edge draws ELK's routed polyline only when neither endpoint has moved off the position ELK computed for it; a dragged node reverts its edges to the live hand-router. + +```mermaid +flowchart TD + A["edge has an elkEdgeRoutes entry?"] -->|no| D["orthogonalPath (hand-routed)"] + A -->|yes| B{"source and target both at their ELK base position?"} + B -->|yes| C[draw the ELK polyline] + B -->|no| D +``` + +### View routing in buildFlowData + +`buildFlowData(diagram, opts)` is the single entry point for node/edge construction; `opts.view` picks one of three independent builders. + +```mermaid +flowchart TD + A["buildFlowData(diagram, opts)"] --> B{"opts.view"} + B -->|"undefined"| C["one node per store; a store both read and written splits into --read/--write copies"] + B -->|"'per-process'"| D["buildPerProcessStores: 2+ stores touched by one process in one direction collapse into a stack node"] + B -->|"'connected'"| E["buildConnectedViewGrouping: cluster/subtype/group/adjacency passes, per-store otherwise"] +``` + +`computeElkLayout` takes the same `opts` (`ComputeElkLayoutOpts extends BuildFlowDataOpts`) so ELK's node/edge id set always matches what `FlowDiagramSvg` builds against. + +### Connected-view grouping order + +Within the connected view, five passes run in a fixed order; each pass claims members from what the previous pass left, per process. + +```mermaid +flowchart TD + A["explicit `cluster:` tag on an edge (no threshold, claims for every touching process)"] --> B["author clusters (opts.clusters, 2+ per process)"] + B --> C["subtype families (opts.subtypeClusters, 2+ per process)"] + C --> D{"collapseLevel === 'groups'?"} + D -->|yes| E["entity groups (opts.entityGroups, 2+ per process)"] + D -->|no| F{"adjacencyStacks !== false?"} + E --> F + F -->|yes| G["adjacency: same kind + same readers + same writers, diagram-wide, 2+"] + F -->|no| H[remaining touches stay plain per-store nodes] + G --> H +``` + +An author cluster's 2-per-process qualification (`qualifyPerProcess`, `STACK_THRESHOLD`) runs before the explicit-tag `existing` check, whether or not a `cluster:--` group already exists from step one โ€” a process touching fewer than two of the cluster's members contributes nothing and is skipped, even when an explicit-tag group with the same id is already forming. Only a process that clears the threshold merges its members into that existing group, or starts a new one when no explicit-tag group exists. A process that touches only one member of a group another process qualified for still renders that member as its own plain node, flagged `duplicated`. + +### Collapse level and a stack's dialog rows + +`buildStackRows` (internal to `flow-layout.ts`) decides what each row inside a stack node represents, independent of which view produced the stack. + +| Collapse level | Row content | +|---|---| +| `stores` | One row per member store; no grouping. | +| `clusters` | One row per explicit `cluster:`-tagged group and per subtype family with 2+ members in this stack, then one row per remaining loose store. | +| `groups` | One row per entity group with 2+ members in this stack (nesting its cluster/subtype and table rows as `children`), then a cluster/subtype spanning 2+ groups as its own row, then loose tables. | -## Conventions worth knowing +A `store` row caps with its own `D#`; a `cluster`/`subtype` row caps `C`; a `group` row caps `G` โ€” never a number, since only a store row is one. `stackRowBodyText(row)` returns the text a row renders (a store row's `displayName`, a grouped row's `