fix(bindx): unite the occurrences of an entity within one server response

[matej21]

Closes #123.

The bug

One query can reach the same entity through several paths, each with its own sub-selection. For example, article.attachments(a => a.name().type()) sits next to article.sections(s => s.article(r => r.attachments(a => a.name()))). Every occurrence is written into the entity's snapshot on its own path:

• the root through useEntity / useEntityList;
• nested ones when HasOneHandle / HasManyListHandle propagate embedded data.

When the narrower occurrence was written last, it replaced the fields the wider one carried, and those fields read back as undefined. Nothing threw. The outcome depended only on render order.

The fix: a response-local union

The occurrences of one server response are equally fresh, so uniting them needs no recency rule.

• Index. When a response enters the store, SnapshotStore.indexServerResponse groups the entity occurrences in it by entity type and id (store/ResponseOccurrenceIndex.ts). Each occurrence of a repeated entity resolves to the frozen shallow union of all of them. An entity that occurs once is left as is, with no allocation.

• Walk plan. Occurrences of one type at one selection node carry the same fields, so uniting them changes nothing. For each response, planOccurrenceWalk lists:

  • the entity types the selection reaches at two or more positions;
  • the selections on a path from the root to those positions.

  A selection with no repeated type skips the walk entirely. Otherwise only those paths are walked, and only those types are collected.

  • The climb to the root tracks tree positions, not selection objects. A reused fragment is one selection object at several positions, and each position has ancestors of its own.
  • The plan is computed per response, not cached. A selection can still grow after its first fetch: mergeSelections extends nested selections in place, and a fragment's selection is shared. The plan costs one pass over the selection tree (about 5 µs for a small cyclic selection) and does not depend on the response size.

• Entity types come from the schema through the selection. The walk starts from the root type and resolves each relation with SchemaRegistry.getRelationTarget(parentType, fieldMeta.fieldName). Two entity types that share an id never meet. A relation whose target the schema cannot resolve is left out of the index; it is never matched by id alone.

• Write sites write resolveServerOccurrence(occurrence) instead of the raw occurrence:

  • useEntity
  • useEntityList (the whole list response is indexed together)
  • HasManyDataGrid for the rows it loads
  • HasOneHandle and HasManyListHandle when they propagate embedded data

• Nested relations unite level by level. The union is shallow: a relation value inside it is one occurrence's raw value. The related entities in that value resolve to their own unions when they are written. So the response is never rewritten, and a cycle in it (an entity reached through itself) cannot produce a cyclic object.

• Paginated has-many: array items and the { edges: [{ node }] } nodes of a connection are occurrences like any other, united by type and id. The list array itself, including its non-enumerable totalCount, is untouched.

• Conflicting scalars: if two occurrences disagree on a scalar they both selected, which one server read should not produce, the occurrence visited last wins. The walk is breadth first, with relations in selection order and list items in list order. The outcome therefore follows from the response and the selection alone.

What this deliberately does not cover

• Separate reads replace each other, as on main. A refetch, a read through another root, and the refresh after a persist are separate responses. They are never merged with what the store already holds.
  • Deciding which of two copies from different reads is newer needs a recency model in the store. The discussion on this PR covers why (see the review comments).
  • A narrower separate read can still replace fields of a wider one in the shared parent snapshot. That is the existing behaviour, left for a separate change.
• EntityLoader (core, not used by the React hooks) writes each response without uniting occurrences, so A narrower nested selection of the same entity overwrites the wider one's relation data (fields read back undefined) #123 remains for its users. Uniting needs the selection and the schema; the loader has only a QuerySpec. Its JSDoc now says so. Covering it means either passing a selection and a schema resolver into the loader, or teaching the index to walk a QuerySpec. Both are left for a follow-up.

Tests

tests/react/relations/hasMany/nestedSameEntityNarrowerSelection.test.tsx runs every case in both read orders: narrower nested occurrence first, and wider direct occurrence first.

• One response:
  • the cycle from A narrower nested selection of the same entity overwrites the wider one's relation data (fields read back undefined) #123 for a has-many, a has-one, and a has-one inside a has-many item;
• an entity reached through a reused fragment, as the article's author and as a section's reviewer, and also as the editor. The editor has a wider has-one and has-many than the fragment's. In the editor-first order this fails when the walk plan skips the fragment's second position;
  • the same across the items of a useEntityList response;
  • a paginated has-many selected with the same params in both occurrences (one aliased key);
  • two entity types sharing an id, with no fields leaking between them.
• Separate reads behave as before:
  • persist, then a narrower read: the persisted values stay;
  • a server change, then a refetch: the new value shows;
  • a fresher read through another root is not overwritten by an older read.

tests/unit/store/responseOccurrenceIndex.test.ts covers the index itself. The walk-plan cases are:

• no repeated type for a selection that reaches every type once;
• a cycle's repeated types are found;
• only the relations leading to a repeated type are walked;
• a response whose selection repeats no type is not walked: schema calls are the same for 10 and 1000 rows;
• the ancestors of every position a reused fragment takes are walked;
• a cyclic selection is walked.

It also covers:

• the union, and that a lone occurrence is returned as is;
• a relation inside a relation;
• connection nodes;
• type separation for a shared id;
• a relation with no known target;
• the deterministic conflict rule;
• a list response indexed together.

Mutation checks:

• index disabled: every nested-first case fails. Without the fix, that order already breaks at the first load. Every direct-first case passes, which shows the separate-read guards match main.
• index keyed by id alone: the shared-id case fails in both orders.

Performance

20k rows, each with three entities (row, has-one, a has-one under it), run under cpu-lease -n 2 --no-smt. Times are the median of 30 and cover indexing plus resolving every row.

Response	Time
not indexed (main path)	0.16 ms
selection repeats no type (fast path, plan computed per response)	0.24 ms
each row also reaches itself through a relation (walked, pruned to the cycle)	19.1 ms
for scale: JSON.parse(JSON.stringify(rows))	23.4 ms

Before the walk plan, the cyclic case walked every relation and took about 44 ms.

Verification

• bun run typecheck: clean
• bun run lint: 0 errors (existing warnings only)
• bun run test: 2137 pass, 0 fail
• Merged with fix(bindx): key has-many state by the schema field name, views inside it #122 (fix/hasmany-view-state) in a scratch worktree: the merge is clean, typecheck is clean, lint has 0 errors, and 2158 tests pass with 0 failures.

Overlap with #122

#122 rewrites has-many store state. This PR does not touch that state. It adds one resolveServerOccurrence call in HasManyListHandle.ensureItemSnapshots and two methods on SnapshotStore. The PRs merge cleanly.

Note for merging with #132

#132 also changes the dependency array of the list-loading effect in useEntityList.ts. Whichever of the two lands second should resolve the conflict as:

[entityType, optionsKey, requestKey, batcher, dispatcher, store, selectionMeta, schemaRegistry]

History

The first two commits of this branch merged a narrower read into what the store already held, across reads. Review showed that no fixed precedence between the related snapshot and the embedded copy is right in both directions. The last commit removes that merge and replaces it with the response-local union described above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ge3kRufpi9b9HMBSBjRze5

[matej21]

Review round 2: the fill rule needs a recency model, and adding one is a store design change

I'm stopping here rather than patching. Both findings are right, and a clean fix changes how the store tracks where a server value came from. That is a maintainer call. Below are the analysis and the options.

What is wrong at 432f5d9

1. The fixed precedence is wrong in one of the two directions. When a narrower read leaves a key out, the fill has two candidate copies:

• the related entity's own snapshot;
• the copy embedded in the parent's relation field.

Neither is always newer.

• Round 1: the snapshot is newer after a persist, or after a read of the entity through another path. The old embedded copy won and was pushed back into the snapshot.
• Round 2: the embedded copy is newer after a parent refetch that no handle has propagated yet. Now the snapshot wins, and the refetched value is dropped.

Repro for round 2 (a root that selects attachments { name type } and sections.article.attachments { name }, nested occurrence read first):

1. Load. type is pdf.
2. The server changes type to docx.
3. Remount and refetch. The nested read lands first and fills type from the related snapshot. Attachment:att-1 and article.attachments both hold pdf in data and serverData, and look clean.

On origin/main this order already fails at step 1 (type is undefined, which is #123 itself). With the direct occurrence read first, both main and this branch pass.

2. The lookup is by id alone (idIndex), so an entity of another type with the same id can leak keys. Elsewhere the store assumes ids are globally unique, which is why I used the index. The lookup should still be by targetType:id. The store has no schema today, so the target type has to come from the caller: parent type plus field name, resolved through SchemaRegistry. Handles and hooks have the schema. Every option below needs this, so it is not a separate decision.

Why the existing propagation state is not enough

hasEmbeddedDataChanged / markEmbeddedDataPropagated looks like a ready-made signal: "unpropagated embedded copy means the embedded copy is newer." It does not hold:

• It tracks references, not recency. A merge always produces a new reference, so every merged value looks unpropagated. The next narrower read would prefer the embedded copy again, and round 1's stale refill comes back.
• "Unpropagated" does not mean "newer". A related entity persisted, or read through another root, after an unpropagated parent refetch is newer than that copy.
• It exists only at the top level. It is keyed by (parent, data key), and has-one uses dataFieldName where has-many uses the alias. Nothing tracks the entities nested deeper inside an embedded copy.

Options

A. Per-field server read clock (recommended)

• Store change: EntitySnapshotStore keeps a monotonic read clock. Every snapshot records, per field, the clock of the server write that last set that field in serverData. A value embedded under a parent's relation field carries that field's clock.
  • Merge: for each unread key, take the copy with the higher clock. The merged field then gets the current clock. That is safe, because the merged value is the newest one known at that moment.
  • Propagation: HasOneHandle / HasManyListHandle pass the parent field's clock into the related entity's refresh. A field whose clock is newer is not replaced.
• Rule: the latest-arriving server value wins per (entity, field), in both directions. That matches how refreshServerData already treats top-level fields.
• Risks:
  • Every writer of the server baseline has to maintain the clock: setData(server), refreshServerData, commit, commitFields, rekey, remove, clear, and snapshot export/import used by undo.
  • Missing one silently reintroduces an older-wins case.
  • It overlaps with fix(bindx): key has-many state by the schema field name, views inside it #122 in SnapshotStore.
• Side effect: it also changes behaviour that exists on main today. There, a parent copy that was never propagated overwrites a newer value in the related snapshot on first read, because the propagation guard compares references, not recency. Reviewers would need to accept that change.

B. Normalize on arrival (eager propagation)

• Store change: when a server read lands, write every embedded entity into its own snapshot right away. This needs the schema for target types, applied recursively. The snapshot is then always at least as new as any embedded copy, and "snapshot wins, embedded copy only when there is no snapshot" becomes correct. Embedded values carry only membership and identity. Render-time propagation of server data becomes redundant.
• Rule: the same one as A, with no clock bookkeeping.
• Risks:
  • Cost moves from read time to fetch time. A refetch of a large tree writes, dirty-checks and notifies every related snapshot up front. That is the fan-out cost Selection-aware parent propagation to stop hub fan-out #86 is about.
  • It is the larger conceptual shift, because the lazy propagation path changes role.

C. Don't fill; hide what the read did not carry

• Keep wholesale replacement, and fix the symptom in handles instead: when a list or has-one is materialized from embedded data that lacks a selected key, re-read the key from the related snapshot rather than from the embedded item.
• Store change: none.
• Risk: it only works when a related snapshot exists. A key nobody has materialized is still lost, which is the original A narrower nested selection of the same entity overwrites the wider one's relation data (fields read back undefined) #123 case. This does not fix A narrower nested selection of the same entity overwrites the wider one's relation data (fields read back undefined) #123. Listed for completeness.

Recommendation

A, as its own PR that adds the clock, with #133 rebased onto it. It keeps the lazy propagation and its performance profile, and it gives one rule that is testable in both directions.

B is simpler to reason about. If you would rather normalize on arrival, it is the better long-term model, but it should be measured against the fan-out concerns in #86 first.

I have not pushed anything for round 2. The branch stays at 432f5d9 and is not ready to merge: it fixes #123 and the persist case, but it regresses the refetch case above.

Once a direction is chosen, these tests go in alongside it, each in both read orders:

• refetch after a server change;
• persist, then a narrower read;
• a fresher read through another root;
• a type-scoped lookup with an id shared by two entity types.