Skip to content

feat(search)!: make path the whole statement of what projection reads#628

Merged
ddeboer merged 1 commit into
mainfrom
feat/close-derive-hole
Jul 19, 2026
Merged

feat(search)!: make path the whole statement of what projection reads#628
ddeboer merged 1 commit into
mainfrom
feat/close-derive-hole

Conversation

@ddeboer

@ddeboer ddeboer commented Jul 18, 2026

Copy link
Copy Markdown
Member

What

Make a SearchField’s path the complete statement of what the projection reads from the graph, so a schema-driven extraction generator (#548) and its contract test become possible.

Today derive closures receive the framed node and read arbitrary predicates through irisOf / literalsOf / firstLiteralOf, so path is not the whole story – in the Dataset Register’s Dataset type, 13 of 23 predicates are readable only from inside a closure. A generator that emits a CONSTRUCT from the schema would silently miss every one of them.

Changes

  • derive loses its FramedNode argument – its signature is now (document) => unknown. A derive computes only from fields declared before it, never from the graph. path says what to read; derive says what to compute from what was read.
  • Internal fields, consistently across every kind. A field declaring no role (output / searchable / filterable / facetable / sortable) is an internal field: projected into the in-memory document so a later derive can read it, then pruned before the document reaches a writer, and absent from the engine collection definition – not stored, not indexed, no RAM. A new shared isInternalField predicate is the single source both the projection (prunes by it) and the Typesense collection definition (omits by it) use, so they cannot disagree. Previously only zero-role text fields behaved this way; the other kinds wrote and declared unconditionally.
  • The IR readers are unexported (irisOf, literalsOf, firstLiteralOf) – they are now internal to projection.
  • Field-model docs (packages/search/README.md, JSDoc) and the projection / collection-definition tests are updated for the new behaviour; test fixtures that previously relied on a zero-role field being projected now declare the role they actually intend.

Breaking

Breaking for @lde/search: the public derive signature changes and the three IR readers are no longer exported. Consumers migrate their derives to internal fields plus document reads – e.g. the Dataset Register replaces irisOf(node, 'urn:dr:class') with a zero-role classes reference field whose value a derive reads off the document. That consumer migration is out of scope here (separate repo, gated on #548).

Notes

Two pre-existing, degenerate edges are left as-is rather than widening scope: a field literally named id with no role would be pruned (an id-named field already clobbered the document key before this change), and a defaultSortingField pointing at an internal field would reference an omitted collection field (a default sort field should be sortable, hence never internal). Both belong in a separate declaration-validation change.

Implements #607.

- Drop the FramedNode argument from `derive`: a derive now computes only
  from the document projected so far, never the graph, so a field's `path`
  is the complete statement of what the projection reads.
- Honour the no-role-means-internal promise for every kind via a shared
  `isInternalField` predicate: an internal field is projected so a later
  derive can read it, then pruned before the writer, and omitted from the
  Typesense collection definition (not stored, not indexed, no RAM).
- Unexport the IR readers (`irisOf`, `literalsOf`, `firstLiteralOf`); they
  are now internal to projection.
- Update the field-model docs and tests for the new behaviour.
@ddeboer
ddeboer merged commit 3f5c21b into main Jul 19, 2026
2 checks passed
@ddeboer
ddeboer deleted the feat/close-derive-hole branch July 19, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant