[NIDX-01E] Cut over Property repair search-after
Parent workstream: #14002
Blocked by: #14011
Summary
Add the bounded doc-value tuple page required by Property repair and move repair tree construction off Bluge. This final NIDX-01 leaf implements only ascending (group, name, entityID, timestamp) ordering, strict search-after, stored SHA projection, and pinned-generation behavior.
Blocked by
NIDX-01D (#14011) must be merged before this work starts. Before applying Backlog, update inherited visitor identifier names to match main; do not widen the behavior.
Pre-agreed test seam
The product seam is one complete Property repair build: provide a committed shard snapshot to the existing repair build operation and observe its persisted repair tree/state output. Tests may exercise the bounded native tuple-page operation through the BanyanDB-owned read-only visitor merged by earlier leaves, but must not target doc-value chunk decoders, heaps, or ICE types.
No generic collector, arbitrary sort expression, offset pagination, or relevance order is part of the seam.
Independent fixture
The checked-in legacy Property shard contains these visible repair rows in declared ascending order:
(g-a, n-a, e-1, 10, sha-a)(g-a, n-a, e-1, 20, sha-b)(g-a, n-b, e-2, 5, sha-c)(g-b, n-a, e-3, 7, sha-d)
With page size 2, page one is rows 1–2; the strict cursor is (g-a,n-a,e-1,20); page two is rows 3–4. The completed repair tree maps g-a/n-a/e-1 → sha-b, g-a/n-b/e-2 → sha-c, and g-b/n-a/e-3 → sha-d.
Requirements
R1. The native page orders encoded doc values by all four ascending components and resumes strictly after the complete prior tuple on the same pinned generation. It returns no duplicate or skipped row.
R2. Each page holds at most the configured page size plus bounded comparison state; it never performs a growing offset scan or materializes the shard.
R3. The existing repair build uses native pages and produces exactly the declared three entity/SHA leaves and its normal generation state.
R4. A generation published after page one does not enter page two. Missing sort values, malformed doc-value chunks, invalid cursors, cancellation, and limit overflow return typed bounded errors and publish no partial repair state. CRC32 remains ignored.
RED and end-to-end contract
- Boundary RED: request the two declared pages through the native tuple-page seam; earlier leaves do not support doc values or search-after.
- E2E RED: run a complete Property repair build at page size
2, publish another legacy generation between pages, and verify the tree contains only the pinned fixture's three declared leaves and SHA values.
Acceptance criteria
go test ./pkg/index/inverted/... ./banyand/property/db -run 'Test(NativeRepairTuplePage|E2EPropertyRepairNative)' -count=1 passes, including under -race.- A call-graph/dependency assertion proves Property repair no longer opens a Bluge reader or constructs a Bluge
TopNSearch.
- Pagination, concurrent publication, corrupt doc values, invalid cursor, cancellation, and bounded-memory cases pass.
make test-ci PKG='./pkg/index/inverted/... ./banyand/property/db', make build, and make pre-push pass.
Scope
Packages: pkg/index/inverted, its private doc-value reader, and banyand/property/db repair.
Out of scope: arbitrary sort, descending repair order, range/prefix/wildcard/MATCH queries, online Property writer/query cutover, snapshots written by native code, merge, expiry, and GC.
Compatibility and rollback
This slice emits only the existing repair state/tree artifacts, not index bytes. Roll back repair reads to the retained Bluge path; the source ICE/snapshot directory remains unchanged.
Lexical non-regression
Repository changes for every implementation leaf before the final removal may delete existing references but must add zero new case-insensitive bluge tokens and zero matching tracked paths. The gate includes imports and aliases, function/type/variable names, filenames and runtime names, strings, comments and messages, tests, fixture/provenance data, scripts, configuration, and generated assets. Compatibility evidence uses neutral legacy oracle or compatibility writer labels plus an immutable revision or content hash rather than adding a retired module name.
This lexical gate applies to repository changes, not to issue or archived-design prose that names the dependency in order to specify its removal.
Design
BDB-NIDX-SPEC-001 revision 0.2 — NIDX-01
[NIDX-01E] Cut over Property repair search-after
Parent workstream: #14002
Blocked by: #14011
Summary
Add the bounded doc-value tuple page required by Property repair and move repair tree construction off Bluge. This final NIDX-01 leaf implements only ascending
(group, name, entityID, timestamp)ordering, strict search-after, stored SHA projection, and pinned-generation behavior.Blocked by
NIDX-01D (#14011) must be merged before this work starts. Before applying
Backlog, update inherited visitor identifier names to match main; do not widen the behavior.Pre-agreed test seam
The product seam is one complete Property repair build: provide a committed shard snapshot to the existing repair build operation and observe its persisted repair tree/state output. Tests may exercise the bounded native tuple-page operation through the BanyanDB-owned read-only visitor merged by earlier leaves, but must not target doc-value chunk decoders, heaps, or ICE types.
No generic collector, arbitrary sort expression, offset pagination, or relevance order is part of the seam.
Independent fixture
The checked-in legacy Property shard contains these visible repair rows in declared ascending order:
(g-a, n-a, e-1, 10, sha-a)(g-a, n-a, e-1, 20, sha-b)(g-a, n-b, e-2, 5, sha-c)(g-b, n-a, e-3, 7, sha-d)With page size
2, page one is rows1–2; the strict cursor is(g-a,n-a,e-1,20); page two is rows3–4. The completed repair tree mapsg-a/n-a/e-1 → sha-b,g-a/n-b/e-2 → sha-c, andg-b/n-a/e-3 → sha-d.Requirements
R1. The native page orders encoded doc values by all four ascending components and resumes strictly after the complete prior tuple on the same pinned generation. It returns no duplicate or skipped row.
R2. Each page holds at most the configured page size plus bounded comparison state; it never performs a growing offset scan or materializes the shard.
R3. The existing repair build uses native pages and produces exactly the declared three entity/SHA leaves and its normal generation state.
R4. A generation published after page one does not enter page two. Missing sort values, malformed doc-value chunks, invalid cursors, cancellation, and limit overflow return typed bounded errors and publish no partial repair state. CRC32 remains ignored.
RED and end-to-end contract
2, publish another legacy generation between pages, and verify the tree contains only the pinned fixture's three declared leaves and SHA values.Acceptance criteria
go test ./pkg/index/inverted/... ./banyand/property/db -run 'Test(NativeRepairTuplePage|E2EPropertyRepairNative)' -count=1passes, including under-race.TopNSearch.make test-ci PKG='./pkg/index/inverted/... ./banyand/property/db',make build, andmake pre-pushpass.Scope
Packages:
pkg/index/inverted, its private doc-value reader, andbanyand/property/dbrepair.Out of scope: arbitrary sort, descending repair order, range/prefix/wildcard/MATCH queries, online Property writer/query cutover, snapshots written by native code, merge, expiry, and GC.
Compatibility and rollback
This slice emits only the existing repair state/tree artifacts, not index bytes. Roll back repair reads to the retained Bluge path; the source ICE/snapshot directory remains unchanged.
Lexical non-regression
Repository changes for every implementation leaf before the final removal may delete existing references but must add zero new case-insensitive
blugetokens and zero matching tracked paths. The gate includes imports and aliases, function/type/variable names, filenames and runtime names, strings, comments and messages, tests, fixture/provenance data, scripts, configuration, and generated assets. Compatibility evidence uses neutrallegacy oracleorcompatibility writerlabels plus an immutable revision or content hash rather than adding a retired module name.This lexical gate applies to repository changes, not to issue or archived-design prose that names the dependency in order to specify its removal.
Design
BDB-NIDX-SPEC-001 revision 0.2 — NIDX-01