Skip to content

[NIDX-04 workstream] Replace the Stream element index #14004

Description

@hanahmily

[NIDX-04 workstream] Replace the Stream element index

Parent: #13990
Blocked by: #14003

Tracking parent — do not apply Backlog. Split this workstream only after NIDX-03 closes, using the actual native writer, reader, posting, and receive seams then present on main.

End-state boundary

banyand/stream.newElementIndex selects the native idx batch writer, filter executor, posting collector, sorter, snapshot path, restart path, and external receiver.

Required behavior

  • Committed batches become visible at the documented near-real-time boundary.
  • Exact, existence, range, AND/OR/NOT, HAVING, IN, prefix, wildcard, and MATCH return fixed document-ID sets.
  • Keyword, simple, standard, and URL analysis produce fixed terms; MATCH AND/OR overrides remain distinct.
  • Document-ID/timestamp postings and ascending/descending doc-value sort are deterministic.
  • Legacy/native snapshots and raw segments are cross-readable; interrupted receive never publishes partial data.
  • Scoring, phrase/fuzzy/geo, highlighting, facets, explanations, collectors, and plugins remain absent.

Just-in-time decomposition gate

After #14003 closes, file ordered leaves around live Stream callers and one observable feature each. Every leaf needs a pre-agreed seam, literal terms/document IDs, a RED command, an e2e command, resource/concurrency bounds, and same-file rollback. Do not merge analyzers, filters, postings, sorters, or receiver code without activating a production path. Apply Backlog to one unblocked leaf only.

Completion criteria

All future leaves merge; the complete newElementIndex role is native; fixture, NRT, analyzer, posting, sort, restart, receive, race, and compatibility suites pass; and this workstream closes before NIDX-05 decomposition.

Lexical non-regression

Repository changes for every implementation leaf before the final removal may delete existing references but must add zero new case-insensitive bluge tokens and zero matching tracked paths. The gate includes imports and aliases, function/type/variable names, filenames and runtime names, strings, comments and messages, tests, fixture/provenance data, scripts, configuration, and generated assets. Compatibility evidence uses neutral legacy oracle or compatibility writer labels plus an immutable revision or content hash rather than adding a retired module name.

This lexical gate applies to repository changes, not to issue or archived-design prose that names the dependency in order to specify its removal.

Design

BDB-NIDX-SPEC-001 revision 0.2 — NIDX-04 behavior

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions