With an NGRAM scalar index on a Utf8 column, a pushed-down contains(col, needle) where needle.len() < 3 returns an empty result set even when rows match, instead of falling back to a full recheck.
NGramIndex::search returns SearchResult::at_least(RowAddrTreeMap::new()) for a sub-trigram needle (no trigram can be derived, so no rows can be guaranteed from the posting lists). AtLeast is documented as a lower bound — a subset of the true matches, "definitely these, possibly more" — so AtLeast(empty) correctly means "the index knows nothing; recheck every row." The scan plan instead treats the returned set as the answer and emits zero rows: silent row loss.
Repro (v9.0.0-beta.21 / rev 1aec1465):
- rows:
["this ramen recipe simmers", "the beta ray shines"], NGRAM index on the column.
contains(text, 'ra') (2 chars, < NGRAM_N) → expected both rows (both contain "ra"), got [].
- the same filter with no index returns both rows.
contains(text, 'ramen') (>= 3 chars) is correct with the index.
Expected: a sub-trigram contains needle degrades to a recheck-all scan and returns the exact matches, matching the no-index result. AtLeast(empty) must never be treated as an exact/complete result set.
Suggested fix: ensure the scalar-index scan path rechecks the predicate against all candidate rows whenever the index returns AtLeast (a lower bound), rather than returning the lower-bound set directly.
Suggested validation: an NGRAM-indexed column with rows matching a 1- and 2-char substring; assert the indexed contains result equals the non-indexed scan result for needles of length 1, 2, and 3.
Related (checked; none is this bug): #4107 "Potential scanner optimization for late materialization plus recheck" concerns the performance of the contains+ngram recheck path (its own example uses contains(caption, 'elephant')), confirming the recheck mechanism is expected to run — this issue is that recheck is skipped entirely for sub-trigram needles, a correctness (wrong-results) bug, not a perf one. #7130 (accelerate regex with ngram, closed) and #7526 (stop-trigram filtering) are unrelated ngram features.
With an NGRAM scalar index on a Utf8 column, a pushed-down
contains(col, needle)whereneedle.len() < 3returns an empty result set even when rows match, instead of falling back to a full recheck.NGramIndex::searchreturnsSearchResult::at_least(RowAddrTreeMap::new())for a sub-trigram needle (no trigram can be derived, so no rows can be guaranteed from the posting lists).AtLeastis documented as a lower bound — a subset of the true matches, "definitely these, possibly more" — soAtLeast(empty)correctly means "the index knows nothing; recheck every row." The scan plan instead treats the returned set as the answer and emits zero rows: silent row loss.Repro (v9.0.0-beta.21 / rev
1aec1465):["this ramen recipe simmers", "the beta ray shines"], NGRAM index on the column.contains(text, 'ra')(2 chars,< NGRAM_N) → expected both rows (both contain "ra"), got[].contains(text, 'ramen')(>= 3 chars) is correct with the index.Expected: a sub-trigram
containsneedle degrades to a recheck-all scan and returns the exact matches, matching the no-index result.AtLeast(empty)must never be treated as an exact/complete result set.Suggested fix: ensure the scalar-index scan path rechecks the predicate against all candidate rows whenever the index returns
AtLeast(a lower bound), rather than returning the lower-bound set directly.Suggested validation: an NGRAM-indexed column with rows matching a 1- and 2-char substring; assert the indexed
containsresult equals the non-indexed scan result for needles of length 1, 2, and 3.Related (checked; none is this bug): #4107 "Potential scanner optimization for late materialization plus recheck" concerns the performance of the
contains+ngram recheck path (its own example usescontains(caption, 'elephant')), confirming the recheck mechanism is expected to run — this issue is that recheck is skipped entirely for sub-trigram needles, a correctness (wrong-results) bug, not a perf one. #7130 (accelerate regex with ngram, closed) and #7526 (stop-trigram filtering) are unrelated ngram features.