fallback list and stats when LanceDB metadata projection is empty

[TurboTheTurtle]

Summary

• retry MemoryStore.list() and MemoryStore.stats() without projection when a projected LanceDB metadata read returns zero rows
• keep the same scope/category filters on the fallback query
• add a regression test for the projection-empty/list-stats mismatch

Closes #607.

Validation

• node --test test/store-list-stats-projection-fallback.test.mjs
• node scripts/verify-ci-test-manifest.mjs
• npm run test:storage-and-schema
• npx tsc --noEmit
• git diff --check

If maintainers squash/rework this PR, please preserve author attribution or include:

Co-authored-by: Andy Ye <35905412+TurboTheTurtle@users.noreply.github.com>

[rwmjhb]

PR #811 Review: fallback list and stats when LanceDB metadata projection is empty

Verdict: RESOLVE-CONFLICTS-FIRST | Author: TurboTheTurtle | Merge state: DIRTY

Pipeline short-circuited at the conflict gate after R0 verification. Deep review deferred until the branch rebases cleanly onto the base.

Problem Statement (R1)

memory_list and memory_stats can report zero memories even when LanceDB contains memories that are retrievable via recall. The PR attempts to keep list/stats aligned with LanceDB-backed recall when projected metadata reads unexpectedly return no rows.

Why This Stopped Here

GitHub reports mergeable=CONFLICTING (merge_state_status=DIRTY) for this PR. Reviewing the diff now would:

1. Give feedback against a branch the author must rewrite anyway
2. Produce findings that may be invalidated by the conflict resolution
3. Waste review cycles on code that cannot be merged as-is

Recommended Action

Author should:

1. Rebase this branch onto the latest base (or merge the base into this branch)
2. Resolve all merge conflicts
3. Push the rebased branch — the re-review will be picked up automatically

---

Reviewed at 2026-05-22T14:13:53Z | R0+R1 gate | Conflict gate

[TurboTheTurtle]

Merged current upstream master into this branch and resolved the manifest conflict by keeping both the upstream timestamp-normalization test entry and this PR's list/stats projection fallback test entry.\n\nVerification run locally:\n\n\nnode --test test/store-list-stats-projection-fallback.test.mjs test/store-timestamp-normalization.test.mjs test/storage-path-normalization.test.mjs\n\n\nResult: 16 passing tests.

[rwmjhb]

PR #811 Review: fallback list and stats when LanceDB metadata projection is empty

Verdict: APPROVE | 6 rounds completed | Value: 55% | Size: MEDIUM | Author: TurboTheTurtle

Value Assessment

Problem: MemoryStore.list() and MemoryStore.stats() can return zero memories when LanceDB contains retrievable memories, apparently because projected metadata reads can return an empty result set. The PR tries to keep list/stats aligned with LanceDB-backed recall by retrying the same filtered query without projection.

Dimension	Assessment
Value Score	55%
Value Verdict	review
Issue Linked	true
Project Aligned	true
Duplicate	false
AI Slop Score	0/6
User Impact	medium
Urgency	medium

Open Questions:

• The provided issue context has no maintainer labels, assignment, or issue comments, so maintainer acknowledgment of #607 cannot be confirmed from this packet.
• Because stale_base is true, confirm current mergeability before spending detailed review time, although this does not reduce the PR's value.

Summary

MemoryStore.list() and MemoryStore.stats() can return zero memories when LanceDB contains retrievable memories, apparently because projected metadata reads can return an empty result set. The PR tries to keep list/stats aligned with LanceDB-backed recall by retrying the same filtered query without projection.

Evaluation Signals

Signal	Value
Blockers	0
Warnings	1
PR Size	MEDIUM
Verdict Floor	approve
Risk Level	high
Value Model	codex
Primary Model	codex
Adversarial Model	claude

Nice to Have

• F2: Fallback filter preservation is not regression-tested
• EF1: Static verification reports new any types
• MR1: Unprojected fallback loads full rows (including vector embeddings) — the opposite of projection's intent
• MR2: Fallback cannot distinguish a projection bug from a genuinely empty result — every empty list/stats now issues two queries

Recommended Action

Ready to merge.

---

Reviewed at 2026-05-23T04:16:07Z | 6 rounds | Value: codex | Primary: codex | Adversarial: claude