Register SQLite per-connection state at engine creation for the vector stores (fixes #1568)

[edwinyyyu]

Purpose of the change

Fix #1568. SQLiteVectorStore and SQLiteVecVectorStore registered SQLite per-connection state (PRAGMA foreign_keys=ON; the sqlite-vec extension loader) through a connect listener in their own __init__, on the engine supplied through their params. This is the structural fault #1545 fixed for the segment store, fixed here with the same convention.

Description

The two faults, as #1568 lays them out:

1. A connect listener fires only for connections opened after it is registered. Anything the engine pooled earlier serves the store with foreign keys off -- a cascade delete silently orphans pending-operation rows, nothing errors -- or without the extension, so every vec statement fails with no such module: vec0.
2. Registering listeners on an engine already serving traffic races SQLAlchemy's event dispatch (RuntimeError: deque mutated during iteration; the issue reports it reproduced in Overhaul segment store: shared tables with incarnation-scoped tenant keys (fixes #1544, #1546, #1549) #1545's concurrency tests).

Both are latent on the production path today, since the DatabaseManager creates each engine immediately before constructing its store, but the params API accepts any engine.

What changed:

• database_manager.py: enable_sqlite_foreign_keys(engine) is added -- verbatim the helper from Overhaul segment store: shared tables with incarnation-scoped tenant keys (fixes #1544, #1546, #1549) #1545, at the same location -- and applied to the SQLiteVectorStore engine at creation; load_sqlite_vec_extension(engine) is applied to the SQLiteVecVectorStore engine at creation. The shared relational engine (async_get_sql_engine) is left to Overhaul segment store: shared tables with incarnation-scoped tenant keys (fixes #1544, #1546, #1549) #1545.
• sqlite_vector_store.py: the listener on the caller-supplied async engine is dropped. The listener on the sync engine the store creates itself two lines above stays, since nothing can have been pooled on it yet. startup() checks PRAGMA foreign_keys and refuses an unenforced engine with a directive naming the helper.
• sqlite_vec_vector_store.py: the loader moves out of __init__ into the module-level load_sqlite_vec_extension, next to the store that needs it. startup() probes pragma_function_list for vec_version and refuses with a directive. The probe keeps the check a scalar; calling vec_version() would need its failure told apart from other OperationalErrors.
• Tests: engine fixtures register the helpers at creation (as Overhaul segment store: shared tables with incarnation-scoped tenant keys (fixes #1544, #1546, #1549) #1545's sqlalchemy_sqlite_engine fixture does). Per store: a refusal test and a pooled-before-construction test. The manager tests assert each helper is applied to the engine before the store is constructed.

Testing

Fail-before on untouched main. Each script pools one connection (SELECT 1) before constructing the store, then exercises it:

register_at_creation=False: pending rows before delete=1, orphaned after=1, PRAGMA foreign_keys=0
register_at_creation=False: create_collection FAILED: OperationalError: (sqlite3.OperationalError) no such module: vec0

The same scripts on this branch, registering at engine creation:

register_at_creation=True: pending rows before delete=1, orphaned after=0, PRAGMA foreign_keys=1
register_at_creation=True: create_collection OK

Without registration each store now refuses at startup():

RuntimeError: SQLite engine does not enforce foreign keys, so deleting a collection would silently orphan its pending-operation rows. Register PRAGMA foreign_keys=ON at engine creation (enable_sqlite_foreign_keys) before constructing the store.
RuntimeError: SQLite engine does not load the sqlite-vec extension, so every vec statement would fail. Register the loader at engine creation (load_sqlite_vec_extension) before constructing the store.

• uv run --frozen --all-extras pytest packages/server/server_tests (default lane, integration excluded): 1873 passed, 3 skipped, 0 failed
• ruff check and ruff format --check: clean
• ty check --project packages/server: 17 diagnostics, the same 17 as main (pre-existing nebulagraph and OpenAI stub imports; none on touched lines)

Compatibility

• Out-of-tree callers constructing either store on their own engine must register the matching helper at engine creation; startup() now fails loudly with the directive otherwise. In-tree construction goes through the DatabaseManager, which does this. No config or schema change.
• Relation to Overhaul segment store: shared tables with incarnation-scoped tenant keys (fixes #1544, #1546, #1549) #1545: enable_sqlite_foreign_keys is byte-identical in both PRs at the same location, and git merge-tree of the two branches is conflict-free, so they can merge in either order.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• Breaking change (a store now refuses an engine without its per-connection state at startup(); only out-of-tree callers constructing stores on their own engines are affected)

Fixes/Closes

Fixes #1568

🤖 Generated with Claude Code

https://claude.ai/code/session_01JdJHiDWgrwNEYfARF8Yvic