Skip to content

Add an optional Neo4j graph read model behind a default-off flag - #6

Open
ShreeBohara wants to merge 1 commit into
mainfrom
feat/neo4j-graph-store
Open

Add an optional Neo4j graph read model behind a default-off flag#6
ShreeBohara wants to merge 1 commit into
mainfrom
feat/neo4j-graph-store

Conversation

@ShreeBohara

Copy link
Copy Markdown
Owner

Builds on #5. Enables the traversals the SQL path structurally cannot do, without making a new datastore load-bearing.

Why

The graph questions that matter for understanding a codebase are transitive: what breaks if I change this file, how does auth reach the database, are there import cycles. The Python implementation can't answer them — hops is hard-capped at 2 in the API because each hop rescans the whole edge list, so depth is quadratic in Python and a single variable-length pattern in Cypher.

Design: projection, not source of truth

code_dependencies (SQL) stays authoritative. Neo4j is projected from it during indexing and rebuildable by re-indexing. Every consequence is deliberate:

  • neo4j_enabled defaults to False — nothing changes unless you turn it on.
  • get_graph_store() returns None when disabled or misconfigured, rather than raising, so callers treat "no graph store" as an ordinary path.
  • Startup verifies connectivity and applies schema, but failure only logs. An unreachable graph database must not stop the API booting.
  • A sync failure during indexing is caught; SQL edges are unaffected. There's a test for exactly this.
  • Repo deletion removes the subgraph before the SQL rows, so a failure leaves authoritative data intact and retryable rather than orphaning a subgraph.

Notable implementation choices

  • sync_repository deletes the subgraph first rather than merging. A MERGE-only sync leaves edges for deleted files behind and drifts into a union of every commit ever indexed.
  • Uniqueness on (repo_id, path) — which is also the index that keeps MERGE off a label scan.
  • Depth is clamped 1..10. An unbounded variable-length pattern is a trivial way to hang the server.
  • COUNT {} subqueries, not Graph Data Science. The in-database GDS plugin requires AuraDB Professional or above, and Aura Graph Analytics sessions are an offline batch shape (2GB, one concurrent session, 30-min TTL) that doesn't fit a synchronous request. There's a test asserting gds. appears nowhere.
  • Batched UNWIND + MERGE at 500 rows, so a large repo isn't one enormous transaction.

New traversals: reachable_from, blast_radius, shortest_path, import_cycles.

What is verified — and what is not

132 tests pass (was 117), ruff clean, app boots with the store returning None by default.

But I want to be explicit: the tests use a fake driver. They cover the queries we send and every fallback path — including that indexing still persists SQL edges when the graph sync fails — but not that the Cypher returns correct results. No query in this PR has executed against a Neo4j server, because there's no Docker on the machine I worked on.

docker/README.md documents the verification path — docker compose up -d neo4j, set NEO4J_ENABLED=true, re-index, then:

MATCH (f:File {repo_id: $repo}) RETURN count(f);
MATCH (:File {repo_id: $repo})-[r:IMPORTS]->(:File) RETURN count(r);

Those should equal count(*) from code_files and code_dependencies. I'd treat this PR as unlanded until someone runs that. The default-off flag is what makes merging it safe in the meantime.

Also

Fixes stale drift in .env.example that still advertised LOCAL_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5 — the HuggingFace id removed from config.py in the first batch, which would have walked a self-hoster straight back into the all-zero-vector index.

AuraDB Free warning, in the docs

A Free instance auto-pauses after 72h idle and a paused instance's hostname stops resolving, so the graph would silently fall back to SQL until someone resumed it by hand. Deleted after 30 days paused. Don't point a public demo at it.

🤖 Generated with Claude Code

Enables the traversals the SQL path structurally cannot do, without making a new
datastore load-bearing.

WHY
The graph questions that matter for understanding a codebase are transitive: what breaks
if I change this file, how does auth reach the database, are there import cycles. The
Python implementation cannot answer them -- hops is hard-capped at 2 in the API because
each hop rescans the whole edge list, so depth is quadratic in Python and a single
variable-length pattern in Cypher.

DESIGN: PROJECTION, NOT SOURCE OF TRUTH
code_dependencies (SQL) stays authoritative. Neo4j is projected from it during indexing
and can be rebuilt by re-indexing. Consequences, all deliberate:
- neo4j_enabled defaults to False, so nothing changes unless it is turned on.
- get_graph_store() returns None when disabled OR misconfigured (missing password)
  rather than raising, so every caller treats "no graph store" as an ordinary path.
- Startup verifies connectivity and applies schema, but a failure only logs -- an
  unreachable graph database must not stop the API booting.
- A sync failure during indexing is caught and logged; the SQL edges are unaffected.
- Repo deletion removes the subgraph before the SQL rows, so a failure leaves the
  authoritative data intact and retryable rather than orphaning a subgraph.

- core/graph/neo4j_store.py: schema (uniqueness on (repo_id, path), which is also the
  index that keeps MERGE off a label scan), batched UNWIND+MERGE ingest at 500 rows,
  and reads. sync_repository deletes the subgraph first rather than merging, because a
  MERGE-only sync leaves edges for deleted files and drifts into a union of every commit
  ever indexed.
- Traversals: reachable_from, blast_radius, shortest_path, import_cycles. Depth is
  clamped (1..10) -- an unbounded variable-length pattern is a trivial way to hang the
  server.
- Degree and centrality use COUNT {} subqueries, NOT Graph Data Science: the in-database
  GDS plugin requires AuraDB Professional or above, and Aura Graph Analytics sessions
  are an offline batch shape (2GB, one concurrent session, 30-minute TTL) that does not
  fit a synchronous request.
- Async driver held as a single long-lived instance (it owns the pool) and closed in the
  lifespan.
- docker-compose.yml: neo4j:5-community as a fourth service at 512m heap + 512m
  pagecache, plus the five settings forwarded to the API.

Docs record the AuraDB Free trap: a Free instance auto-pauses after 72h idle and a
paused instance's hostname stops resolving, so the graph would silently fall back to SQL
until someone resumed it by hand.

Also fixes stale drift in .env.example that still advertised
LOCAL_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5 -- the HuggingFace id removed from
config.py earlier, which would have walked a self-hoster straight back into the
all-zero-vector index.

VERIFIED, AND WHAT IS NOT
15 new tests, 132 total (was 117), ruff clean, app boots with the store returning None
by default. The tests use a fake driver, so they cover the queries we send and every
fallback path -- including that indexing still persists SQL edges when the graph sync
fails -- but NOT that the Cypher returns correct results. No query in this commit has
executed against a Neo4j server. docker/README.md documents the two count queries that
confirm the projection matches SQL once an instance is available.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
codebaseqa-web Ready Ready Preview Aug 10, 2026 1:22am

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant