feat(scripts): add an installer that walks through the missing pieces - #602
Draft
williedoran-neo4j wants to merge 6 commits into
Draft
feat(scripts): add an installer that walks through the missing pieces#602williedoran-neo4j wants to merge 6 commits into
williedoran-neo4j wants to merge 6 commits into
Conversation
This was referenced Aug 12, 2026
williedoran-neo4j
force-pushed
the
willie/examples-setup-installer
branch
2 times, most recently
from
August 13, 2026 08:21
f0549cf to
6a3b069
Compare
Nothing knows what an example needs in order to run, so a contributor finds out by running it and reading the traceback. This adds one source of truth the tooling can share. Requirements come from three places. Packages are resolved by an AST import scan that maps *library symbols* to extras - an example imports OpenAILLM, not openai, so the dependency is invisible from its imports alone. Env vars and datastores are read from os.getenv calls and connection URIs in the source. Anything that leaves no trace - APOC, a pre-existing index, outbound internet, or a component named only in a YAML config - is declared per path in SERVICE_RULES. Stdlib-only by design: it has to run before uv sync, which is exactly when somebody most needs to be told what to install. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All three workflows skipped `uv sync` whenever the venv cache key hit exactly. That trusts the cache to be complete, and it is not always: a cache saved from an interrupted install is restored on every later run with the same key, and the install that would repair it is precisely what gets skipped. Seen on neo4j#596, which changes uv.lock and so minted a fresh key: four e2e jobs failed with `ImportError: cannot import name 'WeaviateNeo4jRetriever'` and the same for Pinecone and Qdrant. The restored .venv was missing the vector-store extras, the test step installed only the 82 default packages, and re-running reproduced it exactly - the same cache was restored each time. Deleting the cache entry fixed it. The failure mode is nasty out of proportion to its cause: it presents as a flaky test in code that did not change, it survives re-runs, and it persists until the cache expires or somebody deletes it by hand. Any PR touching uv.lock is first to hit a new key, so it lands on whoever is least likely to suspect CI. uv sync takes a few seconds when the environment is already correct, which is cheap next to that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
williedoran-neo4j
force-pushed
the
willie/examples-setup-installer
branch
from
August 13, 2026 08:37
6a3b069 to
8c7d15d
Compare
Review feedback on neo4j#597. Two of the tests did not exercise what they claimed, and nothing guarded the tables against a partial edit. - test_an_aliased_symbol_still_maps_to_its_extra never fed the parsed symbols to anything: it built an ExampleRequirements by hand and asserted its extras were empty, which is true by default_factory whatever the code does. The symbol to extra join it names was never executed. - test_install_hints_reports_a_package_behind_no_extra asserted the corpus reports nothing, which is also what the bug it names produced. Both reached for hand-built objects because analyse() raised ValueError on any path outside examples/, so a synthetic file could not be analysed at all. It now returns instead, and both tests run the real pipeline over a temp file. SERVICE_RULES specs are now a TypedDict. They were dict[str, object] and read back through spec.get(), so a mistyped key ("service") or a scalar where a list belongs both silently did nothing - the example quietly lost its requirement. mypy already runs over scripts/ in CI and now rejects both. Adds the invariants that were missing entirely. SYMBOL_EXTRAS and SYMBOL_PROVIDERS are two halves of one fact: their values legitimately differ, since AzureOpenAILLM needs the openai extra but talks to the azure provider, but a symbol in one and not the other means the doctor installs a package it never asks for a key for, or the reverse. Nothing caught that. Nor a service name no label covers, which service_available() reports as satisfied because it cannot probe it. SERVICE_RULES matches are now pinned by count. Asserting only that a pattern matches something protects the nine rules matching exactly one file; the other ten match several, so renaming one of them leaves the pattern non-empty and the requirement silently lost. The count also catches a glob widened until it sweeps in a file that does not want the requirement. Also: read_text() now specifies utf-8, one example being non-ASCII; a duplicate sibling_modules assignment is removed; and install_hints' test is renamed to what it actually asserts, the import name rather than the distribution. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up to the previous commit. Making the install unconditional stops a bad cache entry causing failures, but does not stop one being created. actions/cache saves in a post step, which also runs when the job was cancelled or failed - so a run interrupted during uv sync publishes a half-installed .venv under a key derived from uv.lock alone. Splitting into restore/save makes the save an ordinary step, so an interrupted install never reaches it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds scripts/check_setup.py: resolves every example against the requirement model, probes this machine, and reports the single first thing in the way of each one, grouped so the biggest wins surface first and carrying the command that fixes it. Writes nothing and needs no credentials of its own. Four defects found while building it, fixed here: - A wrong NEO4J_PASSWORD produced a clean bill of health. service_available only opens a TCP socket, and the APOC and index checks both gate on `authenticated`, so all three silently passed while every Neo4j example failed at runtime. A reachable-but-unauthenticated database is now a blocker in its own right. - The missing-index fix advised `setup_examples.py --tier 0`, a flag that exists nowhere. It now points at the example that creates the index. - The Ollama check only ever suggested pulling the chat model, though the embedding examples need a different one. Both are reported. - The unset-key fix named a command this tool does not ship. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
check_setup.py reports what is missing; this walks through providing it. Works in tiers, so the free and local providers work before any cloud account is involved: Python extras and a local Neo4j first, then API keys, then the cloud logins. Every step is skippable and safe to re-run. Keys are read with no terminal echo, validated against the provider with a free list-models call, and written only to a gitignored .env created at mode 0600 - created at that mode rather than chmod-ed afterwards, so the secret never exists on disk world-readable, and written through a temp file and os.replace so an interrupted write cannot truncate the credentials already there. A key that does not validate is never written: a bad value turns a clear "unset" into a confusing 401 later. Nothing is printed in full. The doctor's unset-key fix now points at this tool, which is why the test pinning that string changes here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
williedoran-neo4j
force-pushed
the
willie/examples-setup-installer
branch
from
August 13, 2026 09:06
8c7d15d to
8689267
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third of the chain-B PRs. Stacked on #599 — the diff above contains #597 and #599 until they merge; this PR's own change is the final commit.
check_setup.py(#599) reports what is missing. This walks through providing it.Works in tiers, so the free and local providers work before any cloud account is involved: Python extras and a local Neo4j first, then API keys, then the cloud logins. Every step is skippable and safe to re-run.
Credentials
Keys are read with no terminal echo, validated against the provider with a free list-models call, and written only to a gitignored
.env.Two details worth calling out:
os.replace, so an interrupted write cannot truncate the credentials already there.A key that does not validate is never written: a bad value turns a clear "unset" into a confusing 401 later. Nothing is printed in full.
Note on the diff
The doctor's unset-key fix string changes here, from
export NAME=...to naming this installer — which is why the test pinning that string changes too. Every fix string the doctor prints has to name a command that actually exists at that point in the stack.Type of Change
Complexity
Complexity: Medium
How Has This Been Tested?
Confirmed the OpenAI validator rejects a bogus key and writes nothing, and that an unknown
--providerexits 2. Green on ruff, mypy --strict and the unit suite.Checklist
🤖 Generated with Claude Code