Skip to content

feat(scripts): run the examples this machine is set up for - #598

Draft
williedoran-neo4j wants to merge 1 commit into
neo4j:willie/examples-setup-installerfrom
williedoran-neo4j:willie/examples-live-checker
Draft

feat(scripts): run the examples this machine is set up for#598
williedoran-neo4j wants to merge 1 commit into
neo4j:willie/examples-setup-installerfrom
williedoran-neo4j:willie/examples-live-checker

Conversation

@williedoran-neo4j

@williedoran-neo4j williedoran-neo4j commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Last of the chain-B PRs. Stacked on #602 — the diff above contains #597, #599 and #602 until they merge; this PR's own change is the final commit.

Adds --live to the examples checker: an opt-in, manual-stage run of every example this machine can actually support. Nothing invokes it on commit, on push, or in CI.

The point of the restructure

What "set up for" means is now the doctor's judgement rather than a second opinion.

The version this replaces kept its own gate — a hardcoded RUNNABLE_PROVIDERS = {"openai"} plus a services check — which had drifted from blockers_for. Measured against the real corpus, it skipped 30 examples across 12 providers that were perfectly runnable, whatever credentials you had. Two implementations of the same question, disagreeing.

Delegating deletes RUNNABLE_PROVIDERS, _service_available (a one-line pass-through) and providers_used (already dead), and gains everything the doctor checks: extras, per-provider env vars, credential probes, APOC, indexes, sibling-module imports. A skip line now carries the fix, and apply_env_file means a key living only in .env reaches the subprocess instead of failing a preflight.

Cost

The whitelist was quietly doing one job worth keeping — cost control. Removing it means the doctor green-lights considerably more. So --live now says what it is about to spend, and waits:

--live will run 51 example(s) against live provider APIs. This COSTS MONEY.
Providers billed: anthropic, bedrock, cohere, gemini, mistral, openai, vertexai
Continue? [y/N]

That is real output from this branch. --yes skips the prompt for scripted use.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Documentation update
  • Project configuration change

Complexity

Complexity: Medium

How Has This Been Tested?

  • Unit tests
  • E2E tests
  • Manual tests

Nothing covered the --live path before, so this adds the gating tests: snippets are never run, a blocked example is skipped rather than run, declining the prompt runs nothing and exits 0, and the prompt names the providers that will be billed. Confirmed by hand that declining runs nothing.

Checklist

  • Documentation has been updated
  • Unit tests have been updated
  • E2E tests have been updated
  • Examples have been updated
  • New files have copyright header
  • CLA (https://neo4j.com/developer/cla/) has been signed
  • CHANGELOG.md updated if appropriate

🤖 Generated with Claude Code

@williedoran-neo4j
williedoran-neo4j force-pushed the willie/examples-live-checker branch 3 times, most recently from 1be8f62 to d3ec27c Compare August 12, 2026 10:30
@williedoran-neo4j williedoran-neo4j changed the title feat(scripts): add an opt-in live runner for the examples feat(scripts): run the examples this machine is set up for Aug 12, 2026
@williedoran-neo4j
williedoran-neo4j force-pushed the willie/examples-live-checker branch 4 times, most recently from d024a59 to 41e4f0d Compare August 13, 2026 09:06
Adds --live to the examples checker: an opt-in, manual-stage run of every
example the machine can actually support. Nothing invokes it on commit, on push
or in CI.

What "set up for" means is the doctor's judgement rather than a second opinion.
The version this replaces kept its own gate - a hardcoded {"openai"} whitelist
plus a services check - which had drifted from the doctor and skipped 30
examples across 12 providers that were perfectly runnable. Delegating to
blockers_for() also means a skip line now carries the fix, and that a key living
only in .env reaches the subprocess instead of failing a preflight.

The whitelist was doing one job worth keeping: cost control. So --live now says
what it is about to spend and on which providers, and waits for an answer.
--yes skips the prompt for scripted use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@williedoran-neo4j
williedoran-neo4j force-pushed the willie/examples-live-checker branch from 41e4f0d to 670ec8d Compare August 17, 2026 10:02
@williedoran-neo4j
williedoran-neo4j changed the base branch from main to willie/examples-setup-installer August 17, 2026 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant