feat(dev): run agentex locally without Docker #353
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Stand up the full backend as host processes with embedded datastores — no Docker daemon required — as a lighter alternative to the container stack. `./dev.sh local` (also `make dev-local` / `python -m scripts.dev_local`) provisions: - Postgres via bundled pgserver (unix socket) and Redis via bundled redislite - a Temporal dev server + UI and the agentex worker (--no-temporal to skip) - a local mongod, required for the full stack (--no-mongo / --lean to skip) - an optional OpenTelemetry collector (--no-otel to skip) then runs migrations, supervises uvicorn + the worker, and tears everything down cleanly on SIGINT/SIGTERM. --lean is a minimal Postgres+Redis+API stack; --ephemeral uses a throwaway data dir. The runner is a small scripts/dev_local package (config / services / supervise / runner) so the pure config/env layer stays testable. App-side changes to make the no-Docker path robust (all no-ops when Mongo is configured, as it always is in Docker/prod): - The Temporal worker no longer crashes when MongoDB is unavailable — the Mongo CRUD adapter tolerates an unset database and errors only on real use, so the worker degrades like the API instead of taking down the stack. - Skip the Mongo connection entirely when MONGODB_URI is unset, removing a ~20s startup hang against the implicit localhost:27017 default. - In local mode the backend rewrites agents' host.docker.internal ACP host to loopback (AGENTEX_ACP_HOST_OVERRIDE), so default-scaffolded agents work without manifest edits. Also fix a frontend dev-server process leak in dev.sh (kill the whole make→npm→next tree and sweep orphans; report status by listening port), correct the MongoDB and OpenTelemetry install commands, and document local mode in README and CLAUDE.md.
356b3eb to
aa8bb61
Compare
| raw = acp_url_override | ||
|
|
||
| # Prefer the production deployment's URL when there's no explicit override. | ||
| if raw is None and agent.production_deployment_id: |
There was a problem hiding this comment.
@smoreinis can you take a look here just make sure it doesn't conflict with the preview workflow?
| """ | ||
| # In docker-free local mode, rewrite host.docker.internal -> the host-reachable | ||
| # override so the healthcheck matches how the request path dials the agent. | ||
| acp_url = resolve_acp_url(acp_url) |
There was a problem hiding this comment.
just making sure, no other places we need to do conversion right?
| if acp_url_override: | ||
| return acp_url_override | ||
| """Resolve the ACP URL for an agent, optionally overriding with a specific URL. | ||
|
|
There was a problem hiding this comment.
did we lose the override?
There was a problem hiding this comment.
hi, so ./dev.sh no-docker will pass _ACP_HOST_OVERRIDE_ENV variable, so the function resolve_acp_url will resolve this. Besides this, the logic of this function is kept the same
| Returns the URL unchanged when the override env var is unset or the URL does | ||
| not use the Docker sentinel host, so it is safe to call on every ACP dial. | ||
| """ | ||
| override = os.environ.get(_ACP_HOST_OVERRIDE_ENV) |
There was a problem hiding this comment.
is this where the override moved?
There was a problem hiding this comment.
yes, here we override the environment variable _ACP_HOST_OVERRIDE_ENV. This environment is set and passed from config.py file
| # Development Server | ||
| # | ||
|
|
||
| dev: install-dev ## Start development server with Docker Compose |
There was a problem hiding this comment.
these are both technically local maybe rename to something more meaningful?
|
|
||
| > **MongoDB is required for the full local stack** and is always started — the Temporal | ||
| > worker builds Mongo-backed repositories at startup, so a missing/unreachable Mongo | ||
| > makes the runner fail fast (with an install message) rather than crash the worker. |
danielmillerp
left a comment
danielmillerp
left a comment
There was a problem hiding this comment.
lots of small questions! also as far as testing on PC, not super sure. A lot of our clients do use PCs and I know there are Scaliens who have PCs. What problems do you anticipate?
There was a problem hiding this comment.
Overall this is rad.
One UX suggestion: in addition dev.sh, I think this would be really nice to have in the agentex CLI. I.E. you can run a single agent inline. therefore to start running a single agent you can start quick rather than having both the agentex backend and cli running. then this setup would be suitable for someone testing a multiagent flow.
Not sure if that is possible given the abstraction and integration, but wanted to throw it out there as a north star.
+1 to that! |
@danielmillerp So Im testing this on Windows (via AWS Workspaces) but the embedded Redis uses (the redislite package has no native Windows build (Redis ships no supported Windows server) so the dependency required to run locally without Docker won't even install on native Windows. |
| await asyncio.wait([stop_task, *waiters], return_when=asyncio.FIRST_COMPLETED) | ||
| if not stop.is_set(): | ||
| dead = [name for name, p in procs if p.returncode is not None] | ||
| logger.error( | ||
| "A managed process exited unexpectedly: %s. Shutting down.", | ||
| ", ".join(dead) or "?", | ||
| ) | ||
| return 0 |
There was a problem hiding this comment.
When the API or worker dies before a signal is received, run() logs the crash but falls through to return 0, so main() calls raise SystemExit(0). Any caller that checks $? — including make dev-no-docker and direct python -m scripts.dev_nodocker invocations — sees success even when the dev stack crashed. Returning a non-zero code on the crash path lets CI and scripts reliably detect the failure.
Prompt To Fix With AI
This is a comment left during a code review.
Path: agentex/scripts/dev_nodocker/runner.py
Line: 119-126
Comment:
**Unexpected process crash returns exit code 0**
When the API or worker dies before a signal is received, `run()` logs the crash but falls through to `return 0`, so `main()` calls `raise SystemExit(0)`. Any caller that checks `$?` — including `make dev-no-docker` and direct `python -m scripts.dev_nodocker` invocations — sees success even when the dev stack crashed. Returning a non-zero code on the crash path lets CI and scripts reliably detect the failure.
How can I resolve this? If you propose a fix, please make it concise.
Some findings from testing local dev on Windows:
@NiteshDhanpal had a clear view on how to scope this. His take: don't treat Windows-native local mode as a hard requirement for now (given the redislite/Redis limitation), and use a support matrix instead:
His main concern is a reliable fallback so customers aren't blocked when local setup is painful or impossible, and he thinks a cloud/dev-environment fallback is the most reliable option for that. |
3 participants
What
Adds a docker-free mode that runs the full agentex backend as host processes with embedded datastores — a lighter alternative to the Docker Compose stack, closer to a one-command
langgraph dev-style workflow.The bare
./dev.sh(Docker) is unchanged and now also accepts an explicit./dev.sh dockeralias. The docker-free mode is also available asmake dev-no-dockerandpython -m scripts.dev_nodocker.Why
Standing up a local environment previously required the full Docker stack. This lets a developer run the backend with a single command and no Docker daemon:
pgserver(unix socket) and Redis via bundledredislitemongod— always started; the stack requires it (the Temporal worker builds Mongo-backed repositories at boot)It runs migrations, supervises uvicorn + the worker, and tears everything down cleanly on Ctrl-C / SIGTERM.
--ephemeraluses a throwaway data dir;--mongo-uripoints at an existing MongoDB instead of launching a localmongod.App-side change
Safe no-op wherever the backend runs in Docker / staging / prod (the env var it keys on is unset there):
ACP host rewrite for docker-free mode. Agents register their ACP URL at
host.docker.internal(the SDK default, so a Docker backend can reach an agent on the host), which a host-process backend can't resolve. The runner setsAGENTEX_ACP_HOST_OVERRIDE=127.0.0.1and the backend rewrites only that sentinel host to the override when dialing agents — in the ACP request path, the agent-API-key proxy path, and the Temporal healthcheck. When the env var is unset, the stored URL is used verbatim. Default-scaffolded agents work without manifest edits.Also
dev.sh(kill the wholemake → npm → nexttree and sweep orphans; report status by listening port).brew trustfirst) and OpenTelemetry (release binary; not in Homebrew) install commands.dev-no-dockeruv dependency group (pgserver,redislite,greenlet) pulled only for docker-free mode.README.mdandCLAUDE.md.Contract note
The runner is a package (
scripts/dev_nodocker/), so the direct invocation ispython -m scripts.dev_nodocker(notpython scripts/dev_nodocker.py)../dev.sh no-dockerandmake dev-no-dockerare unchanged for callers.Testing
Manually exercised on macOS (Docker stopped):
--lean— the API passes/healthzwithin ~1s and/readyzreports Postgres, Redis, and MongoDB all healthy; the Temporal worker stays up; teardown frees all ports with no orphaned processes.mongodabsent aborts with an actionable install message (or point at an external instance with--mongo-uri).python -m scripts.dev_nodocker; agents scaffolded byagentex initconnect without manifest edits (ACP host rewrite).Platform support: macOS and Linux only. Not supported on native Windows — the embedded Redis (
redislite) ships no Windows server build; run it under WSL2, whereredislite/pgserver/mongodbehave as on Linux (WSL2 path not yet verified).Greptile Summary
This PR adds a docker-free local development mode (
./dev.sh no-docker/make dev-no-docker/python -m scripts.dev_nodocker) that runs the full agentex backend — Postgres (viapgserver), Redis (viaredislite), Temporal, MongoDB, and an optional OTel collector — as host processes with no Docker daemon required. A companion app-side change rewrites agents' registeredhost.docker.internalACP URLs to127.0.0.1via a newAGENTEX_ACP_HOST_OVERRIDEenv var, applied in the ACP request path, the API-key proxy path, and the Temporal health-check activity.scripts/dev_nodocker/package: pure-configconfig.py, side-effectfulservices.py(provision/teardown each datastore),supervise.py(spawn/stream/health-check), andrunner.py(orchestration + signal handling); all hooked intodev.shandmake.dev.shimprovements: addsno-dockeranddockersubcommands, fixes frontend process-tree leak on stop/restart (kill_tree+sweep_stray_frontends), and records the active mode in.dev-logs/modesostop/status/restartknow which path to use.src/utils/acp_url.pyaddsresolve_acp_url()wired intoagents_acp_use_case._resolve_acp_url,agent_api_keys_use_caseproxy path, andhealthcheck_activities.check_status_activity; it is a no-op when the env var is unset, so Docker / staging / prod are unaffected.Confidence Score: 4/5
Safe to merge — production code changes are confined to the ACP URL rewrite (a no-op when the env var is unset) and the deployment-URL fallback logic; the docker-free runner is dev-only.
The production-path changes are minimal and guarded by an env var that is unset in Docker/staging/prod. The dev runner is well-structured and fail-fast for required services. Two issues from earlier review rounds (runner crash returning exit code 0, empty-string deployment URL bypass) remain open; neither affects Docker or production paths but would affect developers using the new no-docker mode.
agentex/scripts/dev_nodocker/runner.py — the non-zero exit code on crash path should be verified before this becomes part of CI scripts.
Important Files Changed
is None) for the deployment URL fallback — empty-string handling issue previously flagged; resolve_acp_url wired in correctly.Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant devsh as dev.sh no-docker participant runner as runner.py participant services as services.py participant supervise as supervise.py participant api as API (uvicorn) participant agent as Agent process devsh->>runner: asyncio.run(run(cfg)) runner->>services: provision_postgres(cfg) services-->>runner: pg_server, database_url runner->>services: provision_redis(cfg) services-->>runner: redis_server, redis_url runner->>services: provision_mongo(cfg) services->>supervise: wait_for_port(proc, host, port) services-->>runner: mongo_proc, mongo_uri runner->>services: provision_temporal(cfg) services-->>runner: temporal_env, temporal_address runner->>supervise: run_migrations(cfg, env) supervise-->>runner: done runner->>supervise: spawn api + worker runner->>supervise: wait_for_health(api_port) supervise-->>runner: True / False alt Signal or process crash runner->>supervise: terminate worker + api runner->>services: temporal.shutdown + teardown_redis + pg.cleanup end Note over api,agent: ACP URL rewrite (all modes) api->>api: resolve_acp_url(raw_url) Note right of api: Rewrites host.docker.internal to 127.0.0.1 only when AGENTEX_ACP_HOST_OVERRIDE is set. No-op in Docker and prod. api->>agent: HTTP dial at resolved URL%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant devsh as dev.sh no-docker participant runner as runner.py participant services as services.py participant supervise as supervise.py participant api as API (uvicorn) participant agent as Agent process devsh->>runner: asyncio.run(run(cfg)) runner->>services: provision_postgres(cfg) services-->>runner: pg_server, database_url runner->>services: provision_redis(cfg) services-->>runner: redis_server, redis_url runner->>services: provision_mongo(cfg) services->>supervise: wait_for_port(proc, host, port) services-->>runner: mongo_proc, mongo_uri runner->>services: provision_temporal(cfg) services-->>runner: temporal_env, temporal_address runner->>supervise: run_migrations(cfg, env) supervise-->>runner: done runner->>supervise: spawn api + worker runner->>supervise: wait_for_health(api_port) supervise-->>runner: True / False alt Signal or process crash runner->>supervise: terminate worker + api runner->>services: temporal.shutdown + teardown_redis + pg.cleanup end Note over api,agent: ACP URL rewrite (all modes) api->>api: resolve_acp_url(raw_url) Note right of api: Rewrites host.docker.internal to 127.0.0.1 only when AGENTEX_ACP_HOST_OVERRIDE is set. No-op in Docker and prod. api->>agent: HTTP dial at resolved URLReviews (5): Last reviewed commit: "fix comments" | Re-trigger Greptile