Skip to content

fix(envs): declare app paths the server can actually be imported under - #1068

Closed
k21993 wants to merge 5 commits into
huggingface:mainfrom
k21993:fix/awm-manifest-app-target
Closed

k21993 wants to merge 5 commits into
huggingface:mainfrom
k21993:fix/awm-manifest-app-target

Conversation

@k21993

@k21993 k21993 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix seven environment manifests whose app values do not match their container import layout. Cloud providers use this field to launch uvicorn, so an invalid module name can prevent startup even when the Dockerfile's own command works.

Six environments use their installed package name. grid_world_env uses envs.grid_world_env.server.app:app, matching its Dockerfile and PYTHONPATH layout.

Add static regression checks for module resolution and relative imports that escape the declared package. The checks run without optional environment dependencies and include negative cases for the incorrect envs. prefix. They do not replace runtime import or deployment tests.

Type of Change

  • Bug fix

Alignment Checklist

  • Read the project principles and invariants; no API or boundary changes.
  • Reviewed the rebased diff and ran the validation listed below.

RFC Status

  • Not required: corrects existing manifest values and adds regression tests.

Test Plan

  • tests/envs/test_manifest_app_targets.py: 76 passed.
  • Changed test file passes usort, Ruff formatting, and Ruff lint.
  • Environment documentation stubs are in sync.
  • Against current main, these regression tests fail for all seven corrected manifests.
  • Python 3.11 suite with CI's environment exclusions and marker filter: 1,849 passed, 93 skipped, 37 deselected. The remaining REPL subprocess import test passed on a targeted rerun after installing the local REPL package.
  • Repo-wide Ruff lint passes. Formatting checks reproduce existing drift on main: 57 files need Ruff formatting across src/, tests/, and envs/, and usort flags test_grid_world.py and test_julia_env.py. The changed test file is clean.

Note

Low Risk
Manifest-only startup path fixes plus CI guards; no runtime API or application logic changes beyond cloud env bootstrapping.

Overview
Corrects app targets in seven openenv.yaml manifests so Modal/Daytona can start uvicorn from the manifest (not the Dockerfile CMD). Six envs now use <env>.server.app:app instead of server.app:app; agent_world_model_env drops the invalid envs. prefix, and grid_world_env keeps envs.grid_world_env.server.app:app for its repo-root PYTHONPATH layout.

Adds tests/envs/test_manifest_app_targets.py, which statically checks every manifest with an app field: the module exists on disk, and AST analysis flags relative imports that would break under the declared package name—without importing heavy optional deps. Negative tests pin when envs. prefixes must be rejected vs allowed.

Reviewed by Cursor Bugbot for commit 8791ef7. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread envs/agent_world_model_env/openenv.yaml Outdated
@k21993 k21993 changed the title fix(agent_world_model_env): point manifest app at the real server module fix(envs): declare app paths the server can actually be imported under Aug 6, 2026
@k21993

k21993 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@cursor Good catch — you were right, and the fix has been rewritten.

Confirmed your diagnosis by reproduction:

$ cd envs/agent_world_model_env && python -c "import server.app"
  File ".../server/awm_environment.py", line 19, in <module>
    from ..models import AWMListToolsObservation, AWMObservation
ImportError: attempted relative import beyond top-level package

server.app:app was wrong for exactly the reason you gave: no ImportError fallback, so the relative import escapes. The manifest now uses agent_world_model_env.server.app:app, matching the Dockerfile CMD you pointed at.

Two follow-ons from your review:

The bug is not one env. Auditing all 35 manifests turned up seven with the same defect (carla_env, chess_env, finqa_env, grid_world_env, pelican_svg_env, wildfire_env alongside agent_world_model_env). All seven are fixed here. grid_world_env needed envs.grid_world_env... rather than the package form, because it builds from the repo root.

The original test could not have caught this. It only checked that the module file existed on disk, which server/app.py does. It now checks importability under the declared name by walking the reachable import graph with ast — still without importing anything, so optional deps like playwright and carla stay out of CI. Your point about the ImportError fallback is encoded directly: guarded imports are not counted, which is what keeps chat_env and reasoning_gym_env correctly passing on the flat form.

The static check was cross-validated against real import behaviour across all 35 envs: no false positives, no false negatives on every env whose deps are installed.

@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Unable to authenticate your request. Please make sure to connect your GitHub account to Cursor. Go to Cursor

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4cee358. Configure here.

Comment thread tests/envs/test_manifest_app_targets.py Outdated
@bot-ci-comment

bot-ci-comment Bot commented Aug 6, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@k21993

k21993 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@cursor Right again, and this one was worse than its Medium rating and the suite was passing the exact bug the PR exists to fix. Verified before fixing by putting the original value back:

app: envs.agent_world_model_env.server.app:app     # the original bug
71 passed

Fixed in 0b3e94c. The repo-root fallback is now conditional on evidence that the image actually imports under that prefix, rather than on the path existing in the source tree, which it always does.

One refinement worth noting: "built from a repo-root context" is not the right predicate. coding_env is built that way, but pip install ./envs/coding_env/ turns it into a real distribution, so its CMD is coding_env.server.app:app with no prefix. The only thing that evidences the prefix is the image launching it that way, as grid_world_env does (/app/envs/grid_world_env with /app on PYTHONPATH). The helper keys off that.

Same value with the bad manifest restored, after the fix:

FAILED test_manifest_app_target_exists[agent_world_model_env]
  module 'envs.agent_world_model_env.server.app' does not exist under
  envs/agent_world_model_env

Also added the negative tests whose absence let this through. Every check here only proved that the values in the tree pass — nothing asserted that a wrong value fails, which is why the regression was invisible to it:

  • test_rejects_envs_prefix_for_env_root_layout — AWM's old target must not resolve
  • test_accepts_envs_prefix_for_repo_root_layout — grid_world's legitimate target still must
  • test_repo_root_build_context_alone_does_not_allow_the_prefix — pins the two predicates apart

Suite: 1604 passed, 133 skipped.

@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Unable to authenticate your request. Please make sure to connect your GitHub account to Cursor. Go to Cursor

@k21993

k21993 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

CI ran green on an earlier commit here (4cee358), but the current head 0b3e94c has no workflow runs, so the checks look empty. Could someone re-approve the runs? Bugbot is passing on the current head.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 0b3e94ce9bab97b7ac27b350a6a73378ab06f50f: the seven manifest app targets match their actual container import roots, and the AST/Dockerfile-based regressions avoid optional-dependency skips; 76 focused tests pass. No code or alignment defect found. Before merge, refresh onto current main, approve the first-contributor workflow run, and require green exact-head CI; the required GitHub Actions jobs have not run on this branch.

Open in Web View Automation 

Sent by Cursor Automation: Release

@cursor
cursor Bot requested a review from burtenshaw September 10, 2026 06:22
The manifest declared `app: envs.agent_world_model_env.server.app:app`, the
only one of 35 envs not using `server.app:app`. The cloud providers read this
field verbatim and run `cd /app/env && python -m uvicorn <app>`, so this env
fails to start on Modal and Daytona with ModuleNotFoundError: images are built
with the env directory as context, and the installed dist maps
`agent_world_model_env` to `.`, so no `envs` package exists in either layout.

Add a contract test resolving every manifest's app target against the env
directory on disk. It does not import the modules: that would pull in
playwright, carla and the rest of the optional-dependency tail and skip on
exactly the CI machines that should be guarding this.
Seven manifests declared `app: server.app:app` while their server package
imports `from ..models`. Under that name `server` is top-level, so the relative
import climbs past it and the server dies with `ImportError: attempted relative
import beyond top-level package`. Only the cloud providers read this field --
`ModalProvider._discover_server_cmd` and its Daytona twin run
`cd <env root> && python -m uvicorn <app>` -- so the breakage is invisible
locally, where `docker run` uses the Dockerfile CMD instead.

Each manifest now names the same target its own Dockerfile launches: the env's
distribution package for six of them, and `envs.grid_world_env...` for
grid_world_env, which builds from the repo root.

The contract test now checks importability rather than mere file existence,
walking the reachable import graph with `ast`. It does not import the modules:
that would pull in playwright, carla and the rest of the optional-dependency
tail and skip on exactly the CI machines that should be guarding this. Imports
wrapped in a try/except ImportError are not counted, since that fallback is how
most envs already support both layouts.

Validated against real interpreter behaviour across all 35 manifests: no false
positives and no false negatives on every env whose dependencies are installed.
`_resolve` fell back to the repo root unconditionally, and the source tree
always contains `envs/<env>/server/app.py`. Every env therefore accepted an
`envs.` prefix -- including the original
`envs.agent_world_model_env.server.app:app`, the exact value this suite was
added to reject. The repo's layout is not evidence about the container's.

A repo-root build context is not the right signal either: coding_env builds
from the repo root but `pip install ./envs/coding_env/` makes it a real
distribution, so its CMD launches `coding_env.server.app:app` with no prefix.
The prefix is only importable when the image launches it that way, as
grid_world_env does with /app on PYTHONPATH. Derive it from that.

Add the negative tests whose absence let this through: the checks only proved
the values in the tree pass, never that a wrong one fails. One asserts AWM's
old target does not resolve, one that grid_world's legitimate target still
does, and one pins repo-root-context and envs-prefix as different predicates.
@k21993
k21993 force-pushed the fix/awm-manifest-app-target branch from 0b3e94c to add153a Compare September 13, 2026 18:45
@k21993

k21993 commented Sep 13, 2026

Copy link
Copy Markdown
Contributor Author

@burtenshaw I've rebased #1068 and #1070 onto current main and updated both descriptions with fresh validation results. The 76 manifest tests and four lint-hook tests pass, and the broader local checks are documented in each PR. Could you approve the GitHub Actions runs for both PRs so CI can validate the new commits?

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head add153ada3e13720d1962dc8b274476174fa991f: no code or alignment defects found. The seven manifest app: targets match their container import roots, and the dependency-free AST/import-graph guard now rejects the earlier false-positive envs. fallback. All 76 focused tests pass on this head; the same tests fail exactly seven cases on current main, confirming the regressions are meaningful. The merge-tree with current main is clean, including #1138's discovery additions.

This is a narrow manifest correctness fix, but it is not merge-ready yet: the branch is two commits behind current main, no required GitHub Actions workflow has run on this exact head, and no human approval is recorded. Refresh, approve/run Tests/Package/docs/discovery CI, and obtain the already-requested burtenshaw review before merge.

Open in Web View Automation 

Sent by Cursor Automation: Release

cursor Bot added a commit that referenced this pull request Sep 15, 2026
* fix(agent_world_model_env): point manifest app at the real server module

The manifest declared `app: envs.agent_world_model_env.server.app:app`, the
only one of 35 envs not using `server.app:app`. The cloud providers read this
field verbatim and run `cd /app/env && python -m uvicorn <app>`, so this env
fails to start on Modal and Daytona with ModuleNotFoundError: images are built
with the env directory as context, and the installed dist maps
`agent_world_model_env` to `.`, so no `envs` package exists in either layout.

Add a contract test resolving every manifest's app target against the env
directory on disk. It does not import the modules: that would pull in
playwright, carla and the rest of the optional-dependency tail and skip on
exactly the CI machines that should be guarding this.

* fix(envs): declare app paths the server can actually be imported under

Seven manifests declared `app: server.app:app` while their server package
imports `from ..models`. Under that name `server` is top-level, so the relative
import climbs past it and the server dies with `ImportError: attempted relative
import beyond top-level package`. Only the cloud providers read this field --
`ModalProvider._discover_server_cmd` and its Daytona twin run
`cd <env root> && python -m uvicorn <app>` -- so the breakage is invisible
locally, where `docker run` uses the Dockerfile CMD instead.

Each manifest now names the same target its own Dockerfile launches: the env's
distribution package for six of them, and `envs.grid_world_env...` for
grid_world_env, which builds from the repo root.

The contract test now checks importability rather than mere file existence,
walking the reachable import graph with `ast`. It does not import the modules:
that would pull in playwright, carla and the rest of the optional-dependency
tail and skip on exactly the CI machines that should be guarding this. Imports
wrapped in a try/except ImportError are not counted, since that fallback is how
most envs already support both layouts.

Validated against real interpreter behaviour across all 35 manifests: no false
positives and no false negatives on every env whose dependencies are installed.

* test(envs): only accept the envs. prefix where the image really uses it

`_resolve` fell back to the repo root unconditionally, and the source tree
always contains `envs/<env>/server/app.py`. Every env therefore accepted an
`envs.` prefix -- including the original
`envs.agent_world_model_env.server.app:app`, the exact value this suite was
added to reject. The repo's layout is not evidence about the container's.

A repo-root build context is not the right signal either: coding_env builds
from the repo root but `pip install ./envs/coding_env/` makes it a real
distribution, so its CMD launches `coding_env.server.app:app` with no prefix.
The prefix is only importable when the image launches it that way, as
grid_world_env does with /app on PYTHONPATH. Derive it from that.

Add the negative tests whose absence let this through: the checks only proved
the values in the tree pass, never that a wrong one fails. One asserts AWM's
old target does not resolve, one that grid_world's legitimate target still
does, and one pins repo-root-context and envs-prefix as different predicates.

---------

Co-authored-by: Karthik Suresh <7954591+k21993@users.noreply.github.com>
Co-authored-by: burtenshaw <ben.burtenshaw@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: benjamin.burtenshaw <benjamin.burtenshaw@huggingface.co>
@cursor cursor Bot closed this in #1164 Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants