Skip to content

Use TorchX main from GitHub with dependency compatibility fixes#429

Open
hemildesai wants to merge 4 commits intomainfrom
codex/urllib3-constraints
Open

Use TorchX main from GitHub with dependency compatibility fixes#429
hemildesai wants to merge 4 commits intomainfrom
codex/urllib3-constraints

Conversation

@hemildesai
Copy link
Contributor

@hemildesai hemildesai commented Feb 23, 2026

Summary

  • switch torchx dependency to GitHub main (git+https://github.com/meta-pytorch/torchx.git@main)
  • remove cryptography<43 pin and keep cryptography as a normal dependency
  • keep uv overrides for cryptography and urllib3>=2.6.3
  • add direct importlib-metadata dependency (was previously only transitive)
  • add TorchX compatibility fixes for API changes on main:
    • nemo_run/run/logs.py: support torchx.util.colors with fallback to legacy module
    • nemo_run/run/torchx_backend/runner.py: support new _validate(..., cfg) signature with backward compatibility
  • refresh uv.lock (TorchX now resolves from commit 029982c43d3b94a013be322d78d5dd2246542fe0)

Validation

  • uv sync --extra skypilot succeeds and installs TorchX from GitHub main
  • uv run --group lint -- ruff check passes
  • uv run --group lint -- ruff format --diff passes
  • targeted regressions pass:
    • uv run pytest test/run/test_experiment.py::test_experiment_dryrun_with_cleanup test/run/torchx_backend/test_api.py::test_run_dryrun_with_executor test/run/test_logs.py
  • full CI-style pytest command run locally:
    • uv run -- coverage run --branch --source=nemo_run -a -m pytest
    • result: 1189 passed, 6 failed
    • remaining failures are environment-specific in this macOS workstation run:
      • 5 Docker scheduler tests fail without a running Docker daemon
      • 1 macOS tar metadata test (test_concatenate_tar_files_non_linux_integration) includes ._* files

Signed-off-by: Hemil Desai <hemild@nvidia.com>
Signed-off-by: Hemil Desai <hemild@nvidia.com>
Signed-off-by: Hemil Desai <hemild@nvidia.com>
@hemildesai hemildesai changed the title Add urllib3 >=2.6.3 dependency constraint and uv override Use TorchX main from GitHub with dependency compatibility fixes Feb 24, 2026
Signed-off-by: Hemil Desai <hemild@nvidia.com>
"fabric>=3.2.2",
"fiddle>=0.3.0",
"torchx>=0.7.0",
"torchx @ git+https://github.com/meta-pytorch/torchx.git@main",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks PyPi compatibility. We can use uv.sources to override with a vcs dependency in uv-environments only

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants