-
Notifications
You must be signed in to change notification settings - Fork 13
ENH: Dockerize lung tutorial workflow #129
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
maximilianofir
wants to merge
14
commits into
Project-MONAI:main
Choose a base branch
from
maximilianofir:maxo/dockerized_lungs_workflow
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
b23a0f9
ENH: Dockerize lung tutorial workflow
maximilianofir 178f0c4
FIX: Preserve lung workflow on 282,782-vertex model
maximilianofir e81ac81
FIX: Include lung bundle in Docker build
maximilianofir 81018d3
FIX: Propagate Docker sudo fallback
maximilianofir 837f43e
DOC: Clarify mesh viewer playback input
maximilianofir 595c583
FIX: Discover animated USD transforms
maximilianofir f95ef6a
FIX: Refresh animated USD appearance
maximilianofir 3f4ad29
TST: Verify lung-only tutorial labelmap
maximilianofir f3e241c
DOC: Keep workshop token out of history
maximilianofir 862a6b6
FIX: Raise workshop disk requirement
maximilianofir 5bc2abc
FIX: Validate motion sequence connectivity
maximilianofir 7dcf188
REF: Simplify lung workshop deployment
maximilianofir b16a900
FIX: Keep launchable viewer and caches available
maximilianofir 3be442f
ENH: Serve course docs on Brev
maximilianofir File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| * | ||
| !Dockerfile | ||
| !LICENSE | ||
| !MANIFEST.in | ||
| !README.md | ||
| !pyproject.toml | ||
| !docker/ | ||
| !docker/** | ||
| !src/ | ||
| !src/** | ||
| !tutorials/ | ||
| !tutorials/** | ||
| !utils/ | ||
| !utils/create_motion_comparison_usd.py | ||
| !utils/lung_bundle.py | ||
| tutorials/network_weights/ | ||
| tutorials/output/ | ||
| **/__pycache__/ | ||
| **/*.py[cod] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,7 @@ venv* | |
| build | ||
| *.egg-info | ||
| *.log | ||
| .cache/physiotwin4d/ | ||
| __pycache__ | ||
| _version.py | ||
| htmlcov | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| ARG CUDA_VERSION=12.6.3 | ||
| ARG UBUNTU_VERSION=24.04 | ||
|
|
||
| FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION} AS builder | ||
|
|
||
| ENV DEBIAN_FRONTEND=noninteractive \ | ||
| PATH=/opt/venv/bin:$PATH \ | ||
| UV_COMPILE_BYTECODE=1 \ | ||
| UV_LINK_MODE=copy \ | ||
| UV_NO_PROGRESS=1 \ | ||
| UV_PYTHON_DOWNLOADS=never \ | ||
| VIRTUAL_ENV=/opt/venv | ||
|
|
||
| RUN apt-get update \ | ||
| && apt-get install --yes --no-install-recommends \ | ||
| build-essential \ | ||
| ca-certificates \ | ||
| git \ | ||
| ninja-build \ | ||
| python3 \ | ||
| python3-dev \ | ||
| python3-venv \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| COPY --from=ghcr.io/astral-sh/uv:0.11.11 /uv /usr/local/bin/uv | ||
|
|
||
| WORKDIR /build | ||
| COPY pyproject.toml README.md LICENSE MANIFEST.in ./ | ||
|
|
||
| RUN uv venv --python /usr/bin/python3 /opt/venv \ | ||
| && uv pip install --python /opt/venv/bin/python --no-cache setuptools \ | ||
| && uv pip install --python /opt/venv/bin/python --no-cache \ | ||
| --index https://download.pytorch.org/whl/cu126 \ | ||
| torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \ | ||
| && uv pip install --python /opt/venv/bin/python --no-cache \ | ||
| --no-build-isolation-package torch-scatter \ | ||
| --extra cuda12 --extra viewer \ | ||
| --requirement pyproject.toml | ||
|
|
||
| COPY src ./src | ||
| RUN uv pip install --python /opt/venv/bin/python --no-cache --no-deps . | ||
|
|
||
|
|
||
| FROM nvidia/cuda:${CUDA_VERSION}-base-ubuntu${UBUNTU_VERSION} | ||
|
|
||
| ENV DEBIAN_FRONTEND=noninteractive | ||
|
|
||
| RUN apt-get update \ | ||
| && apt-get install --yes --no-install-recommends \ | ||
| ca-certificates \ | ||
| g++ \ | ||
| libegl1 \ | ||
| libgl1 \ | ||
| libglib2.0-0 \ | ||
| libgomp1 \ | ||
| libsm6 \ | ||
| libxext6 \ | ||
| libxrender1 \ | ||
| python3 \ | ||
| python3-dev \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| COPY --from=builder /opt/venv /opt/venv | ||
|
|
||
| ENV HF_HOME=/cache/huggingface \ | ||
| MPLBACKEND=Agg \ | ||
| MPLCONFIGDIR=/cache/matplotlib \ | ||
| PATH=/opt/venv/bin:$PATH \ | ||
| PYTHONUNBUFFERED=1 \ | ||
| PYVISTA_OFF_SCREEN=true \ | ||
| TORCH_HOME=/cache/torch \ | ||
| TORCHINDUCTOR_CACHE_DIR=/cache/torchinductor \ | ||
| TOTALSEG_HOME_DIR=/cache/totalsegmentator \ | ||
| TRITON_CACHE_DIR=/cache/triton \ | ||
| VTK_DEFAULT_OPENGL_WINDOW=vtkEGLRenderWindow \ | ||
| XDG_CACHE_HOME=/cache/xdg | ||
|
|
||
| WORKDIR /workspace/monai-physio | ||
| COPY --chown=1000:1000 tutorials ./tutorials | ||
| COPY --chown=1000:1000 \ | ||
| docker/download-lung-bundles.sh \ | ||
| docker/serve-course-docs.sh \ | ||
| docker/tutorial-shell.sh \ | ||
| docker/view-meshes.sh \ | ||
| ./docker/ | ||
| COPY --chown=1000:1000 \ | ||
| utils/create_motion_comparison_usd.py \ | ||
| utils/lung_bundle.py \ | ||
| ./utils/ | ||
|
|
||
| RUN chmod 755 docker/*.sh \ | ||
| && chmod 644 utils/*.py \ | ||
| && chmod -R a+rX tutorials \ | ||
| && mkdir -p data network_weights tests/baselines tutorials/network_weights \ | ||
| tutorials/output /cache \ | ||
| && chmod 1777 tests/baselines \ | ||
| && chown -R 1000:1000 /workspace/monai-physio /cache | ||
|
|
||
| USER 1000:1000 | ||
| CMD ["bash"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,177 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| set -Eeuo pipefail | ||
|
|
||
| readonly NGC_NAMESPACE="nvcr.io/0569033758414229" | ||
| readonly NGC_IMAGE="${NGC_NAMESPACE}/physiomotion:v0.8-cu126" | ||
| DEFAULT_COURSE_DOCS_IMAGE="${NGC_NAMESPACE}/physiomotion-course-docs:latest" | ||
| readonly DEFAULT_COURSE_DOCS_IMAGE | ||
| COURSE_DOCS_IMAGE="${MONAI_PHYSIO_COURSE_DOCS_IMAGE:-${DEFAULT_COURSE_DOCS_IMAGE}}" | ||
| readonly COURSE_DOCS_IMAGE | ||
| readonly LOCAL_IMAGE="monai-physio:tutorials" | ||
| readonly LOCAL_COURSE_DOCS_IMAGE="monai-physio-course-docs:latest" | ||
| readonly INSTALL_DIR="${HOME}/monai-physio" | ||
|
|
||
| require_parameter() { | ||
| local name="$1" | ||
| if [[ -z "${!name:-}" ]]; then | ||
| echo "${name} must be supplied as a required Brev launch parameter" >&2 | ||
| exit 1 | ||
| fi | ||
| } | ||
|
|
||
| require_parameter NGC_API_KEY | ||
| require_parameter HF_TOKEN | ||
|
|
||
| if [[ "${NGC_API_KEY}" =~ [[:space:]] ]]; then | ||
| echo "NGC_API_KEY must not contain whitespace" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| command -v docker >/dev/null 2>&1 || { | ||
| echo "Docker is required" >&2 | ||
| exit 1 | ||
| } | ||
| command -v nvidia-smi >/dev/null 2>&1 || { | ||
| echo "nvidia-smi is required" >&2 | ||
| exit 1 | ||
| } | ||
| if [[ "$(uname -m)" != "x86_64" ]]; then | ||
| echo "The tutorial image requires an x86_64 Brev instance" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| use_sudo=false | ||
| if docker info >/dev/null 2>&1; then | ||
| : | ||
| elif command -v sudo >/dev/null 2>&1 && sudo -n true >/dev/null 2>&1; then | ||
| use_sudo=true | ||
| else | ||
| echo "The setup user cannot access Docker or passwordless sudo" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| docker_cmd() { | ||
| if [[ "${use_sudo}" == "true" ]]; then | ||
| sudo docker "$@" | ||
| else | ||
| docker "$@" | ||
| fi | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| registry_config_dir="$(mktemp -d)" | ||
| registry_docker_cmd() { | ||
| docker_cmd --config "${registry_config_dir}" "$@" | ||
| } | ||
|
|
||
| logged_in=false | ||
| cleanup() { | ||
| if [[ "${logged_in}" == "true" ]]; then | ||
| registry_docker_cmd logout nvcr.io >/dev/null 2>&1 || true | ||
| fi | ||
| unset NGC_API_KEY | ||
| rm -rf "${registry_config_dir}" | ||
| } | ||
| trap cleanup EXIT | ||
|
|
||
| echo "Authenticating to the private NGC registry..." | ||
| printf '%s' "${NGC_API_KEY}" | | ||
| registry_docker_cmd login nvcr.io --username '$oauthtoken' --password-stdin \ | ||
| >/dev/null | ||
| logged_in=true | ||
|
|
||
| echo "Pulling ${NGC_IMAGE}..." | ||
| registry_docker_cmd pull "${NGC_IMAGE}" | ||
| docker_cmd tag "${NGC_IMAGE}" "${LOCAL_IMAGE}" | ||
|
|
||
| echo "Pulling ${COURSE_DOCS_IMAGE}..." | ||
| registry_docker_cmd pull "${COURSE_DOCS_IMAGE}" | ||
| docker_cmd tag "${COURSE_DOCS_IMAGE}" "${LOCAL_COURSE_DOCS_IMAGE}" | ||
|
|
||
| echo "Checking GPU access and core Python dependencies..." | ||
| docker_cmd run --rm --gpus all --entrypoint python "${LOCAL_IMAGE}" -c ' | ||
| import torch | ||
| import monai_physio | ||
|
|
||
| if not torch.cuda.is_available(): | ||
| raise RuntimeError("CUDA is not available inside the tutorial container") | ||
| print(f"GPU ready: {torch.cuda.get_device_name(0)}") | ||
| ' | ||
|
|
||
| image_id="$(docker_cmd image inspect "${LOCAL_IMAGE}" --format '{{.Id}}')" | ||
| course_docs_image_id="$( | ||
| docker_cmd image inspect "${LOCAL_COURSE_DOCS_IMAGE}" --format '{{.Id}}' | ||
| )" | ||
| cleanup | ||
| logged_in=false | ||
| trap - EXIT | ||
|
|
||
| echo "Preparing the persistent workspace at ${INSTALL_DIR}..." | ||
| mkdir -p "${INSTALL_DIR}" | ||
| docker_cmd run --rm \ | ||
| --user "$(id -u):$(id -g)" \ | ||
| --volume "${INSTALL_DIR}:/deployment" \ | ||
| --entrypoint bash \ | ||
| "${LOCAL_IMAGE}" \ | ||
| -c 'cp -a /workspace/monai-physio/. /deployment/' | ||
|
|
||
| echo "Downloading workshop bundles and tutorial data..." | ||
| MONAI_PHYSIO_DOCKER_USE_SUDO="${use_sudo}" \ | ||
| MONAI_PHYSIO_IMAGE="${LOCAL_IMAGE}" \ | ||
| "${INSTALL_DIR}/docker/download-lung-bundles.sh" | ||
| unset HF_TOKEN | ||
|
|
||
| docs_root="${HOME}/monai-physio-course-docs" | ||
| if [[ ! -e "${docs_root}/current" ]]; then | ||
| release_name="image-${course_docs_image_id#sha256:}" | ||
| release_dir="${docs_root}/releases/${release_name}" | ||
| temporary_link="${docs_root}/.current-${release_name}-$$" | ||
|
|
||
| echo "Installing the course documentation from ${COURSE_DOCS_IMAGE}..." | ||
| mkdir -p "${release_dir}" | ||
| docs_container_id="$(docker_cmd create "${LOCAL_COURSE_DOCS_IMAGE}")" | ||
| if ! docker_cmd cp \ | ||
| "${docs_container_id}:/usr/share/nginx/html/." \ | ||
| "${release_dir}/"; then | ||
| docker_cmd rm "${docs_container_id}" >/dev/null | ||
| exit 1 | ||
| fi | ||
| docker_cmd rm "${docs_container_id}" >/dev/null | ||
|
|
||
| if [[ ! -f "${release_dir}/index.html" ]]; then | ||
| echo "The course documentation image does not contain index.html" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| ln -s "releases/${release_name}" "${temporary_link}" | ||
| mv -Tf "${temporary_link}" "${docs_root}/current" | ||
| fi | ||
|
|
||
| echo "Starting the course documentation server..." | ||
| MONAI_PHYSIO_DOCKER_USE_SUDO="${use_sudo}" \ | ||
| MONAI_PHYSIO_IMAGE="${LOCAL_IMAGE}" \ | ||
| "${INSTALL_DIR}/docker/serve-course-docs.sh" | ||
|
|
||
| cat <<EOF | ||
|
|
||
| MONAI Physio is ready. | ||
|
|
||
| Private image: ${NGC_IMAGE} | ||
| Local image: ${LOCAL_IMAGE} | ||
| Image ID: ${image_id} | ||
| Course image: ${COURSE_DOCS_IMAGE} | ||
| Course ID: ${course_docs_image_id} | ||
| Workspace: ${INSTALL_DIR} | ||
| Course docs: http://127.0.0.1:8000/ | ||
|
|
||
| The workshop bundles and public tutorial data have been downloaded and verified. | ||
| Run: | ||
|
|
||
| cd ${INSTALL_DIR} | ||
| ./docker/tutorial-shell.sh | ||
|
|
||
| The NGC and Hugging Face credentials were used only during setup and have been | ||
| removed from the setup environment and Docker client configuration. | ||
| EOF | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| #!/usr/bin/env bash | ||
| # Download the private lung workshop bundles through the tutorial image. | ||
|
|
||
| set -Eeuo pipefail | ||
|
|
||
| repo_root=$(cd "$(dirname "$0")/.." && pwd) | ||
| cache_dir="$repo_root/.cache/monai-physio" | ||
| container_root="/workspace/monai-physio" | ||
| image="${MONAI_PHYSIO_IMAGE:-monai-physio:tutorials}" | ||
| bundle_repo="${LUNG_BUNDLE_REPO:-maximilianofir/physioMotionWorkshop}" | ||
| bundle_revision="${LUNG_BUNDLE_REVISION:-a6127dd1d2e27c5b59ed3a81c5b4e7490b4bd1bf}" | ||
| docker_command=(docker) | ||
| if [[ "${MONAI_PHYSIO_DOCKER_USE_SUDO:-false}" == "true" ]]; then | ||
| docker_command=(sudo --preserve-env=HF_TOKEN docker) | ||
| fi | ||
|
|
||
| if [[ -z "${HF_TOKEN:-}" ]]; then | ||
| echo "HF_TOKEN is required to download the private workshop bundles" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| mkdir -p "$cache_dir/home" "$repo_root/tutorials/network_weights" \ | ||
| "$repo_root/tutorials/output" | ||
|
|
||
| "${docker_command[@]}" run --rm \ | ||
| --user "$(id -u):$(id -g)" \ | ||
| --volume "$repo_root:$container_root" \ | ||
| --volume "$cache_dir:/cache" \ | ||
| --workdir "$container_root" \ | ||
| --env HOME=/cache/home \ | ||
| --env HF_HOME=/cache/huggingface \ | ||
| --env HF_TOKEN \ | ||
| --env LOGNAME=monai-physio \ | ||
| --env PYTHONPATH="$container_root/src" \ | ||
| --env USER=monai-physio \ | ||
| "$image" \ | ||
| python utils/lung_bundle.py \ | ||
| --repository-root "$container_root" \ | ||
| --repo-id "$bundle_repo" \ | ||
| --revision "$bundle_revision" \ | ||
| --profile course \ | ||
| --profile offline-segmentation | ||
|
|
||
| echo "Downloading the public TCIA-4DLung input for Tutorial 1..." | ||
| "${docker_command[@]}" run --rm \ | ||
| --user "$(id -u):$(id -g)" \ | ||
| --volume "$repo_root:$container_root" \ | ||
| --volume "$cache_dir:/cache" \ | ||
| --workdir "$container_root" \ | ||
| --env HOME=/cache/home \ | ||
| --env XDG_CACHE_HOME=/cache/xdg \ | ||
| "$image" \ | ||
| monai-physio-download-data TCIA-4DLung --directory data/TCIA-4DLung | ||
|
|
||
| chest_ct_file="$repo_root/data/Chest-CT/Chest-CT.mha" | ||
| if [[ -s "$chest_ct_file" ]]; then | ||
| echo "Reusing the public Chest-CT input at $chest_ct_file" | ||
| else | ||
| echo "Downloading the public Chest-CT input for Tutorial 7..." | ||
| "${docker_command[@]}" run --rm \ | ||
| --user "$(id -u):$(id -g)" \ | ||
| --volume "$repo_root:$container_root" \ | ||
| --volume "$cache_dir:/cache" \ | ||
| --workdir "$container_root" \ | ||
| --env HOME=/cache/home \ | ||
| --env XDG_CACHE_HOME=/cache/xdg \ | ||
| "$image" \ | ||
| monai-physio-download-data Chest-CT --directory data/Chest-CT | ||
| fi |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.