docs(pytorch_xdit): add a first-time setup guide and correct stale instructions - #325
Open
speriaswamy-amd wants to merge 1 commit into
Open
Conversation
…structions
The README described a workflow that did not reproduce. Reworked around a
numbered first-run path — connect to the node, install the hf CLI, set HF_HOME,
`hf download` each model, `docker pull` the image, run sanity checks, write a
minimal single-node cluster.json, run the module — with the exact commands.
Corrections to existing content:
- WAN cache size was "~40GB"; a real `hf download` of Wan-AI/Wan2.2-I2V-A14B
is ~118GB.
- Config examples used hf_home: "/home/{user-id}", which is not a valid HF
cache root; they now match the shipped configs ({home}/.cache/huggingface).
- Output paths were documented as ${output_base_dir}/wan_22_${hostname}_outputs;
directories are now named from the cluster SSH target.
- Site-specific SLURM commands (a named reservation and account) were presented
as the way to get a node. Replaced with the actual requirement: a target with
Docker, the image pulled, and ROCm device nodes.
New sections cover the multi-node hf_home caveat (it is staged per node, so an
unshared $HOME means one full copy per node), the
CVS_PYTORCH_XDIT_SKIP_DOCKER_SYSTEM_PRUNE opt-out, which placeholders resolve
in test configs versus cluster files, and troubleshooting entries for the new
missing-image and incomplete-snapshot failures.
Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 12 of 12 in a stack that replaces #184. Base: #324.
Docs only. The README described a workflow that did not reproduce.
What changed
Reworked around a numbered first-run path — connect to the node, install the
hfCLI, setHF_HOME,hf downloadeach model,docker pullthe image, run sanity checks, write a minimal single-nodecluster.json, run the module — with the exact commands.Corrections:
hf downloadofWan-AI/Wan2.2-I2V-A14Bis ~118GB.hf_home: "/home/{user-id}", which is not a valid HF cache root; they now match the shipped configs.${output_base_dir}/wan_22_${hostname}_outputs; directories are now named from the cluster SSH target (fix(pytorch_xdit): key benchmark output dirs on the cluster target, not hostname #320).New sections: the multi-node
hf_homecaveat (staged per node, so an unshared$HOMEmeans one full copy per node — AIMVT-81/AIMVT-130), theCVS_PYTORCH_XDIT_SKIP_DOCKER_SYSTEM_PRUNEopt-out, which placeholders resolve in test configs vs cluster files, and troubleshooting entries for the new missing-image and incomplete-snapshot failures.