Optional Terraform module: autoscaling RL worker fleet + S3 job queue - #5
Open
mmjerge wants to merge 2 commits into
Open
Optional Terraform module: autoscaling RL worker fleet + S3 job queue#5mmjerge wants to merge 2 commits into
mmjerge wants to merge 2 commits into
Conversation
Deploys N single-GPU workers (DL base AMI + provision_worker.sh at boot), the kv-worker IAM role (canonical-splits read, results read/write, tag-scoped self-stop), the results bucket with optional collaborator read grants, and SSH key/security group. Fleet-of-singles rationale and job queue usage documented in terraform/README.md. Fully optional: training code has no dependency on this directory; distinct resource names so it coexists with the hand-built infra.
…pacity hunting) Replaces fixed aws_instance workers with an autoscaling group spanning every default-VPC subnet and a prioritized list of L40S types (g6e.2/4/8xlarge). EC2 searches the (type x AZ) grid and backfills toward desired_capacity automatically -- no manual InsufficientInstanceCapacity retries. Workers are stateless queue consumers (worker_loop at boot, self-stop on drain); optional spot mode (capacity-optimized) for when jobs can checkpoint/resume.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optional Terraform module for the RL worker fleet
Infrastructure-as-code for the experiment fleet that ran (and is running) the RL campaigns, so anyone on the project can deploy an identical setup into their own AWS account. Fully optional: the training code has no dependency on this directory.
What it deploys
worker_countautomatically -- this replaced manual instance launching after a day ofInsufficientInstanceCapacitywhack-a-mole across p4d/g6e sizes and four AZs. Fleet-of-singles rationale: the GRPO/RLOO loop is single-GPU, so the experiment matrix (seeds x cache configs x tasks) parallelizes across independent workers, and single-GPU types actually have capacity.scripts/provision_worker.sh(repo checkout, pinned Python stack frozen from the reference box, canonical HELMET splits from S3, model download, smoke test), then startsscripts/worker_loop.sh-- claim a job from the S3 queue, run it, sync logs/artifacts to S3, park failures inqueue/failed/, self-stop when the queue drains.Battle-tested
This config is extracted from the live fleet currently running the 7B @32k flagship campaign (3 workers, S3 queue). The sharp edges it encodes were all hit in practice: capacity scarcity, the 30GB-optimizer-state OOM (paged 8-bit optimizer now in the driver), silent queue-draining on job failure (now parked visibly), and zombie GPU processes after kills.
Usage
See
terraform/README.md--terraform apply -var worker_count=N -var ssh_public_key_path=..., then drop job scripts intos3://<results-bucket>/queue/pending/.