What problem are you facing?
A Leader/Worker engine has one Worker member template, but worker.nodes can fan that template out to multiple follower pods. The current vLLM multiprocessing example works only for worker.nodes: 1: it hard-codes --node-rank=0 on the Leader and --node-rank=1 on the Worker. With more than one follower, every worker would start with rank 1.
MODELPLANE_LEADER_ADDRESS already hides LWS_LEADER_ADDRESS, but there is no equivalent rank contract. Using LWS_WORKER_INDEX directly would couple the member command to LeaderWorkerSet. A Grove-backed implementation would need different inputs: a leader address derived from GROVE_PCSG_NAME, GROVE_PCSG_INDEX, and GROVE_HEADLESS_SERVICE, plus a worker rank derived from GROVE_PCLQ_POD_INDEX + 1. The same ModelDeployment should not need different engine commands for LWS and Grove.
How could Modelplane help solve your problem?
Inject two backend-neutral values into every container of a multi-node engine:
MODELPLANE_LEADER_ADDRESS: the address of the gang's Leader, preserving the existing contract.MODELPLANE_RANK: 0 for the Leader and a unique 1..worker.nodes value for follower pods.
The user would continue to own all engine flags:
command:
- /bin/sh
- -c
- >-
exec vllm serve /mnt/models
--distributed-executor-backend=mp
--nnodes=4
--node-rank=$(MODELPLANE_RANK)
--master-addr=$(MODELPLANE_LEADER_ADDRESS)
Modelplane would only supply the values. It would not infer or inject --nnodes, --node-rank, --master-addr, or other engine-specific flags.
Each deployment backend can derive the values from its native coordination metadata. For LWS, the Leader address can continue to alias LWS_LEADER_ADDRESS, workers can alias LWS_WORKER_INDEX, and the Leader gets rank 0. For Grove, the address and rank can be derived from the GROVE_* metadata above. When the backend's variables are inserted before the member's environment, these can be dependent PodSpec env vars. When ordering or arithmetic prevents that, the backend could establish them in the command prelude instead. A command prelude must preserve the existing command/args pass-through behavior and account for the difference between kubelet $(VAR) expansion and shell ${VAR} expansion.
Success means that a gang with worker.nodes: 3 receives ranks 0, 1, 2, and 3; every pod receives the same Leader address; and the member templates contain no LWS_* or GROVE_* references. Existing uses of MODELPLANE_LEADER_ADDRESS remain compatible.
Related: #141 covers user-defined environment values sourced from pod fields; this issue is the platform-owned coordination contract across deployment backends.
What problem are you facing?
A Leader/Worker engine has one Worker member template, but
worker.nodescan fan that template out to multiple follower pods. The current vLLM multiprocessing example works only forworker.nodes: 1: it hard-codes--node-rank=0on the Leader and--node-rank=1on the Worker. With more than one follower, every worker would start with rank 1.MODELPLANE_LEADER_ADDRESSalready hidesLWS_LEADER_ADDRESS, but there is no equivalent rank contract. UsingLWS_WORKER_INDEXdirectly would couple the member command to LeaderWorkerSet. A Grove-backed implementation would need different inputs: a leader address derived fromGROVE_PCSG_NAME,GROVE_PCSG_INDEX, andGROVE_HEADLESS_SERVICE, plus a worker rank derived fromGROVE_PCLQ_POD_INDEX + 1. The same ModelDeployment should not need different engine commands for LWS and Grove.How could Modelplane help solve your problem?
Inject two backend-neutral values into every container of a multi-node engine:
MODELPLANE_LEADER_ADDRESS: the address of the gang's Leader, preserving the existing contract.MODELPLANE_RANK:0for the Leader and a unique1..worker.nodesvalue for follower pods.The user would continue to own all engine flags:
Modelplane would only supply the values. It would not infer or inject
--nnodes,--node-rank,--master-addr, or other engine-specific flags.Each deployment backend can derive the values from its native coordination metadata. For LWS, the Leader address can continue to alias
LWS_LEADER_ADDRESS, workers can aliasLWS_WORKER_INDEX, and the Leader gets rank 0. For Grove, the address and rank can be derived from theGROVE_*metadata above. When the backend's variables are inserted before the member's environment, these can be dependent PodSpec env vars. When ordering or arithmetic prevents that, the backend could establish them in the command prelude instead. A command prelude must preserve the existing command/args pass-through behavior and account for the difference between kubelet$(VAR)expansion and shell${VAR}expansion.Success means that a gang with
worker.nodes: 3receives ranks 0, 1, 2, and 3; every pod receives the same Leader address; and the member templates contain noLWS_*orGROVE_*references. Existing uses ofMODELPLANE_LEADER_ADDRESSremain compatible.Related: #141 covers user-defined environment values sourced from pod fields; this issue is the platform-owned coordination contract across deployment backends.