Propagate image pull secrets to compute-domain-daemon pod specs - #2860
Propagate image pull secrets to compute-domain-daemon pod specs#2860karthikvetrivel wants to merge 1 commit into
Conversation
Signed-off-by: Karthik Vetrivel <kvetrivel@nvidia.com>
📝 WalkthroughWalkthroughThe controller Deployment now conditionally sets Merge Risk: 🟡 Moderate · up to Configured image pull secrets are now propagated for newly created compute-domain daemon pods, but existing DaemonSets may retain stale credentials or miss new ones after configuration changes, causing private-image pulls to fail. This should be resolved before merge. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: QUIET
Plan: Enterprise
Run ID: 40e86fd9-63a0-4cc3-b663-445ece3a8126
📒 Files selected for processing (1)
manifests/state-dra-driver/0600_controller-deployment.yaml
Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.
| {{- if .DRADriver.Spec.ImagePullSecrets }} | ||
| # Comma-separated secret names propagated to the compute-domain-daemon | ||
| # DaemonSet pod specs created by the controller. | ||
| - name: CD_DAEMON_IMAGE_PULL_SECRET_NAMES | ||
| value: "{{- range $i, $s := .DRADriver.Spec.ImagePullSecrets }}{{ if $i }},{{ end }}{{ $s }}{{ end }}" | ||
| {{- end }} |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🔎 Supported by static analysis
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 8 \
'CD_DAEMON_IMAGE_PULL_SECRET_NAMES|imagePullSecrets|ImagePullSecrets' \
--glob '*.go' \
--glob '*_test.go' \
--glob '*.yaml'Repository: NVIDIA/gpu-operator
Length of output: 157
🤖 get_repo_knowledge executed:
get_repo_knowledge NVIDIA/gpu-operator /tmp/coderabbit-repo-knowledge/nvidia-gpu-operator-eed98df4
Length of output: 873
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- target manifest ---'
sed -n '85,135p' manifests/state-dra-driver/0600_controller-deployment.yaml
printf '%s\n' '--- repository matches ---'
rg -n -C 6 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES|compute-domain-controller|compute-domain-daemon|ImagePullSecrets|imagePullSecrets' . --glob '!vendor/**' --glob '!node_modules/**' || true
printf '%s\n' '--- tracked related files ---'
git ls-files | rg -i 'dra|compute.domain|controller' | head -120Repository: NVIDIA/gpu-operator
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo pipefail
find . -type f -print | rg -i 'dra|compute|domain' | head -200
rg -n -C 10 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES|imagePullSecrets|ImagePullSecrets' . || trueRepository: NVIDIA/gpu-operator
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '85,135p' manifests/state-dra-driver/0600_controller-deployment.yaml
rg -n -C 8 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES|compute-domain-controller|compute-domain-daemon|ImagePullSecrets|imagePullSecrets' . || true
git ls-files | rg -i 'dra|compute.domain|controller' | head -120Repository: NVIDIA/gpu-operator
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- target ---'
cat -n manifests/state-dra-driver/0600_controller-deployment.yaml | sed -n '95,125p'
printf '%s\n' '--- exact variable references ---'
rg -n -F 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES' . || true
printf '%s\n' '--- DRA controller image and related configuration ---'
rg -n -C 5 'compute-domain-controller|compute-domain-daemon|dra-driver|ImagePullSecrets' \
manifests config deployments api go.mod Makefile README.md docs 2>/dev/null | head -300 || true
printf '%s\n' '--- repository conventions ---'
cat /tmp/coderabbit-repo-knowledge/nvidia-gpu-operator-eed98df4/conventions/repo-wide.mdRepository: NVIDIA/gpu-operator
Length of output: 28216
🏁 Script executed:
#!/bin/bash
set -euo pipefail
cat -n manifests/state-dra-driver/0600_controller-deployment.yaml | sed -n '95,125p'
rg -n -F 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES' . || true
rg -n -C 5 'compute-domain-controller|compute-domain-daemon|dra-driver|ImagePullSecrets' manifests config deployments api go.mod Makefile README.md docs 2>/dev/null | head -300 || true
cat /tmp/coderabbit-repo-knowledge/nvidia-gpu-operator-eed98df4/conventions/repo-wide.mdRepository: NVIDIA/gpu-operator
Length of output: 27397
🌐 Web query:
"CD_DAEMON_IMAGE_PULL_SECRET_NAMES" k8s-dra-driver-gpu
💡 Result:
The parameter CD_DAEMON_IMAGE_PULL_SECRET_NAMES is a configuration setting used in the Kubernetes SIGs NVIDIA DRA (Dynamic Resource Allocation) driver to specify Kubernetes secrets required for pulling container images for the ComputeDomain (CD) daemon [1]. This parameter allows administrators to provide a list of image pull secrets that the CD daemon should use when pulling images from private registries [1]. It was introduced to ensure that the driver can authenticate with registries requiring secure access when deploying the necessary daemon components [1]. To use this configuration, you typically pass it during the installation or upgrade of the dra-driver-nvidia-gpu Helm chart [2][3]. You can verify the availability of this and other parameters for your specific version of the chart by running: helm show values nvidia/dra-driver-nvidia-gpu For further details on deploying the driver and configuring image pull secrets, refer to the official documentation and the values.yaml file provided in the Helm chart repository [2][3].
Citations:
- 1: kubernetes-sigs/dra-driver-nvidia-gpu@v25.12.0...v0.4.0
- 2: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-intro-install.html
- 3: https://github.com/NVIDIA/k8s-dra-driver-gpu/wiki/Installation
🌐 Web query:
site:github.com/NVIDIA/k8s-dra-driver-gpu CD_DAEMON_IMAGE_PULL_SECRET_NAMES
💡 Result:
The term CD_DAEMON_IMAGE_PULL_SECRET_NAMES does not appear as a standard, documented configuration parameter in the NVIDIA DRA Driver for GPUs project [1]. Technical analysis of the project's source code and configuration templates indicates that image pull secret management is typically handled through similarly named flags or environment variables, such as IMAGE_PULL_SECRETS, which is used to specify comma-separated names for pod templates [2]. It is common in this project for configuration to be managed via Helm chart values or specific environment variables defined in individual component binaries (such as the GPU kubelet plugin or compute-domain-daemon) [2][3][4][5]. If you are encountering CD_DAEMON_IMAGE_PULL_SECRET_NAMES, it may be a custom or deprecated environment variable name used in a specific deployment environment rather than an official part of the driver's public API [1]. To configure image pull secrets for components in the NVIDIA DRA driver, it is recommended to use the supported Helm chart values provided by the project's official installation documentation [6].
Citations:
- 1: https://github.com/NVIDIA/k8s-dra-driver-gpu
- 2: https://github.com/NVIDIA/k8s-dra-driver-gpu/blob/main/cmd/gpu-kubelet-plugin/main.go
- 3: GitHub pull request 259 in NVIDIA/k8s-dra-driver-gpu (link omitted to avoid creating a cross-reference)
- 4: kubernetes-sigs/dra-driver-nvidia-gpu@v25.3.0-rc.2...v25.3.0-rc.3
- 5: GitHub pull request 379 in NVIDIA/k8s-dra-driver-gpu (link omitted to avoid creating a cross-reference)
- 6: https://github.com/NVIDIA/k8s-dra-driver-gpu/wiki/Installation
🌐 Web query:
"CD_DAEMON_IMAGE_PULL_SECRET_NAMES" "k8s-dra-driver-gpu"
💡 Result:
The environment variable CD_DAEMON_IMAGE_PULL_SECRET_NAMES is used in the NVIDIA Dynamic Resource Allocation (DRA) driver for GPUs to specify the names of Kubernetes image pull secrets required to authenticate with private container registries when pulling the ComputeDomain (CD) daemon image [1]. This parameter ensures that the CD daemon, which is critical for managing GPU resources and communication in multi-node environments, can be successfully pulled if the image resides in a restricted or private registry [1]. It was introduced to allow administrators to pass these secret names to the daemon configuration, as reflected in the NVIDIA DRA driver development history [1]. In practice, if you are deploying the k8s-dra-driver-gpu, you should: 1. Create a Kubernetes Secret containing your registry credentials in the relevant namespace. 2. Provide the name of that secret via the CD_DAEMON_IMAGE_PULL_SECRET_NAMES variable in your configuration (e.g., in your Helm chart values or environment configuration) [1]. Note that the NVIDIA DRA driver is increasingly being integrated into the broader NVIDIA GPU Operator ecosystem, which may simplify secret management for component images in future deployments [2][3].
Citations:
- 1: kubernetes-sigs/dra-driver-nvidia-gpu@v25.12.0...v0.4.0
- 2: https://github.com/NVIDIA/k8s-dra-driver-gpu/releases
- 3: https://gitlab.com/nvidia/kubernetes/gpu-operator
🏁 Script executed:
#!/bin/bash
set -euo pipefail
api='https://api.github.com/repos/NVIDIA/k8s-dra-driver-gpu'
printf '%s\n' '--- tag ---'
curl -fsSL "$api/git/ref/tags/v0.5.0" | jq -r '.object.type, .object.sha'
printf '%s\n' '--- tree paths containing controller or compute-domain ---'
sha=$(curl -fsSL "$api/git/ref/tags/v0.5.0" | jq -r '.object.sha')
curl -fsSL "$api/git/trees/$sha?recursive=1" |
jq -r '.tree[]?.path' |
rg -i 'compute.?domain|controller|daemon' | head -120
printf '%s\n' '--- exact variable in tag tree ---'
curl -fsSL "$api/git/trees/$sha?recursive=1" |
jq -r '.tree[]?.path' |
while read -r path; do
case "$path" in *.go|*.yaml|*.yml|*.md)
body=$(curl -fsSL "https://raw.githubusercontent.com/NVIDIA/k8s-dra-driver-gpu/$sha/$path" || true)
if printf '%s' "$body" | rg -q 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES'; then
printf '%s\n' "$path"
printf '%s\n' "$body" | rg -n -C 8 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES'
fi
esac
doneRepository: NVIDIA/gpu-operator
Length of output: 138
🏁 Script executed:
#!/bin/bash
set -euo pipefail
api='https://api.github.com/repos/NVIDIA/k8s-dra-driver-gpu'
sha=$(curl -fsSL "$api/git/ref/tags/v0.5.0" | jq -r '.object.sha')
curl -fsSL "$api/git/trees/$sha?recursive=1" |
jq -r '.tree[]?.path' |
rg -i 'compute.?domain|controller|daemon' | head -120
curl -fsSL "$api/git/trees/$sha?recursive=1" |
jq -r '.tree[]?.path' |
while read -r path; do
case "$path" in *.go|*.yaml|*.yml|*.md)
body=$(curl -fsSL "https://raw.githubusercontent.com/NVIDIA/k8s-dra-driver-gpu/$sha/$path" || true)
if printf '%s' "$body" | rg -q 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES'; then
printf '%s\n' "$path"
printf '%s\n' "$body" | rg -n -C 8 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES'
fi
esac
doneRepository: NVIDIA/gpu-operator
Length of output: 138
🏁 Script executed:
#!/bin/bash
set -euo pipefail
repo=https://github.com/NVIDIA/k8s-dra-driver-gpu.git
printf '%s\n' '--- tag commit ---'
git ls-remote "$repo" 'refs/tags/v0.5.0' 'refs/tags/v0.5.0^{}'
printf '%s\n' '--- candidate paths at v0.5.0 ---'
curl --fail --max-time 20 -sSL \
'https://api.github.com/repos/NVIDIA/k8s-dra-driver-gpu/git/trees/v0.5.0?recursive=1' |
jq -r '.tree[]?.path' |
rg -i 'compute.?domain|controller|daemon|pull.secret' | head -160Repository: NVIDIA/gpu-operator
Length of output: 8370
🏁 Script executed:
#!/bin/bash
set -euo pipefail
repo=https://github.com/NVIDIA/k8s-dra-driver-gpu.git
git ls-remote "$repo" 'refs/tags/v0.5.0' 'refs/tags/v0.5.0^{}'
curl --fail --max-time 20 -sSL \
'https://api.github.com/repos/NVIDIA/k8s-dra-driver-gpu/git/trees/v0.5.0?recursive=1' |
jq -r '.tree[]?.path' |
rg -i 'compute.?domain|controller|daemon|pull.secret' | head -160Repository: NVIDIA/gpu-operator
Length of output: 8317
🏁 Script executed:
#!/bin/bash
set -euo pipefail
base=https://raw.githubusercontent.com/NVIDIA/k8s-dra-driver-gpu/90b3a5917f0cbeb73a3c669e1e63690f89d0f437
for f in \
cmd/compute-domain-controller/main.go \
cmd/compute-domain-controller/controller.go \
cmd/compute-domain-controller/daemonset.go \
cmd/compute-domain-controller/daemonsetpods.go \
deployments/helm/dra-driver-nvidia-gpu/templates/controller.yaml; do
printf '\n--- %s ---\n' "$f"
curl --fail --max-time 20 -sSL "$base/$f" |
rg -n -C 10 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES|IMAGE_PULL_SECRET|ImagePullSecrets|imagePullSecrets|Split|split|DaemonSet' || true
doneRepository: NVIDIA/gpu-operator
Length of output: 21440
🏁 Script executed:
#!/bin/bash
set -euo pipefail
base=https://raw.githubusercontent.com/NVIDIA/k8s-dra-driver-gpu/90b3a5917f0cbeb73a3c669e1e63690f89d0f437
for f in cmd/compute-domain-controller/main.go cmd/compute-domain-controller/controller.go cmd/compute-domain-controller/daemonset.go cmd/compute-domain-controller/daemonsetpods.go deployments/helm/dra-driver-nvidia-gpu/templates/controller.yaml; do
printf '\n--- %s ---\n' "$f"
curl --fail --max-time 20 -sSL "$base/$f" |
rg -n -C 10 'CD_DAEMON_IMAGE_PULL_SECRET_NAMES|IMAGE_PULL_SECRET|ImagePullSecrets|imagePullSecrets|Split|split|DaemonSet' || true
doneRepository: NVIDIA/gpu-operator
Length of output: 21440
🏁 Script executed:
#!/bin/bash
set -euo pipefail
base=https://raw.githubusercontent.com/NVIDIA/k8s-dra-driver-gpu/90b3a5917f0cbeb73a3c669e1e63690f89d0f437
for f in templates/compute-domain-daemon.tmpl.yaml cmd/compute-domain-controller/computedomain.go cmd/compute-domain-controller/controller.go; do
printf '\n--- %s ---\n' "$f"
curl --fail --max-time 20 -sSL "$base/$f" |
nl -ba |
rg -n -C 12 'imagePullSecrets|ImagePullSecretNames|DaemonSetManager|\.Create\(|Create\(ctx' || true
doneRepository: NVIDIA/gpu-operator
Length of output: 598
🏁 Script executed:
#!/bin/bash
set -euo pipefail
base=https://raw.githubusercontent.com/NVIDIA/k8s-dra-driver-gpu/90b3a5917f0cbeb73a3c669e1e63690f89d0f437
for f in templates/compute-domain-daemon.tmpl.yaml cmd/compute-domain-controller/computedomain.go cmd/compute-domain-controller/controller.go; do
printf '\n--- %s ---\n' "$f"
curl --fail --max-time 20 -sSL "$base/$f" |
grep -n -C 12 -E 'imagePullSecrets|ImagePullSecretNames|DaemonSetManager|\.Create\(|Create\(ctx' || true
doneRepository: NVIDIA/gpu-operator
Length of output: 5421
Update existing compute-domain DaemonSets when image pull secrets change.
compute-domain-controller correctly parses comma-separated CD_DAEMON_IMAGE_PULL_SECRET_NAMES values, and an omitted variable produces no imagePullSecrets. However, its Create path returns an existing DaemonSet without updating it. Existing compute-domain-daemon DaemonSets can retain removed secrets and miss new ones, which can cause image pulls to fail. Reconcile the DaemonSet pod template when DRADriver.Spec.ImagePullSecrets changes.
1 participant
No description provided.