diff --git a/gpu-operator/gpu-operator-rdma.rst b/gpu-operator/gpu-operator-rdma.rst index 2d3893f00..fb883a5a3 100644 --- a/gpu-operator/gpu-operator-rdma.rst +++ b/gpu-operator/gpu-operator-rdma.rst @@ -32,8 +32,8 @@ NVIDIA recommends using the DMA-BUF rather than using the ``nvidia-peermem`` ker The Operator uses GDS driver version 2.17.5 or newer. This version and higher is only supported with the NVIDIA Open GPU Kernel module driver. -In GPU Operator v25.3.0 and later, the ``driver.kernelModuleType`` default is ``auto``, for the supported driver versions. -This configuration allows the GPU Operator to choose the recommended driver kernel module type depending on the driver branch and the GPU devices available. +In GPU Operator v25.3.0 and later, the ``driver.kernelModuleType`` default is ``auto``, for the supported driver versions. +This configuration allows the GPU Operator to choose the recommended driver kernel module type depending on the driver branch and the GPU devices available. Newer driver versions will use the open kernel module by default, however to make sure you are using the open kernel module, include ``--set driver.kernelModuleType=open`` command-line argument in your helm Operator install command. In conjunction with the Network Operator, the GPU Operator can be used to diff --git a/gpu-operator/life-cycle-policy.rst b/gpu-operator/life-cycle-policy.rst index 73f26e11e..4744f24df 100644 --- a/gpu-operator/life-cycle-policy.rst +++ b/gpu-operator/life-cycle-policy.rst @@ -88,10 +88,11 @@ Refer to :ref:`Upgrading the NVIDIA GPU Operator` for more information. :header-rows: 2 * - :rspan:`1` Component - - :cspan:`1` GPU Operator Version + - :cspan:`2` GPU Operator Version * - v26.7.0 - v26.7.1 + - v26.7.2 * - NVIDIA GPU Driver - | `610.57.04 `_ @@ -101,66 +102,72 @@ Refer to :ref:`Upgrading the NVIDIA GPU Operator` for more information. | `610.57.04 `_ | `595.91.07 `_ (**R**, **D**) | `580.173.02 `_ + - | `615.71.09 `_ + | `610.57.04 `_ + | `595.91.07 `_ (**R**, **D**) + | `580.173.02 `_ * - NVIDIA Driver Manager for Kubernetes - `v0.12.0 `__ - - `v0.12.1 `__ + - :cspan:`1` `v0.12.1 `__ * - NVIDIA Container Toolkit - `1.20.0 `__ - - `1.20.1 `__ + - :cspan:`1` `1.20.1 `__ * - NVIDIA Device Plugin for Kubernetes - `0.20.0 `__ - - `0.20.1 `__ + - :cspan:`1` `0.20.1 `__ * - DRA Driver for NVIDIA GPUs - - :cspan:`1` `v${dra_version} `__ + - :cspan:`1` `v0.5.0 `__ + - `v${dra_version} `__ * - DCGM Exporter - `v4.6.0-4.8.3 `__ - - `v4.6.1-4.8.4 `__ + - :cspan:`1` `v4.6.1-4.8.4 `__ * - Node Feature Discovery - - :cspan:`1` `v0.19.0 `__ + - :cspan:`2` `v0.19.0 `__ * - | NVIDIA GPU Feature Discovery | for Kubernetes - `0.20.0 `__ - - `0.20.1 `__ + - :cspan:`1` `0.20.1 `__ * - NVIDIA MIG Manager for Kubernetes - `0.15.0 `__ - - `0.15.1 `__ + - :cspan:`1` `0.15.1 `__ * - DCGM - `4.6.0-1 `__ - - `4.6.1-1 `__ + - :cspan:`1` `4.6.1-1 `__ * - Validator for NVIDIA GPU Operator - v26.7.0 - v26.7.1 + - v26.7.2 * - NVIDIA KubeVirt GPU Device Plugin - - :cspan:`1` `v1.6.0 `__ + - :cspan:`2` `v1.6.0 `__ * - NVIDIA vGPU Device Manager - `v0.5.0 `__ - - `v0.5.1 `__ + - :cspan:`1` `v0.5.1 `__ * - NVIDIA GDS Driver |gds|_ - - :cspan:`1` `2.29.4 `__ + - :cspan:`2` `2.29.4 `__ * - | NVIDIA Confidential Computing | Manager for Kubernetes - `v0.4.3 `__ - - `v0.4.4 `__ + - :cspan:`1` `v0.4.4 `__ * - NVIDIA GDRCopy Driver - - :cspan:`1` `v2.6 `__ + - :cspan:`2` `v2.6 `__ * - NVIDIA Kata Sandbox Device Plugin - - :cspan:`1` `v0.0.5 `__ + - :cspan:`2` `v0.0.5 `__ .. _gds-open-kernel: diff --git a/gpu-operator/nvidia-driver-configuration.rst b/gpu-operator/nvidia-driver-configuration.rst index a5088f80c..5121e7152 100644 --- a/gpu-operator/nvidia-driver-configuration.rst +++ b/gpu-operator/nvidia-driver-configuration.rst @@ -310,13 +310,13 @@ The following table describes some of the fields in the custom resource. - None * - ``default`` - - Specifies whether the resource is the fallback driver configuration for GPU nodes that do not + - Optional. Specifies whether the resource is the fallback driver configuration for GPU nodes that do not match a non-default resource. Only one resource can be the default, and a default resource cannot specify ``nodeSelector``. - ``false`` * - ``driverType`` - - Specifies one of the following: + - Optional. Specifies one of the following: - ``gpu`` to use the NVIDIA data-center GPU driver. - ``vgpu`` to use the NVIDIA vGPU guest driver. diff --git a/gpu-operator/release-notes.rst b/gpu-operator/release-notes.rst index e416bb22e..a82afb21e 100644 --- a/gpu-operator/release-notes.rst +++ b/gpu-operator/release-notes.rst @@ -33,6 +33,61 @@ Refer to the :ref:`GPU Operator Component Matrix` for a list of software compone ---- +.. _v26.7.2: + +26.7.2 +====== + +New Features +------------ + +* Updated software component versions: + + - DRA Driver for NVIDIA GPUs v0.5.1 + +Improvements +------------ + +* The NVIDIA DCGM and NVIDIA DCGM Exporter daemon sets, pods, and services now include the ``app.kubernetes.io/component`` label with the value ``nvidia-dcgm`` or ``nvidia-dcgm-exporter``. + The label applies to deployments managed by the ``ClusterPolicy`` resource and by the ``GPUCluster`` resource. + If you specify the ``app.kubernetes.io/component`` label in the ``daemonsets.labels`` field, the Operator does not apply your value to operands that already have the label, such as NVIDIA DCGM, NVIDIA DCGM Exporter, and the NVIDIA GPU Driver. + The Operator logs a message when it skips a conflicting value. + (`GPU Operator PR #3017 `__) + +Fixed Issues +------------ + +* Fixed an issue on Red Hat OpenShift Container Platform where workloads that request MPS sharing remained in the ``ContainerCreating`` state when you install the DRA Driver for NVIDIA GPUs with the ``GPUCluster`` resource and enable the ``MPSSupport`` feature gate. + The MPS control daemon now runs with the DRA driver service account, and the ``nvidia-dra-driver`` SCC now allows host PID access. + (`GPU Operator PR #3008 `__) + +* Fixed an issue on Red Hat OpenShift Container Platform where ComputeDomain daemon pods failed to start when you install the DRA Driver for NVIDIA GPUs with the ``GPUCluster`` resource and use ComputeDomains. + The ComputeDomain daemon service account can now use the ``anyuid`` SCC, and the ComputeDomain daemon role can now delete ``ComputeDomainClique`` resources. + (`GPU Operator PR #3011 `__, `Issue #3010 `__) + +* Fixed an issue where changing a node label used in an ``NVIDIADriver`` node selector did not always reassign the node to the matching NVIDIA driver resource. + The Operator now reconciles driver ownership when any label in any NVIDIA driver node selector is added, changed, or removed. + (`GPU Operator PR #3003 `__, `Issue #3004 `__) + +* Fixed an issue where the MPS control daemon did not reload after an MPS sharing configuration change. + The configuration manager now signals the ``mps-control-daemon`` process by the process name that the container starts. + (`GPU Operator PR #2974 `__) + +* Fixed an issue where the Operator could not update recurring Kubernetes events, such as ``GPUDriverUpgrade`` events. + The Operator cluster role now includes the ``patch`` verb for events. + (`GPU Operator PR #2995 `__) + +* Fixed an issue where OLM installations on Arm64 GPU nodes failed to start NVIDIA vGPU Device Manager with ``Exec format error``. + The OLM bundle now pins the multi-architecture image index digest for NVIDIA vGPU Device Manager v0.5.1. + (`GPU Operator PR #3013 `__, `Issue #3012 `__) + +* Fixed an issue where Operator Lifecycle Manager (OLM) upgrades could fail with ``spec.default: Required value`` when validating existing ``NVIDIADriver`` resources. + The ``spec.default`` and ``spec.driverType`` fields are now optional, with defaults of ``false`` and ``gpu``, respectively. + (`GPU Operator PR #3046 `__) + + +---- + .. _v26.7.1: 26.7.1 diff --git a/openshift/install-gpu-ocp.rst b/openshift/install-gpu-ocp.rst index 5902f2deb..2cdc1e661 100644 --- a/openshift/install-gpu-ocp.rst +++ b/openshift/install-gpu-ocp.rst @@ -139,7 +139,7 @@ As a cluster administrator, you can install the **NVIDIA GPU Operator** using th subscription.operators.coreos.com/gpu-operator-certified created -#. Optional: Log in to the web console and navigate to the **Ecosystem** > **Installed Operators** and select **NVIDIA GPU Operator**. +#. Optional: Log in to the web console and navigate to the **Ecosystem** > **Installed Operators** and select **NVIDIA GPU Operator**. In versions before 4.20, navigate to **Operators** > **Installed Operators** and select **NVIDIA GPU Operator**. In the ``Project: nvidia-gpu-operator`` the following is displayed: @@ -199,7 +199,7 @@ when using **NVIDIA vGPU**. Refer to the appropriate sections below. .. _create-cluster-policy-web-console: Create the cluster policy using the web console ------------------------------------------------ +=============================================== #. In the OpenShift Container Platform web console, from the side menu, select **Ecosystem** > **Installed Operators** (for versions before 4.20, look for **Operators** > **Installed Operators**), and click **NVIDIA GPU Operator**. @@ -224,14 +224,13 @@ Create the cluster policy using the web console .. _verify-gpu-operator-install-ocp: Create the cluster policy using the CLI ---------------------------------------- - +======================================= #. Create the ClusterPolicy: .. code-block:: console - $ oc get csv -n nvidia-gpu-operator $STARTING_CSV -o jsonpath='{.metadata.annotations.alm-examples}' | jq -r 'map(select(.kind == "ClusterPolicy")) | .[0]' > clusterpolicy.json + $ oc get csv -n nvidia-gpu-operator $STARTING_CSV -o jsonpath='{.metadata.annotations.alm-examples}' | jq -r 'map(select(.kind == "ClusterPolicy")) | .[0]' > clusterpolicy.json .. note:: $STARTING_CSV is the value of the ``startingCSV`` field in the ``Subscription`` CR sample created in the :ref:`install-gpu-ocp` section. @@ -258,7 +257,7 @@ Create the ClusterPolicy instance with NVIDIA vGPU *************************************************************************** Prerequisites --------------- +============== Before creating the ClusterPolicy for NVIDIA vGPU, ensure the following: @@ -288,7 +287,7 @@ Before creating the ClusterPolicy for NVIDIA vGPU, ensure the following: -n nvidia-gpu-operator Create the cluster policy using the web console ------------------------------------------------ +=============================================== #. In the OpenShift Container Platform web console, from the side menu, select **Ecosystem** > **Installed Operators** and select **NVIDIA GPU Operator**. In versions before 4.20, navigate to **Operators** > **Installed Operators** and select **NVIDIA GPU Operator**. @@ -296,8 +295,8 @@ Create the cluster policy using the web console #. Select the **ClusterPolicy** tab, then click **Create ClusterPolicy**. The platform assigns the default name *gpu-cluster-policy*. #. Under the **NVIDIA GPU/vGPU Driver config** section fill in the following information - - * Specify the ``secretName`` as the name of the licensing ``Secret`` created during the prerequisite steps for NVIDIA vGPU (for example, ``licensing-config``). + + * Specify the ``secretName`` as the name of the licensing ``Secret`` created during the prerequisite steps for NVIDIA vGPU (for example, ``licensing-config``). .. note:: @@ -319,13 +318,13 @@ Create the cluster policy using the web console Create the cluster policy using the CLI ---------------------------------------- +======================================= #. Create the ClusterPolicy: .. code-block:: console - $ oc get csv -n nvidia-gpu-operator $STARTING_CSV -o jsonpath='{.metadata.annotations.alm-examples}' | jq -r 'map(select(.kind == "ClusterPolicy")) | .[0]') > clusterpolicy.json + $ oc get csv -n nvidia-gpu-operator $STARTING_CSV -o jsonpath='{.metadata.annotations.alm-examples}' | jq -r 'map(select(.kind == "ClusterPolicy")) | .[0]') > clusterpolicy.json .. note:: $STARTING_CSV is the value of the ``startingCSV`` field in the ``Subscription`` CR sample created in the :ref:`install-gpu-ocp` section. diff --git a/repo.toml b/repo.toml index 116c0da3a..9761da848 100644 --- a/repo.toml +++ b/repo.toml @@ -176,7 +176,7 @@ docs_root = "${root}/gpu-operator" project = "gpu-operator" name = "NVIDIA GPU Operator" version = "26.7" # Update repo_docs.projects.openshift.version to match latest patch version maj.min.patch -source_substitutions = { minor_version = "26.7", version = "v26.7.1", recommended = "595.91.07", dra_version = "0.5.0", kata_version = "4.0.0" } +source_substitutions = { minor_version = "26.7", version = "v26.7.2", recommended = "595.91.07", dra_version = "0.5.1", kata_version = "4.0.0" } copyright_start = 2020 sphinx_exclude_patterns = [ "life-cycle-policy.rst",