GPU Operators

This guide shows how to deploy, manage and verify GPU resources in RKE2 clusters. It covers installation, host requirements, and validation for both NVIDIA and AMD GPU operators.

Deploy NVIDIA operator

The NVIDIA operator allows administrators of Kubernetes clusters to manage GPUs just like CPUs. It includes everything needed for pods to operate GPUs.

Host OS requirements

To expose the GPU to the pod correctly, the NVIDIA kernel drivers and the libnvidia-ml library must be correctly installed in the host OS (Operating System). The NVIDIA Operator can automatically install drivers and libraries on some operating systems. Refer to the NVIDIA documentation for information on supported operating system releases.

Starting with GPU Operator v26.3.x, the operator can also manage driver and library installation on any operating system, provided the OS vendor or administrator supplies a compatible driver container image.

Installation of NVIDIA components on the host OS is outside the scope of this document. Refer to the NVIDIA documentation for driver installation instructions if you are using an NVIDIA vGPU.

Checks for pre-installed NVIDIA drivers/libraries

The following three commands should return a correct output if the kernel driver was correctly installed:

  • lsmod | grep nvidia returns a list of nvidia kernel modules. For example:

    nvidia_uvm           2129920  0
    nvidia_drm            131072  0
    nvidia_modeset       1572864  1 nvidia_drm
    video                  77824  1 nvidia_modeset
    nvidia               9965568  2 nvidia_uvm,nvidia_modeset
    ecc                    45056  1 nvidia
  • cat /proc/driver/nvidia/version returns the NVRM and GCC version of the driver. For example:

    NVRM version: NVIDIA UNIX Open Kernel Module for x86_64  555.42.06  Release Build  (abuild@host)  Thu Jul 11 12:00:00 UTC 2024
    GCC version:  gcc version 7.5.0 (SUSE Linux)
  • find /usr/ -iname libnvidia-ml.so returns a path to the libnvidia-ml.so library. For example:

    /usr/lib64/libnvidia-ml.so

    This library is used by Kubernetes components to interact with the kernel driver.

Operator installation

Once the OS is ready and RKE2 is running, install the GPU Operator with the following yaml manifest.

  • v25.10.x

  • v26.3.x

  • v26.7.x

  • DRA (Experimental)

  • NRI (Experimental)

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v25.10.1
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock

NVIDIA GPU Operator v25.10.x uses Container Device Interface (CDI) specification and that simplifies operations: we don’t need to pass extra envvars to comply with the security requirements and the workloads do not need to pass the runtimeClassName: nvidia anymore. It requires containerd 2.0.

The NVIDIA operator restarts containerd with a hangup call which restarts RKE2

End of Support

Per NVIDIA’s GPU Operator life cycle policy, v25.10.x (and all older major versions) is now End of Support and no longer receives patch, bug fix, or CVE updates. Please upgrade to a currently supported major version.

There are two installation options available.

If drivers and libraries are pre-installed or you are using a supported operating system by nvidia, please use the following manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.3.3
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock

If your operating system vendor supplies a compatible driver image, you can use the driver value field to point to it. For example, in SLES 16.0, you can use the following manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.3.3
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock
    driver:
      repository: registry.suse.com/third-party/nvidia
      usePrecompiled: true
      version: 595 # This depends on the nvidia driver that works with your GPU architecture
Deprecated

Per NVIDIA’s GPU Operator life cycle policy, v26.3.x is now Deprecated: it still receives critical bug fix and CVE patch updates, but no new features. We recommend moving to v26.7.x below.

There are two installation options available.

If drivers and libraries are pre-installed or you are using a supported operating system by nvidia, please use the following manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.7.2
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock

If your operating system vendor supplies a compatible driver image, you can use the driver value field to point to it. For example, in SLES 16.0, you can use the following manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.7.2
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock
    driver:
      repository: registry.suse.com/third-party/nvidia
      usePrecompiled: true
      version: 595 # This depends on the nvidia driver that works with your GPU architecture
Minimum containerd version

Starting with v26.7.0, the GPU Operator requires containerd 2.0 or later.

Dynamic Resource Allocation (DRA)

v26.7.x also supports installing with a GPUCluster custom resource instead of ClusterPolicy, enabling GPU allocation via native Kubernetes DRA. See the "DRA (Experimental)" tab.

v26.7.x introduces a GPUCluster custom resource as an alternative to ClusterPolicy. It deploys the DRA driver for NVIDIA GPUs and creates the gpu.nvidia.com, mig.nvidia.com, and vfio.gpu.nvidia.com DeviceClass`es, enabling GPU allocation via native Kubernetes Dynamic Resource Allocation (`ResourceClaim/ResourceClaimTemplate) instead of the classic device-plugin extended-resource model (nvidia.com/gpu).

A cluster can run either ClusterPolicy or GPUCluster, not both. Per NVIDIA, in-place migration from an existing ClusterPolicy install to GPUCluster is not supported — if a ClusterPolicy resource already exists, use a new cluster for the GPUCluster workflow.

Minimum requirements

Kubernetes v1.34+ (for the resource.k8s.io DeviceClass API), NVIDIA driver 580 or later, and a container runtime with CDI support enabled.

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.7.2
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock
    clusterPolicy:
      deployCR: false
    gpuCluster:
      deployCR: true
    driver:
      nvidiaDriverCRD:
        enabled: true
      repository: registry.suse.com/third-party/nvidia
      usePrecompiled: true
      version: 595 # This depends on the nvidia driver that works with your GPU architecture

After a few minutes, verify the GPUCluster reconciled successfully:

kubectl get gpucluster gpu-cluster

Expected output:

NAME          STATUS   AGE
gpu-cluster   ready    8m

Confirm the `DeviceClass`es were created:

kubectl get deviceclass

Expected output:

NAME                                        AGE
compute-domain-daemon.nvidia.com            7m
compute-domain-default-channel.nvidia.com   7m
gpu.nvidia.com                              7m
mig.nvidia.com                              7m
vfio.gpu.nvidia.com                         7m

Run a pod requesting a GPU via a ResourceClaimTemplate instead of the classic nvidia.com/gpu extended resource:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: single-gpu
spec:
  spec:
    devices:
      requests:
      - name: gpu
        exactly:
          deviceClassName: gpu.nvidia.com
---
apiVersion: v1
kind: Pod
metadata:
  name: nbody-gpu-benchmark-dra
spec:
  restartPolicy: OnFailure
  containers:
  - name: cuda-container
    image: nvcr.io/nvidia/k8s/cuda-sample:nbody
    args: ["nbody", "-gpu", "-benchmark"]
    resources:
      claims:
      - name: gpu
  resourceClaims:
  - name: gpu
    resourceClaimTemplateName: single-gpu

Check that the ResourceClaim was allocated and the pod ran successfully:

kubectl get resourceclaim
kubectl logs nbody-gpu-benchmark-dra

Node Resource Interface (NRI) specification is a pluggable extension mechanism built into container runtimes like containerd and CRI-O that allows custom plugins to intercept container lifecycle events on a node. It is considered the future integration mechanism for GPUs.

NVIDIA considers NRI as experimental

NRI is not tied to a specific GPU Operator release — it’s enabled by adding the following to the valuesContent of whichever version’s manifest you’re using (v26.3.x or v26.7.x above):

    cdi:
      nriPluginEnabled: true
Version Gate

NRI requires containerd 2.1. Containerd 2.1 is available as of September 2025 releases: v1.31.13+rke2r1, v1.32.9+rke2r1, v1.33.5+rke2r1, v1.34.1+rke2r1

The checks below apply to the ClusterPolicy-based (device-plugin) tabs above. If you installed with GPUCluster/DRA instead, see the verification steps in that tab instead — allocatable GPUs aren’t exposed as the nvidia.com/gpu extended resource in DRA mode.

After a few minutes, you can perform the following checks to verify that everything is working as expected:

  1. Assuming the drivers and libnvidia-ml.so library were previously installed, check if the operator detects them correctly:

    kubectl get node $NODENAME -o jsonpath='{.metadata.labels}' |  grep "nvidia.com"

    You should see labels specifying driver and GPU (e.g. nvidia.com/gpu.machine or nvidia.com/cuda.driver.major).

  2. Check if the GPU was added by nvidia-device-plugin-daemonset as an allocatable resource in the node:

    kubectl get node $NODENAME -o jsonpath='{.status.allocatable}'

    You should see "nvidia.com/gpu": followed by the number of GPUs in the node.

  3. Check that the container runtime binary exists (it gets installed by the nvidia-container-toolkit-daemonset):

    ls /usr/local/nvidia/toolkit/nvidia-container-runtime
  4. (Only if not using NRI) Verify if containerd config was updated to include the NVIDIA container runtime:

    grep nvidia /var/lib/rancher/rke2/agent/etc/containerd/config.toml
  5. Run a pod to verify that the GPU resource can successfully be scheduled on a pod and the pod can detect it

    apiVersion: v1
    kind: Pod
    metadata:
      name: nbody-gpu-benchmark
      namespace: default
    spec:
      restartPolicy: OnFailure
      containers:
      - name: cuda-container
        image: nvcr.io/nvidia/k8s/cuda-sample:nbody
        args: ["nbody", "-gpu", "-benchmark"]
        resources:
          limits:
            nvidia.com/gpu: 1

Deploy AMD operator on SLES

The AMD GPU Operator allows administrators of Kubernetes clusters to manage AMD GPUs just like CPUs.

On SLES, the operator uses precompiled amdgpu driver images from the Trusted container images repository built against the codestream’s GA kernel. KMM relocates the modules to the node’s running kernel using SUSE’s stable kABI guarantee—​no recompilation needed across kernel updates within the same codestream. The resulting image is pushed to your registry of choice.

Driver images for SLES

The prebuild driver images for specific SLES versions are available from the following links:

Official AMD documentation

For full installation instructions, DeviceConfig reference, and troubleshooting, refer to the AMD GPU Operator official documentation.

Prerequisites

  • Verify that a supported AMD GPU PCI device is present.

    lspci -nn | grep -i "amd\|ati\|display\|vga"
  • Verify that the RKE2 cluster is up and running.

    kubectl get nodes
  • Install cert-manager. It is required for webhook TLS certificates.

Installing AMD GPU Operator

Once the prerequisites are met and RKE2 is running, install the AMD GPU Operator:

helm repo add rocm https://rocm.github.io/gpu-operator
helm repo update

helm install amd-gpu-operator rocm/gpu-operator-charts \
  --namespace kube-amd-gpu \
  --create-namespace \
  --version <gpu-operator-version> \
  --set crds.defaultCR.install=false

For the latest release version, see the AMD GPU Operator Helm installation documentation.

If using a local plain-HTTP registry for driver images during testing, configure containerd before applying the DeviceConfig. See RKE2 containerd registry configuration.

Create DeviceConfig

Follow the AMD GPU Operator driver installation documentation to create a DeviceConfig resource. On SLES, set the following field to use prebuilt driver images from registry.suse.com:

spec:
  driver:
    imageBuild:
      baseImageRegistry: "registry.suse.com"

For the full DeviceConfig spec reference, see the configuration reference.

Refer to AMD GPU Driver release history for available driver versions.

Verify the installation

After a few minutes, perform the following checks:

  1. Check that the kernel module was loaded on the node.

    lsmod | grep amdgpu
  2. Check that the GPU was added as an allocatable resource.

    kubectl get nodes -o custom-columns="NAME:.metadata.name,GPU:.status.allocatable.amd\.com/gpu"

    You should see 1 or more for multi-GPU nodes in the GPU column.

  3. Check that the node labeller applied GPU hardware labels.

    kubectl get node -o jsonpath='{.items[0].metadata.labels.amd\.com/gpu\.driver-version}{"\n"}'
    kubectl get node -o jsonpath='{.items[0].metadata.labels.amd\.com/gpu\.product-name}{"\n"}'
    kubectl get node -o jsonpath='{.items[0].metadata.labels.kmm\.node\.kubernetes\.io/version-module\.kube-amd-gpu\.amdgpu-sles}{"\n"}'

Troubleshooting

For troubleshooting, refer to the AMD GPU Operator Troubleshooting documentation.

Cleanup

For steps to uninstall the AMD GPU Operator and its dependencies, refer to the official documentaion.