GPU Operators

This guide shows how to deploy, manage and verify GPU resources in RKE2 clusters. It covers installation, host requirements, and validation for both NVIDIA and AMD GPU operators.

Deploy NVIDIA operator

The NVIDIA operator allows administrators of Kubernetes clusters to manage GPUs just like CPUs. It includes everything needed for pods to operate GPUs.

Host OS requirements

To expose the GPU to the pod correctly, the NVIDIA kernel drivers and the libnvidia-ml library must be correctly installed in the host OS (Operating System). The NVIDIA Operator can automatically install drivers and libraries on some operating systems. Refer to the NVIDIA documentation for information on supported operating system releases.

Starting with GPU Operator v26.3.x, the operator can also manage driver and library installation on any operating system, provided the OS vendor or administrator supplies a compatible driver container image.

Installation of the NVIDIA components on your host OS is out of the scope of this document; reference the NVIDIA documentation for instructions.

Checks for pre-installed NVIDIA drivers/libraries

The following three commands should return a correct output if the kernel driver was correctly installed:

  • lsmod | grep nvidia returns a list of nvidia kernel modules. For example:

    nvidia_uvm           2129920  0
nvidia_drm            131072  0
nvidia_modeset       1572864  1 nvidia_drm
video                  77824  1 nvidia_modeset
nvidia               9965568  2 nvidia_uvm,nvidia_modeset
ecc                    45056  1 nvidia

  • cat /proc/driver/nvidia/version returns the NVRM and GCC version of the driver. For example:

    NVRM version: NVIDIA UNIX Open Kernel Module for x86_64  555.42.06  Release Build  (abuild@host)  Thu Jul 11 12:00:00 UTC 2024
GCC version:  gcc version 7.5.0 (SUSE Linux)

  • find /usr/ -iname libnvidia-ml.so returns a path to the libnvidia-ml.so library. For example:

    /usr/lib64/libnvidia-ml.so

    This library is used by Kubernetes components to interact with the kernel driver.

Operator installation

Once the OS is ready and RKE2 is running, install the GPU Operator with the following yaml manifest.

  • v26.3.x

  • v26.3.x with NRI

  • v25.3.x

  • v25.10.x

There are two installation options available.

If drivers and libraries are pre-installed or you are using a supported operating system by nvidia, please use the following manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.3.2
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock

If your operating system vendor supplies a compatible driver image, you can use the driver value field to point to it. For example, in SLES 16.0, you can use the following manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.3.2
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock
    driver:
      repository: registry.suse.com/third-party/nvidia
      usePrecompiled: true
      version: 595 # This depends on the nvidia driver that works with your GPU architecture

Node Resource Interface (NRI) specification is a pluggable extension mechanism built into container runtimes like containerd and CRI-O that allows custom plugins to intercept container lifecycle events on a node. It is considered the future integration mechanism for GPUs.

NVIDIA considers NRI as experimental

If you want to try it out, please use the following manifest:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v26.3.1
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    cdi:
      nriPluginEnabled: true
Version Gate

NRI requires containerd 2.1. Containerd 2.1 is available as of September 2025 releases: v1.31.13+rke2r1, v1.32.9+rke2r1, v1.33.5+rke2r1, v1.34.1+rke2r1

 
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v25.3.4
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock
      - name: ACCEPT_NVIDIA_VISIBLE_DEVICES_ENVVAR_WHEN_UNPRIVILEGED
        value: "false"
      - name: ACCEPT_NVIDIA_VISIBLE_DEVICES_AS_VOLUME_MOUNTS
        value: "true"
    devicePlugin:
      env:
      - name: DEVICE_LIST_STRATEGY
        value: volume-mounts

The envvars ACCEPT_NVIDIA_VISIBLE_DEVICES_ENVVAR_WHEN_UNPRIVILEGED, ACCEPT_NVIDIA_VISIBLE_DEVICES_AS_VOLUME_MOUNTS and DEVICE_LIST_STRATEGY are required to properly isolate GPU resources as explained in this nvidia doc

The NVIDIA operator restarts containerd with a hangup call which restarts RKE2

 
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: gpu-operator
  namespace: kube-system
spec:
  repo: https://helm.ngc.nvidia.com/nvidia
  chart: gpu-operator
  version: v25.10.1
  targetNamespace: gpu-operator
  createNamespace: true
  valuesContent: |-
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock

NVIDIA GPU Operator v25.10.x uses Container Device Interface (CDI) specification and that simplifies operations: we don’t need to pass extra envvars to comply with the security requirements and the workloads do not need to pass the runtimeClassName: nvidia anymore. It requires containerd 2.0.

The NVIDIA operator restarts containerd with a hangup call which restarts RKE2

After a few minutes, you can perform the following checks to verify that everything is working as expected:

  1. Assuming the drivers and libnvidia-ml.so library were previously installed, check if the operator detects them correctly:

    kubectl get node $NODENAME -o jsonpath='{.metadata.labels}' |  grep "nvidia.com"

    You should see labels specifying driver and GPU (e.g. nvidia.com/gpu.machine or nvidia.com/cuda.driver.major).

  2. Check if the GPU was added by nvidia-device-plugin-daemonset as an allocatable resource in the node:

    kubectl get node $NODENAME -o jsonpath='{.status.allocatable}'

    You should see "nvidia.com/gpu": followed by the number of GPUs in the node.

  3. Check that the container runtime binary exists (it gets installed by the nvidia-container-toolkit-daemonset):

    ls /usr/local/nvidia/toolkit/nvidia-container-runtime

  4. (Only if not using NRI) Verify if containerd config was updated to include the NVIDIA container runtime:

    grep nvidia /var/lib/rancher/rke2/agent/etc/containerd/config.toml

  5. Run a pod to verify that the GPU resource can successfully be scheduled on a pod and the pod can detect it

    apiVersion: v1
kind: Pod
metadata:
  name: nbody-gpu-benchmark
  namespace: default
spec:
  restartPolicy: OnFailure
  # runtimeClassName: nvidia <== Only needed for v25.3.x
  containers:
  - name: cuda-container
    image: nvcr.io/nvidia/k8s/cuda-sample:nbody
    args: ["nbody", "-gpu", "-benchmark"]
    resources:
      limits:
        nvidia.com/gpu: 1

RKE2 will now use PATH to find alternative container runtimes, in addition to checking the default paths used by the container runtime packages. In order to use this feature, you must modify the RKE2 service’s PATH environment variable to add the directories containing the container runtime binaries.

It’s recommended that you modify one of this two environment files:

  • /etc/default/rke2-server # or rke2-agent

  • /etc/sysconfig/rke2-server # or rke2-agent

This example will add the PATH in /etc/default/rke2-server:

PATH changes should be done with care to avoid placing untrusted binaries in the path of services that run as root.

 
echo PATH=$PATH >> /etc/default/rke2-server

Deploy AMD operator on SLES

The AMD GPU Operator allows administrators of Kubernetes clusters to manage AMD GPUs just like CPUs.

On SLES, the operator uses precompiled amdgpu driver images from the Trusted container images repository built against the codestream’s GA kernel. KMM relocates the modules to the node’s running kernel using SUSE’s stable kABI guarantee—​no recompilation needed across kernel updates within the same codestream. The resulting image is pushed to your registry of choice.

Driver images for SLES

The prebuild driver images for specific SLES versions are available from the following links:
Official AMD documentation

For full installation instructions, DeviceConfig reference, and troubleshooting, refer to the AMD GPU Operator official documentation.

Prerequisites

  • Verify that a supported AMD GPU PCI device is present.

    lspci -nn | grep -i "amd\|ati\|display\|vga"

  • Verify that the RKE2 cluster is up and running.

    kubectl get nodes

  • Install cert-manager. It is required for webhook TLS certificates.

Installing AMD GPU Operator

Once the prerequisites are met and RKE2 is running, install the AMD GPU Operator:

helm repo add rocm https://rocm.github.io/gpu-operator
helm repo update

helm install amd-gpu-operator rocm/gpu-operator-charts \
  --namespace kube-amd-gpu \
  --create-namespace \
  --version <gpu-operator-version> \
  --set crds.defaultCR.install=false

For the latest release version, see the AMD GPU Operator Helm installation documentation.

If using a local plain-HTTP registry for driver images during testing, configure containerd before applying the DeviceConfig. See RKE2 containerd registry configuration.

Create DeviceConfig

Follow the AMD GPU Operator driver installation documentation to create a DeviceConfig resource. On SLES, set the following field to use prebuilt driver images from registry.suse.com:

spec:
  driver:
    imageBuild:
      baseImageRegistry: "registry.suse.com"

For the full DeviceConfig spec reference, see the configuration reference.

Refer to AMD GPU Driver release history for available driver versions.

Verify the installation

After a few minutes, perform the following checks:

  1. Check that the kernel module was loaded on the node.

    lsmod | grep amdgpu

  2. Check that the GPU was added as an allocatable resource.

    kubectl get nodes -o custom-columns="NAME:.metadata.name,GPU:.status.allocatable.amd\.com/gpu"

    You should see 1 or more for multi-GPU nodes in the GPU column.

  3. Check that the node labeller applied GPU hardware labels.

    kubectl get node -o jsonpath='{.items[0].metadata.labels.amd\.com/gpu\.driver-version}{"\n"}'
kubectl get node -o jsonpath='{.items[0].metadata.labels.amd\.com/gpu\.product-name}{"\n"}'
kubectl get node -o jsonpath='{.items[0].metadata.labels.kmm\.node\.kubernetes\.io/version-module\.kube-amd-gpu\.amdgpu-sles}{"\n"}'

Troubleshooting

For troubleshooting, refer to the AMD GPU Operator Troubleshooting documentation.

Cleanup

For steps to uninstall the AMD GPU Operator and its dependencies, refer to the official documentaion.