Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / SUSE AI requirements

SUSE AI requirements

Publication Date: 19 Dec 2024
WHAT?

Hardware, software and networking requirements for successful deployment and operation of SUSE AI.

WHY?

To ensure efficient operation of SUSE AI.

EFFORT

Less than 30 minutes of reading and a basic knowledge of hardware and SUSE Rancher Prime: RKE2 environment.

This article describes the software, hardware and networking requirements for the cluster nodes where you plan to install SUSE AI.

1 Hardware requirements

For successful deployment and operation, SUSE AI has the same hardware prerequisites as an SUSE Rancher Prime: RKE2 cluster.

1.1 Recommended hardware (basic functionality)

  • RAM
  • CPU
  • Disk space
  • Networking

At least 32 GB of RAM per node. This is the minimum recommendation for the control plane node. Additional resources may be needed for the worker nodes based on workload.

1.2 Recommended hardware (for High Availability)

Important
Important

While 32 GB of RAM is the minimum for basic functionality, a production-grade deployment with high availability, multi-node clusters, or running resource-intensive applications like AI/ML workloads might require more.

  • RAM
  • CPU
  • Disk space
  • Networking

64 GB or more per node is recommended for larger clusters or to run applications with high resource demands.

Tip
Tip

For more detailed hardware recommendations, refer to the official SUSE Rancher Prime: RKE2 installation requirements documentation at https://documentation.suse.com/cloudnative/rke2/latest/en/install/requirements.html.

2 GPU hardware

To run AI/ML workloads, such as training machine learning models or running inference workloads, deploy cluster nodes with compatible NVIDIA GPUs to gain acceleration.

2.1 Using the NVIDIA GPU Operator

Configuring and managing nodes with hardware resources can require multiple configurations for software components. These include drivers, container runtimes and libraries. To use NVIDIA GPUs in a Kubernetes cluster, you need to configure the NVIDIA GPU Operator. Because GPU is a special resource in the cluster, you need to install the following components to enable deployment of workloads for processing on the GPU.

  • NVIDIA drivers (to enable CUDA)

  • Kubernetes device plug-in

  • Container runtime

  • Other tools to provide capabilities such as monitoring or automatic node labeling

To ensure that the NVIDIA GPU Operator is installed correctly, the Kubernetes cluster must meet the following prerequisites:

  • All worker nodes must run the same operating system version to use the NVIDIA GPU Driver container.

  • Nodes must be configured with a container engine, such as Docker (CE/EE), containerd or Podman.

  • Nodes should be equipped with NVIDIA GPUs.

  • Nodes should have NVIDIA drivers installed.

2.2 Supported GPUs

The NVIDIA GPU Operator is compatible with a range of NVIDIA GPUs. For a full list of supported GPUs, refer to NVIDIA GPU Operator Platform Support.

3 Application-specific requirements

The SUSE AI stack consists of multiple applications. We recommend running each application on nodes that meet or exceed the corresponding hardware requirements.

3.1 Milvus hardware and software requirements

This topic describes requirements for the Milvus application.

3.1.1 Hardware requirements

3.1.1.1 Minimum requirements

The following requirements are for basic Milvus deployment on a single node or a small scale.

  • RAM
  • CPU
  • Disk space
  • Networking

A minimum of 32 GB of RAM.

3.1.1.2 Recommended hardware for large-scale workloads

The following requirements are for multi-node Milvus clusters or heavy workloads, such as large vector databases.

  • RAM
  • CPU
  • Disk space
  • Networking

A minimum of 64 GB of RAM per node.

3.1.1.3 CPU instruction set requirements

The following CPU instruction sets are required for Milvus:

  • SSE4.2

  • AVX

  • AVX2

  • AVX-512

3.1.2 Software requirements

Running Milvus requires specific versions of the following software:

Kubernetes

SUSE-supported versions of SUSE Rancher Prime: RKE2 that use Kubernetes 1.18 or higher.

Helm

The recommended version is 3.5.0 or later.

3.1.3 Additional considerations

Disk and storage
  • Storage type: SSDs or NVMe SSDs are highly recommended for fast read/write access to large datasets and high-performance vector retrieval.

  • Metadata and data storage: For large-scale deployments, ensure that metadata and vector data are stored on fast disks (SSD or NVMe).

Network

For high-performance clusters, especially for large-scale deployments, ensure high-bandwidth network connectivity between nodes.

3.1.4 For more information

For more detailed hardware recommendations, refer to the official Milvus and prerequisite Docker documentation.

3.2 Ollama hardware and software requirements

The version of Ollama provided with SUSE AI is optimized for NVIDIA GPU hardware. This section guides you through the steps for configuring Ollama on an NVIDIA-enabled system, including necessary configurations for both the hardware and software.

Tip
Tip: General recommendations

3.2.1 Hardware requirements

  • NVIDIA GPU
  • RAM
  • Disk space
  • The recommended GPU models include Tesla, A100, V100, RTX 30 series, or other compatible NVIDIA GPUs.

  • Ensure that the CUDA Compute Capability of your GPU is compatible with the required version of Ollama.

3.2.2 Software requirements

  • NVIDIA Docker (nvidia-docker)
  • CUDA Toolkit
  • NVIDIA driver

3.3 Open WebUI hardware and software requirements

While Open WebUI has no specific hardware dependencies beyond those of the underlying platform, consider the following guidelines for optimal performance.

  • Because Open WebUI shares most dependencies with Milvus and Ollama, follow the hardware requirements mentioned in Milvus requirements and Ollama requirements.

  • Stable network connection is essential, particularly if Open WebUI is integrated with other services or databases. Ensure sufficient bandwidth for Web traffic and API calls.

  • To interact with the Open WebUI interface, use standard Web browsers such as Google Chrome, Mozilla Firefox or Microsoft Edge.