Applies to SUSE AI 1.0

3 Application-specific requirements #

The SUSE AI stack consists of multiple applications. We recommend running each application on nodes that meet or exceed the corresponding hardware requirements.

3.1 SUSE Rancher Prime requirements #

3.1.1 Minimum hardware requirements #

Nodes for HA setup: At least 3 nodes.
RAM: A minimum of 32 GB of RAM.
CPU: At least 8 CPU cores.
Disk space: At least 200 GB of storage, preferably SSD.

3.1.2 For more information #

For more detailed recommendations, refer to the following official documentation:

3.2 SUSE Security requirements #

3.2.1 Minimum hardware requirements #

Nodes for HA setup

The following container instances run on existing cluster nodes:

1 Manager instance
3 Controller instances
1 Enforcer instance on each cluster node
2 Scanner & Updater instances

RAM

A minimum of 2 GB of RAM.

CPU

At least 2 CPU cores.

Disk space

At least 5 GB of storage, preferably SSD.

3.2.2 For more information #

For more detailed recommendations, refer to the following official documentation:

SUSE Security system requirements

3.3 SUSE Observability requirements #

3.3.1 Minimum hardware requirements #

Nodes for HA setup: At least 3 nodes.
RAM: A minimum of 32 GB of RAM.
CPU: At least 16 CPU cores.
Disk space: At least 5 GB of storage, preferably SSD.

3.3.2 For more information #

For more detailed recommendations, refer to the following official documentation:

SUSE Observability requirements for SUSE Rancher Prime

3.4 Milvus requirements #

This topic describes requirements for the Milvus application.

3.4.1 Hardware requirements #

3.4.1.1 Minimum requirements #

The following requirements are for basic Milvus deployment on a single node or a small scale.

RAM: A minimum of 32 GB of RAM.
CPU: At least 8 CPU cores.
Disk space: At least 100 GB of storage, preferably SSD.
Networking: A stable connection with 1 Gbps network bandwidth.

3.4.1.2 Recommended hardware for large-scale workloads #

The following requirements are for multi-node Milvus clusters or heavy workloads, such as large vector databases.

RAM: A minimum of 64 GB of RAM per node.
CPU: 8–16 CPU cores per node or more.
Disk space: 500 GB or more of high-speed storage, ideally SSD or NVMe SSD.
Networking: 10 Gbps Ethernet or faster for high-performance clusters.

3.4.1.3 CPU instruction set requirements #

The following CPU instruction sets are required for Milvus:

SSE4.2
AVX
AVX2
AVX-512

You can list the supported CPU sets on your host by running the following command:

> grep -m1 '^flags' /proc/cpuinfo

3.4.2 Software requirements #

Running Milvus requires specific versions of the following software:

Kubernetes: SUSE-supported versions of SUSE Rancher Prime: RKE2 that use Kubernetes 1.18 or higher.
Helm: The recommended version is 3.5.0 or later.

3.4.3 Additional considerations #

Disk and storage

Storage type: SSDs or NVMe SSDs are highly recommended for fast read/write access to large datasets and high-performance vector retrieval.
Metadata and data storage: For large-scale deployments, ensure that metadata and vector data are stored on fast disks (SSD or NVMe).

Network

For high-performance clusters, especially for large-scale deployments, ensure high-bandwidth network connectivity between nodes.

3.4.4 For more information #

For more detailed hardware recommendations, refer to the official Milvus and prerequisite Docker documentation.

3.5 Ollama requirements #

The version of Ollama provided with SUSE AI is optimized for NVIDIA GPU hardware. This section guides you through the steps for configuring Ollama on an NVIDIA-enabled system, including necessary configurations for both the hardware and software.

Tip: General recommendations

Run Ollama on NVIDIA GPU nodes. Since Ollama is GPU-optimized, using the power of NVIDIA GPUs is essential for maximum performance. This ensures that the application runs efficiently and fully uses the hardware capabilities.
Assign applications to specific nodes. SUSE AI provides a mechanism to assign applications, such as Ollama, to specific nodes. For more details, refer to https://documentation.suse.com/suse-ai/1.0/html/AI-deployment-intro/index.html#ai-gpu-nodes-assigning.

3.5.1 Hardware requirements #

NVIDIA GPU

The recommended GPU models include Tesla, A100, V100, RTX 30 series, or other compatible NVIDIA GPUs.
Ensure that the CUDA Compute Capability of your GPU is compatible with the required version of Ollama.

RAM

At least 16 GB of RAM is recommended. However, higher amounts (32 GB or more) may be necessary for larger models or workloads.

Disk space

At least 50 GB of free disk space is recommended for storing the container images and any data files processed by Ollama.

3.5.2 Software requirements #

NVIDIA Docker (nvidia-docker): You must install nvidia-docker (the NVIDIA Container Toolkit) to allow Docker containers to use the GPU. Refer to https://documentation.suse.com/cloudnative/rke2/latest/en/advanced.html#_deploy_nvidia_operator for more details.
CUDA Toolkit: You must install the CUDA version supported by your GPU model. For most recent GPUs, CUDA 11.0 or later is required. Refer to CUDA Toolkit installation guide for more details.
NVIDIA driver: Install the NVIDIA driver compatible with your GPU model. Its version must be compatible with the installed CUDA toolkit.
Tip
You can check your GPU driver version by running the nvidia-smi command.

3.6 Open WebUI requirements #

While Open WebUI has no specific hardware dependencies beyond those of the underlying platform, consider the following guidelines for optimal performance.

Because Open WebUI shares most dependencies with Milvus and Ollama, follow the hardware requirements mentioned in Section 3.4, “Milvus requirements” and Section 3.5, “Ollama requirements”.
Stable network connection is essential, particularly if Open WebUI is integrated with other services or databases. Ensure sufficient bandwidth for Web traffic and API calls.
To interact with the Open WebUI interface, use standard Web browsers such as Google Chrome, Mozilla Firefox or Microsoft Edge.