SUSE AI requirements
- WHAT?
Hardware, software and networking requirements for successful deployment and operation of SUSE AI.
- WHY?
To ensure efficient operation of SUSE AI.
- EFFORT
Less than 30 minutes of reading and a basic knowledge of hardware and SUSE Rancher Prime: RKE2 environment.
This article describes the software, hardware and networking requirements for the cluster nodes where you plan to install SUSE AI.
1 Hardware requirements #
For successful deployment and operation, SUSE AI has the same hardware prerequisites as an SUSE Rancher Prime: RKE2 cluster.
1.1 Recommended hardware (basic functionality) #
- RAM
- CPU
- Disk space
- Networking
At least 32 GB of RAM per node. This is the minimum recommendation for the control plane node. Additional resources may be needed for the worker nodes based on workload.
1.2 Recommended hardware (for High Availability) #
While 32 GB of RAM is the minimum for basic functionality, a production-grade deployment with high availability, multi-node clusters, or running resource-intensive applications like AI/ML workloads might require more.
- RAM
- CPU
- Disk space
- Networking
64 GB or more per node is recommended for larger clusters or to run applications with high resource demands.
For more detailed hardware recommendations, refer to the official SUSE Rancher Prime: RKE2 installation requirements documentation at https://documentation.suse.com/cloudnative/rke2/latest/en/install/requirements.html.
2 GPU hardware #
To run AI/ML workloads, such as training machine learning models or running inference workloads, deploy cluster nodes with compatible NVIDIA GPUs to gain acceleration.
2.1 Using the NVIDIA GPU Operator #
Configuring and managing nodes with hardware resources can require multiple configurations for software components. These include drivers, container runtimes and libraries. To use NVIDIA GPUs in a Kubernetes cluster, you need to configure the NVIDIA GPU Operator. Because GPU is a special resource in the cluster, you need to install the following components to enable deployment of workloads for processing on the GPU.
NVIDIA drivers (to enable CUDA)
Kubernetes device plug-in
Container runtime
Other tools to provide capabilities such as monitoring or automatic node labeling
To ensure that the NVIDIA GPU Operator is installed correctly, the Kubernetes cluster must meet the following prerequisites:
All worker nodes must run the same operating system version to use the NVIDIA GPU Driver container.
Nodes must be configured with a container engine, such as Docker (CE/EE), containerd or Podman.
Nodes should be equipped with NVIDIA GPUs.
Nodes should have NVIDIA drivers installed.
2.2 Supported GPUs #
The NVIDIA GPU Operator is compatible with a range of NVIDIA GPUs. For a full list of supported GPUs, refer to NVIDIA GPU Operator Platform Support.
3 Application-specific requirements #
The SUSE AI stack consists of multiple applications. We recommend running each application on nodes that meet or exceed the corresponding hardware requirements.
3.1 Milvus hardware and software requirements #
This topic describes requirements for the Milvus application.
3.1.1 Hardware requirements #
3.1.1.1 Minimum requirements #
The following requirements are for basic Milvus deployment on a single node or a small scale.
- RAM
- CPU
- Disk space
- Networking
A minimum of 32 GB of RAM.
3.1.1.2 Recommended hardware for large-scale workloads #
The following requirements are for multi-node Milvus clusters or heavy workloads, such as large vector databases.
- RAM
- CPU
- Disk space
- Networking
A minimum of 64 GB of RAM per node.
3.1.1.3 CPU instruction set requirements #
The following CPU instruction sets are required for Milvus:
SSE4.2
AVX
AVX2
AVX-512
3.1.2 Software requirements #
Running Milvus requires specific versions of the following software:
- Kubernetes
SUSE-supported versions of SUSE Rancher Prime: RKE2 that use Kubernetes 1.18 or higher.
- Helm
The recommended version is 3.5.0 or later.
3.1.3 Additional considerations #
- Disk and storage
Storage type: SSDs or NVMe SSDs are highly recommended for fast read/write access to large datasets and high-performance vector retrieval.
Metadata and data storage: For large-scale deployments, ensure that metadata and vector data are stored on fast disks (SSD or NVMe).
- Network
For high-performance clusters, especially for large-scale deployments, ensure high-bandwidth network connectivity between nodes.
3.1.4 For more information #
For more detailed hardware recommendations, refer to the official Milvus and prerequisite Docker documentation.
3.2 Ollama hardware and software requirements #
The version of Ollama provided with SUSE AI is optimized for NVIDIA GPU hardware. This section guides you through the steps for configuring Ollama on an NVIDIA-enabled system, including necessary configurations for both the hardware and software.
Run Ollama on NVIDIA GPU nodes. Since Ollama is GPU-optimized, using the power of NVIDIA GPUs is essential for maximum performance. This ensures that the application runs efficiently and fully uses the hardware capabilities.
Assign applications to specific nodes. SUSE AI provides a mechanism to assign applications, such as Ollama, to specific nodes. For more details, refer to https://documentation.suse.com/suse-ai/1.0/html/AI-deployment-intro/index.html#ai-gpu-nodes-assigning.
3.2.1 Hardware requirements #
- NVIDIA GPU
- RAM
- Disk space
The recommended GPU models include Tesla, A100, V100, RTX 30 series, or other compatible NVIDIA GPUs.
Ensure that the CUDA Compute Capability of your GPU is compatible with the required version of Ollama.
3.2.2 Software requirements #
- NVIDIA Docker (nvidia-docker)
- CUDA Toolkit
- NVIDIA driver
You must install nvidia-docker (the NVIDIA Container Toolkit) to allow Docker containers to use the GPU. Refer to https://documentation.suse.com/suse-ai/1.0/html/NVIDIA-Operator-installation/index.html for more details.
3.3 Open WebUI hardware and software requirements #
While Open WebUI has no specific hardware dependencies beyond those of the underlying platform, consider the following guidelines for optimal performance.
Because Open WebUI shares most dependencies with Milvus and Ollama, follow the hardware requirements mentioned in Milvus requirements and Ollama requirements.
Stable network connection is essential, particularly if Open WebUI is integrated with other services or databases. Ensure sufficient bandwidth for Web traffic and API calls.
To interact with the Open WebUI interface, use standard Web browsers such as Google Chrome, Mozilla Firefox or Microsoft Edge.
4 Legal Notice #
Copyright© 2006–2024 SUSE LLC and contributors. All rights reserved.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”.
For SUSE trademarks, see https://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.
All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors, nor the translators shall be held liable for possible errors or the consequences thereof.