Applies to SUSE AI Factory

3 Workflow #

To deploy SUSE AI Factory in an air-gapped environment, follow the procedures below.

In a connected environment, download all required Helm charts, container images and models to a local data store as described in Section 3.1, “Downloading artifacts”.
Transfer the data store to the air-gapped environment as described in Section 3.2, “Make downloaded data available to the air-gapped environment”.
Set up or use existing OCI-compliant local registries in the air-gapped environment to mirror the Helm charts and container images as described in Section 3.3, “Set up OCI-compliant registries in the air-gapped environment”.
Upload the Helm charts and container images from the data store to the local registries as described in Section 3.4, “Upload Helm charts and containers”.
Update the RKE2 private registry configuration so that the repositories are properly mirrored as described in RKE2 private registry configuration.
Create a model cache PVC in RKE2 as described in Section 3.5, “Create a PVC for the model cache”.
Copy the model cache from the local data store into the PVC as described in Section 3.6, “Transfer model cache to the PVC”.
Configure the charts with the model cache PVC as described in Section 3.7, “Configure Helm charts to use the model cache PVC”.

3.1 Downloading artifacts #

In a connected environment, download all the necessary artifacts to the local data store.

3.1.1 Helm charts and containers #

For Helm charts and container images, use any registry mirroring tool, such as regsync, Skopeo, crane, Helm, Docker, or Podman. Refer to the documentation of the specific tool for instructions on downloading the artifacts to the local data store.

For example, to download the Open WebUI Helm chart from the SUSE Application Collection OCI registry:

> helm pull --username <SUSE app username> --password <SUSE application password> oci://dp.apps.rancher.io/charts/open-webui --version 8.19.1

To pull the Open WebUI container image from the SUSE Application Collection OCI registry:

> docker login dp.apps.rancher.io
> docker pull dp.apps.rancher.io/containers/open-webui:0.6.41-14.80

Tip

For mirroring container images, we recommend using the SUSE-AI-get-images.sh and SUSE-AI-load-images.sh scripts from the SUSE AI air-gapped repository.

3.1.2 Models #

To cache NVIDIA NIM or Hugging Face models, refer to the NVIDIA NIM air-gap deployment documentation.

3.2 Make downloaded data available to the air-gapped environment #

Make the downloaded data available inside the air-gapped environment using copy, rsync, or another suitable data transfer method.

3.3 Set up OCI-compliant registries in the air-gapped environment #

Ensure one or more OCI-compliant private registries are available inside the air-gapped environment to mirror the artifacts. If your organization does not have an existing registry, you can use any commercially supported option, such as Harbor. Refer to the documentation of the specific registry you plan to deploy for more details. We strongly recommend configuring endpoint protection with production TLS certificates.

3.3.1 Configure SUSE AI Factory to access image registries? #

After air-gapped offline registries exist, configure SUSE AI Factory to access them by specifying endpoints for their repositories:

From the main left panel, click SUSE AI Factory › Settings.
In the Advanced › Registry Endpoints section, enter endpoint URLs for local SUSE Application Collection, SUSE Registry or NVIDIA registry.
Confirm with Apply in the bottom right corner.

Air-gapped registry endpoints image::aif-airgapped-endpoints.png[Screenshot showing how to configure registry endpoints for air-gapped deployment,width=100%]

3.4 Upload Helm charts and containers #

Use any registry mirroring tool, such as regsync, Skopeo, crane, Helm, Docker, or Podman, to upload the Helm charts and container images to your local private registries. Refer to the documentation of the specific tool for instructions on uploading the artifacts.

For example, to upload the Open WebUI Helm chart to your local registry:

> helm push \
  --username <local registry username> \
  --password <local registry password> \
  open-webui-8.19.1.tgz oci://my.local.registry.com/suse-app-co/charts

To tag and push the Open WebUI container image to your local registry:

> docker tag dp.apps.rancher.io/containers/open-webui:0.6.41-14.80 my.local.registry.com/suse-app-co/containers/open-webui:0.6.41-14.80
> docker push my.local.registry.com/suse-app-co/containers/open-webui:0.6.41-14.80

3.5 Create a PVC for the model cache #

Refer to the Kubernetes PVC documentation on how to create a PVC with ReadWriteMany access mode.

3.6 Transfer model cache to the PVC #

There are multiple ways to transfer data from a local store into the PVC. For example, you can create a simple job to mount both the local data store and the PVC as volumes, and then copy the data over from inside the pod.

3.7 Configure Helm charts to use the model cache PVC #

Configuring the NVIDIA Blueprint RAG to use an existing PVC is not currently supported. See https://github.com/NVIDIA-AI-Blueprints/rag/issues/693 for more details.

3.8 Configure RKE2 private registries #

Follow the RKE2 private registry configuration documentation to configure the registry mirrors. If your local private registries use prefix paths to namespace the artifacts, use rewrite rules. You must configure one entry per mirror. Perform the private registry configuration on all nodes in all clusters inside the air-gapped environment.