Applies to SUSE AI Factory

3 Workflow #

To deploy SUSE AI Factory in an air-gapped environment, follow the procedures below.

In a connected environment, download all required Helm charts, container images and models to a local data store as described in Section 3.1, “Downloading artifacts”.
Transfer the data store to the air-gapped environment as described in Section 3.2, “Make downloaded data available to the air-gapped environment”.
Set up or use existing OCI-compliant local registries in the air-gapped environment to mirror the Helm charts and container images as described in Section 3.3, “Set up OCI-compliant registries in the air-gapped environment”.
Upload the Helm charts and container images from the data store to the local registries as described in Section 3.4, “Upload Helm charts and containers”.
Update the RKE2 private registry configuration so that the repositories are properly mirrored as described in RKE2 private registry configuration.
Create a model cache PVC in RKE2 as described in Section 3.5, “Create a PVC for the model cache”.
Copy the model cache from the local data store into the PVC as described in Section 3.6, “Transfer model cache to the PVC”.
Configure the charts with the model cache PVC as described in Section 3.7, “Configure Helm charts to use the model cache PVC”.

3.1 Downloading artifacts #

In a connected environment, download all the necessary artifacts to the local data store.

3.1.1 Helm charts and containers #

For Helm charts and container images, use any registry mirroring tool, such as regsync, Skopeo, crane, Helm, Docker, or Podman. Refer to the documentation of the specific tool for instructions on downloading the artifacts to the local data store.

For example, to download the Open WebUI Helm chart from the SUSE Application Collection OCI registry:

> helm pull --username <SUSE app username> --password <SUSE application password> oci://dp.apps.rancher.io/charts/open-webui --version 8.19.1

To pull the Open WebUI container image from the SUSE Application Collection OCI registry:

> docker login dp.apps.rancher.io
> docker pull dp.apps.rancher.io/containers/open-webui:0.6.41-14.80

Tip

For mirroring container images, we recommend using the SUSE-AI-get-images.sh and SUSE-AI-load-images.sh scripts from the SUSE AI air-gapped repository.

3.1.2 Models #

To cache NVIDIA NIM or Hugging Face models, refer to the NVIDIA NIM air-gap deployment documentation.

3.2 Make downloaded data available to the air-gapped environment #

Make the downloaded data available inside the air-gapped environment using copy, rsync, or another suitable data transfer method.

3.3 Set up OCI-compliant registries in the air-gapped environment #

Ensure one or more OCI-compliant private registries are available inside the air-gapped environment to mirror the artifacts. If your organization does not have an existing registry, you can use any commercially supported option, such as Harbor. Refer to the documentation of the specific registry you plan to deploy for more details. We strongly recommend configuring endpoint protection with production TLS certificates.

3.3.1 How do I configure SUSE AI Factory to access mirrored chart repositories? #

After the private registries are available, configure SUSE AI Factory with the corresponding chart repository endpoints:

From the main left panel, select SUSE AI Factory › Settings.
In the SUSE Application Collection, SUSE Registry or NVIDIA section, enter the endpoint URL of the corresponding mirrored chart repository.
If the registry uses a certificate issued by a private CA, select the secret that holds the PEM-encoded CA certificate and the key within that secret.
Confirm with Apply in the bottom right corner.

Because blueprints reference a chart repository by its identity rather than by its URL, the bundled blueprints deploy unchanged after you point these endpoints at your private chart repositories.

Note: Container image registries

The endpoint fields configure Helm chart repositories only. They do not redirect container image pulls. In particular, the NVIDIA endpoint does not redirect container images that resolve from nvcr.io. Configure the RKE2 and containerd registry mirrors on every cluster node as described in Section 3.8, “Configure RKE2 private registries”.

Air-gapped chart repository endpoints image::aif-airgapped-endpoints.png[Screenshot showing how to configure chart repository endpoints for an air-gapped deployment,width=100%]

3.4 Upload Helm charts and containers #

Use any registry mirroring tool, such as regsync, Skopeo, crane, Helm, Docker, or Podman, to upload the Helm charts and container images to your local private registries. Refer to the documentation of the specific tool for instructions on uploading the artifacts.

For example, to upload the Open WebUI Helm chart to your local registry:

> helm push \
  --username <local registry username> \
  --password <local registry password> \
  open-webui-8.19.1.tgz oci://my.local.registry.com/suse-app-co/charts

To tag and push the Open WebUI container image to your local registry:

> docker tag dp.apps.rancher.io/containers/open-webui:0.6.41-14.80 my.local.registry.com/suse-app-co/containers/open-webui:0.6.41-14.80
> docker push my.local.registry.com/suse-app-co/containers/open-webui:0.6.41-14.80

3.5 Create a PVC for the model cache #

Refer to the Kubernetes PVC documentation on how to create a PVC with ReadWriteMany access mode.

3.6 Transfer model cache to the PVC #

There are multiple ways to transfer data from a local store into the PVC. For example, you can create a simple job to mount both the local data store and the PVC as volumes, and then copy the data over from inside the pod.

3.7 Configure Helm charts to use the model cache PVC #

Configuring the NVIDIA Blueprint RAG to use an existing PVC is not currently supported. See https://github.com/NVIDIA-AI-Blueprints/rag/issues/693 for more details.

3.8 Configure RKE2 private registries #

Follow the RKE2 private registry configuration documentation to configure the registry mirrors. If your local private registries use prefix paths to namespace the artifacts, use rewrite rules. You must configure one entry per mirror. Perform the private registry configuration on all nodes in all clusters inside the air-gapped environment.