4 Ollama #

Ollama (https://ollama.ai) is a tool for running and managing language models locally on your computer. It offers a simple interface to download, run and interact with models without relying on cloud resources.

4.1 How do I get the values list for Ollama Helm chart? #

Before deploying Ollama, it is important to know more about the supported configurations.

(Optional) To get the values for a specific Ollama chart version, log in to SUSE Application Collection and identify the version. To prevent the password from being stored in the shell history, save it to the password.txt file.
```
> skopeo login dp.apps.rancher.io \
  --username <APPCO-USERNAME> \
  --password-stdin < password.txt
> skopeo list-tags docker://dp.apps.rancher.io/charts/ollama
```
Log in to SUSE Application Collection where Ollama chart is stored. To prevent the password from being stored in the shell history, save it to the password.txt file.
```
> helm registry login dp.apps.rancher.io \
  --username <APPCO-USERNAME> \
  --password-stdin < appco-password.txt
```
Display all values of the Ollama Helm chart. Omit --version to display the latest chart.
```
> helm show values oci://dp.apps.rancher.io/charts/ollama \
  --version <VERSION>
```

4.2 How do I install Ollama? #

You can install Ollama either using Rancher Prime or on the command line with Helm.

4.2.1 Installing with Rancher Prime #

Refer to AI application installation for a general installation procedure.

4.2.2 Installing with Helm #

Log in to SUSE Application Collection. To prevent the password from being stored in the shell history, save it to the password.txt file.
```
> helm registry login dp.apps.rancher.io \
  --username=<APPCO-USERNAME> \
  --password-stdin < password.txt
```

Create a namespace for Ollama.

> kubectl create namespace ollama-system \
  --dry-run=client \
  -o yaml | kubectl apply -f -

(Optional) If you have not created image pull secrets for SUSE Application Collection, create them now. To prevent the password from being stored in the shell history, save it to the password.txt file.

> kubectl create secret docker-registry application-collection \
  --docker-server=dp.apps.rancher.io \
  --docker-username=<APPCO-USERNAME> \
  --docker-password="$(< password.txt)" \
  -n ollama-system \
  --dry-run=client -o yaml | kubectl apply -f -

(Optional) To override default installation values, list them in a values.yaml file. For a complete list of Ollama chart values, refer to Section 4.1, “How do I get the values list for Ollama Helm chart?”.

Install Ollama. To keep the default installation values, omit -f values.yaml.

> helm install ollama \
  oci://dp.apps.rancher.io/charts/ollama \
  -n ollama-system \
  -f values.yaml

4.3 What custom values examples are available for Ollama? #

To override the default values during the Helm chart installation or update, you can create a YAML file with custom values. Then, apply these values by specifying the path to the custom values file with the -f option of the helm command.

Example 4.1: Enabling or disabling the GPU section #

Ollama can run optimized for NVIDIA and AMD GPUs if the following conditions are fulfilled:

The GPU driver and operator are installed as described in Installing GPU Operators on RKE2.
The workloads are set to run on GPU-enabled nodes as described in Assigning GPU resources.

If you do not want Ollama to use the GPU hardware, remove the gpu section from the custom values file or disable it:

 ollama:
  [...]
  gpu:
    enabled: false
    type: 'nvidia'
    number: 1

Example 4.2: Basic custom values file with GPU and two models pulled at startup #

global:
  imagePullSecrets:
  - application-collection
ingress:
  enabled: false
defaultModel: "gemma:2b"
ollama:
  models:
    pull:
      - "gemma:2b"
      - "llama3.1"
    run:
      - "gemma:2b"
      - "llama3.1"
  gpu:
    enabled: true
    type: 'nvidia'
    number: 1
    nvidiaResource: "nvidia.com/gpu"
persistentVolume: 1
  enabled: true
  storageClass: local-path 2

1	Without the `persistentVolume` option enabled, changes made to Ollama, such as downloading other LLM, are lost when the container is restarted.
2	Use `local-path` storage only for testing purposes. For production use, we recommend using a storage solution suitable for persistent storage, such as SUSE Storage.

Example 4.3: Basic custom values file with Ingress and no GPU #

ollama:
  models:
    pull:
      - llama2
    run:
      - llama2
  persistentVolume:
    enabled: true
    storageClass: local-path 1
ingress:
  enabled: true
  hosts:
  - host: <OLLAMA_API_URL>
    paths:
      - path: /
        pathType: Prefix

1	Use `local-path` storage (requires installing the corresponding provisioner) only for testing purposes. For production use, we recommend using a storage solution suitable for persistent storage, such as SUSE Storage.

4.4 How do I upgrade Ollama? #

You can upgrade Ollama either using Rancher Prime or on the command line with Helm.

4.4.1 Upgrading with Rancher Prime #

Refer to AI application upgrade for a general upgrade procedure.

4.4.2 Upgrading with Helm #

Log in to SUSE Application Collection. To prevent the password from being stored in the shell history, save it to the password.txt file.
```
> helm registry login dp.apps.rancher.io \
  --username=<APPCO-USERNAME> \
  --password-stdin < password.txt
```

Upgrade Ollama.

> helm upgrade ollama \
  oci://dp.apps.rancher.io/charts/ollama \
  -n ollama-system \
  --version <VERSION_NUMBER> \ 1
  --wait --timeout 15m \
  --force-conflicts

1	If you omit `--version`, Ollama gets upgraded to the latest available version.

4.5 How do I remove Ollama? #

You can remove Ollama either using Rancher Prime or on the command line with Helm.

4.5.1 Removal using Rancher Prime #

Refer to AI application removal for a general removal procedure.

4.5.2 Removal using Helm #

To remove Ollama using Helm, use the following command:

> helm uninstall ollama -n ollama-system