This is unreleased documentation for SUSE® Storage 1.11 (Dev).

Talos Linux

Requirements

The following requirements must be met before installing SUSE Storage on a Talos Linux cluster.

System Extensions

Some SUSE Storage-dependent binary executables are not present in the default Talos root filesystem. To have access to these binaries, Talos offers system extension mechanism to extend the installation.

  • siderolabs/iscsi-tools: this extension enables iscsid daemon and iscsiadm to be available to all nodes for the Kubernetes persistent volumes operations.

  • siderolabs/util-linux-tools: this extension enables linux tool to be available to all nodes. For example, the fstrim binary is used for SUSE Storage volume trimming.

The most straightforward method is to patch the extensions onto existing Talos Linux nodes. Alternatively, you can use the Talos Linux Image Factory and select the desired extensions under System Extensions. This approach is especially useful when combined with Infrastructure as Code (IaC) tools.

customization:
  systemExtensions:
    officialExtensions:
      - siderolabs/iscsi-tools
      - siderolabs/util-linux-tools

For detailed instructions, see the Talos documentation on System Extensions and Boot Assets.

Pod Security

SUSE Storage requires pod security enforce: "privileged".

By default, Talos Linux applies a baseline pod security profile across namespaces, except for the kube-system namespace. This default setting restricts SUSE Storage’s ability to manage and access system resources. For more information, see Root and Privileged Permission and the Talos documentation on Pod Security.

From Talos Linux v1.10.x Onwards

Data Path Mounts

Since Talos Linux deprecated .machine.disks, it is recommended to use UserVolumeConfig to mount a disk for SUSE Storage. See What’s new in Talos v1.10 for more details.

Optionally, it is also recommended to create a VolumeConfig to specify the size of Talos system volumes. This is recommended, as it avoids the need to set defaultSettings.storageReservedPercentageForDefaultDisk.

For more disk configuration options, see the Talos documentation.

You must provide additional data path mounts to make them accessible to the Kubernetes kubelet container.

These mounts allow access to host directories and attach the volumes required by Longhorn components.

The default data path for Longhorn is /var/lib/longhorn. To use the configuration below in Talos, first set your default data path to /var/mnt/longhorn. The method to do this depends on your deployment method.

machine:
  kubelet:
    extraMounts:
      - destination: /var/mnt/longhorn
        type: bind
        source: /var/mnt/longhorn
        options:
          - bind
          - rshared
          - rw

Next, create a UserVolumeConfig to mount the disk for SUSE Storage. It would be automatically mounted to /var/mnt/longhorn on the configured node.

apiVersion: v1alpha1
kind: UserVolumeConfig
name: longhorn # name is used to identify the volume /var/mnt/<name>
provisioning:
  diskSelector:
    match: disk.transport == "nvme"
  grow: false
  maxSize: 1700GB

For detailed instructions on UserVolumeConfig and VolumeConfig, see the Talos documentation on Block configuration. Refer to the Talos API Reference for more configuration details.

Up to Talos Linux v1.9.x

Data Path Mounts

You must provide additional data path mounts to be accessible to the Kubernetes kubelet container.

These mounts are necessary to provide access to the host directories, and attach volumes required by Longhorn components.

The default data path for SUSE Storage is /var/lib/longhorn. In order to use the below configuration in Talos, we must first set our default data path to /var/mnt/longhorn. The method to do this will depend on your deployment method.

machine:
  kubelet:
    extraMounts:
      - destination: /var/lib/longhorn
        type: bind
        source: /var/lib/longhorn
        options:
          - bind
          - rshared
          - rw

For detailed steps, see the Talos documentation on Editing Machine Configuration.

V2 Data Engine

To use V2 volumes, all nodes must meet the V2 Data Engine prerequisites.

machine:
  sysctls:
    vm.nr_hugepages: "1024"
  kernel:
    modules:
      - name: nvme_tcp
      - name: vfio_pci
#     - name: uio_pci_generic

Talos Linux v1.7.x and earlier versions do not include the uio_pci_generic kernel module. If the system device supports vfio_pci, which is the preferred kernel module for SPDK application deployment, you are not required to install and enable the uio_pci_generic kernel driver. For more information, see System Configuration User Guide in the SPDK documentation.

You can use uio_pci_generic if vfio_pci is incompatible with your system or specific hardware. Future versions of Talos Linux are expected to include native support for uio_pci_generic. For more information, see Issue #9236.

Talos Linux Upgrades

From v1.8.x Onwards

The --preserve flag is no longer required. It is now automatically set for the talosctl upgrade command, as noted here.

Up to v1.8.x

When upgrading a Talos Linux node, always include the --preserve option in the command. This option explicitly tells Talos to keep ephemeral data intact.

Example:

talosctl upgrade --nodes 10.20.30.40 --image ghcr.io/siderolabs/installer:v1.7.6 --preserve

If the --preserve option is not included, Talos wipes /var/lib/longhorn, destroying all replicas stored on that node.

Recovering from an Upgraded Node without Preserving Data

If you were unable to include the --preserve option in the upgrade command, perform the following steps:

  1. On the SUSE Storage UI, go to the Nodes screen.

  2. Select the upgraded node, and then select Edit node and disks in the Operation menu.

  3. On the Edit Node and Disks screen, set Scheduling to Disable, delete the disk, and then click Save.

  4. Select the upgraded node again, and then select Edit node and disks in the Operation menu.

  5. On the Edit Node and Disks screen, add a disk and configure the following settings:

    • Path: Specify /var/lib/longhorn/.

    • Storage Reserved: Specify a value that matches your requirements. The default value is 30 Gi.

    • Scheduling: Select Enable.

  6. Click Save.

SUSE Storage synchronizes the replicas based on the configured settings.

References