Documentation survey

This is unreleased documentation for SUSE® Storage 1.10 (Dev).

Multiple Disks

SUSE Storage supports using more than one disk on the nodes to store the volume data.

By default, SUSE Storage stores volume data in the /var/lib/longhorn directory on the host. To use a different disk for storage, you can add a new disk and disable scheduling for the default directory. This approach provides the flexibility to manage storage based on your specific requirements.

Add a Disk

Before adding a disk to SUSE Storage, mount it to a directory on the Longhorn node’s host.

  1. Choose a Disk: Select the physical or virtual disk for Longhorn storage and format it with an extent-based filesystem (for example, ext4 or XFS).

  2. Mount the Disk: Mount the disk to a directory on the host, such as /mnt/example-disk. Ensure the directory is accessible and correctly configured.

After the disk is mounted, you can add it to SUSE Storage by using either the UI or the kubectl command-line tool.

Using the SUSE Storage UI

  1. Go to the Nodes tab, select a node, and choose Edit Disks from the dropdown menu.

  2. Add the disk’s mount path to the disk list.

Using kubectl

  1. Run kubectl edit node.longhorn.io <node-name> to modify the Longhorn node resource.

  2. Add the disk path to spec.disks. For example:

    ...
spec:
  ...
  disks:
    ...
    example-disk:
      allowScheduling: true
      diskDriver: ""
      diskType: filesystem
      evictionRequested: false
      path: /mnt/example-disk
      storageReserved: 0
      tags: []
...

  3. Save and exit the editor.

Once a disk is added:

  • SUSE Storage automatically detects the disk’s storage details, such as maximum and available capacity.

  • If the disk is suitable for storing volume data, SUSE Storage begins scheduling volumes to it.

  1. You cannot add a disk path that another SUSE Storage disk already uses.

  2. SUSE Storage uses the filesystem ID to detect duplicate mounts. Therefore, you cannot add a disk with the same filesystem ID as another disk on the same node.

    For more information, see issue #2477.

Root Disk Reservation

Optionally, you can use the Space Reserved field in the UI or spec.disks.<disk-name>.storageReserved to reserve a portion of disk space (in bytes) for other purposes. SUSE Storage will not use this reserved space for volume data.

To maintain node stability when compute resources (for example, memory or disk) are under pressure, kubelet requires some space to remain free. If these critical resources are exhausted, it can lead to node instability.

By default, SUSE Storage reserves 30% of the root disk space (/var/lib/longhorn) to prevent issues such as DiskPressure conditions from kubelet, especially after scheduling multiple volumes. The storage-reserved-percentage-for-default-disk setting controls this behavior.

Use an Alternative Path for a Disk on the Node

If you prefer to use a different path for a disk (rather than the original mount point), you can use mount --bind to create an alternative path. Do not use a symbolic link (ln -s), as symbolic links are not properly resolved inside Longhorn pods.

Ensure the alternative path is remounted after a node reboot, for example, by adding it to /etc/fstab.

Remove a Disk

Nodes and disks can be excluded from future scheduling. Note that any storage already scheduled on a node will not be automatically released when scheduling is disabled for that node.

To remove a disk:

  1. Disable scheduling for the disk.

  2. Ensure that no replicas or backing images remain on the disk, including any in an error state. For instructions on how to evict replicas from disabled disks, see Select Disks or Nodes for Eviction.

Once the disk is empty and scheduling is disabled, you can safely remove it from the node configuration.

Configuration

Two global settings affect volume scheduling:

  • StorageOverProvisioningPercentage defines the maximum total storage that can be scheduled on a disk, relative to its usable capacity. The formula is:

ScheduledStorage / (MaximumStorage - ReservedStorage)

By default, this setting is 100%.

For example, on a 200 GiB disk with 50 GiB reserved, SUSE Storage considers 150 GiB of usable space. With the default setting, SUSE Storage can schedule up to 150 GiB of volume data.

Because workloads typically do not consume the entire allocated volume size, and SUSE Storage uses sparse files to store data, increasing this setting is generally safe and can help optimize disk utilization.

  • StorageMinimalAvailablePercentage specifies the minimum percentage of free space that must remain on a disk to schedule new replicas. The formula is:

AvailableStorage / MaximumStorage

By default, this setting is 25%.

For example, for a 200 GiB disk with 50 GiB reserved, SUSE Storage stops scheduling new replicas if available space falls below 37.5 GiB (25% of 150 GiB). A new volume also will not be scheduled if its size would push available space below that limit.

This setting helps prevent disks from becoming too full, which could lead to scheduling failures or volume operation issues.

SUSE Storage currently cannot fully enforce the StorageMinimalAvailablePercentage limit in all scenarios because:

  1. Longhorn volumes might use more space than their requested size, especially when snapshots are taken.

  2. SUSE Storage allows over-provisioning by default.