Settings

The default engine image is the image used by the manager and can be changed only when the manager starts.

Every SUSE Storage release includes a new engine image. A green arrow will appear on the Longhorn volumes not using the default engine, indicating that the volume needs to be upgraded.

The default instance manager image is the image used by the manager and can be changed only when the manager starts.

The default backing image manager image is the image used by the manager and can be changed only when the manager starts.

SUSE Storage uses the support bundle manager image to generate support bundles.

A default image is provided during installation and upgrade, and you can also change this in the settings.

An example of the support bundle manager image:

Default value: longhornio/support-bundle-kit:v0.0.14

The current SUSE Storage version.

The latest available version of SUSE Storage. This is automatically updated by the Upgrade Checker.

The latest stable version of every minor release line. It is updated by the Upgrade Checker automatically.

Default value: block-if-contains-last-replica

Define the policy to use when a node with the last healthy replica of a volume is drained. Available options:

Each option has benefits and drawbacks. See Node Drain Policy Recommendations for help deciding which is most appropriate in your environment.

Default value: false

SUSE Storage will automatically detach volumes that are manually attached to the nodes which are cordoned. This prevent the draining process stuck by the PDB of instance-manager which still has running engine on the node.

Default value: true

SUSE Storage will generate system snapshot during replica rebuild, and if a user doesn’t setup a recurring snapshot schedule, all the system generated snapshots would be left in the replica, and user has to delete them manually, this setting allow SUSE Storage to automatically cleanup system generated snapshot before and after replica rebuild.

Default value: true

If enabled, when running a recurring backup job, SUSE Storage takes a new snapshot before creating the backup. SUSE Storage retains only the snapshot used by the last backup job even if the value of the retain parameter is not 1.

If disabled, this setting ensures that the retained snapshots directly correspond to the backups on the remote backup target.

Default value: true

If enabled, SUSE Storage will automatically delete the workload pod that is managed by a controller (e.g. deployment, statefulset, daemonset, etc.) when Longhorn volume is detached unexpectedly (e.g. during Kubernetes upgrade, Docker reboot, or network disconnect). By deleting the pod, its controller restarts the pod and Kubernetes handles volume reattachment and remount.

If disabled, SUSE Storage will not delete the workload pod that is managed by a controller. You will have to manually restart the pod to reattach and remount the volume.

Default value: true

If enabled, volumes will be automatically salvaged when all the replicas become faulty e.g. due to network disconnection. SUSE Storage will try to figure out which replica(s) are usable, then use them for the volume.

Default value: 0

This setting controls how SUSE Storage automatically upgrades volumes' engines to the new default engine image after upgrading Longhorn manager.

The value of this setting specifies the maximum number of engines per node that are allowed to upgrade to the default engine image at the same time.

If the value is 0, SUSE Storage will not automatically upgrade volumes' engines to default version.

Default value: 5

This setting controls how many volumes on a node can restore the backup concurrently.

SUSE Storage blocks the backup restore once the restoring volume count exceeds the limit.

Set the value to 0 to disable backup restore.

Default value: false

If no other disks exist, create the default disk automatically, only on nodes with the Kubernetes label node.longhorn.io/create-default-disk=true .

If disabled, the default disk will be created on all new nodes when the node is detected for the first time.

This option is useful if you want to scale the cluster but don’t want to use storage on the new nodes, or if you want to customize disks for Longhorn nodes.

Default value: longhorn.io/v1beta2

The current customer resource’s API version, e.g. longhorn.io/v1beta2. Set by manager automatically.

Default value: disabled

We say a Longhorn volume has data locality if there is a local replica of the volume on the same node as the pod which is using the volume. This setting specifies the default data locality when a volume is created from the SUSE Storage UI. For Kubernetes configuration, update the dataLocality in the StorageClass

The available modes are:

Default value: /var/lib/longhorn/

Default path to use for storing data on a host.

Can be used with Create Default Disk on Labeled Nodes option, to make SUSE Storage only use the nodes with specific storage mounted at, for example, /opt/longhorn when scaling the cluster.

Default value: longhorn-static

The storageClassName is for persistent volumes (PVs) and persistent volume claims (PVCs) when creating PV/PVC for an existing Longhorn volume. Notice that it’s unnecessary for users to create the related StorageClass object in Kubernetes since the StorageClass would only be used as matching labels for PVC bounding purposes. The "storageClassName" needs to be an existing StorageClass. Only the StorageClass named longhorn-static will be created if it does not exist. By default 'longhorn-static'.

Default value: {"v1":"3","v2":"3"}

The default number of replicas when creating the volume from SUSE Storage UI. For Kubernetes, update the numberOfReplicas in the StorageClass

The recommended way of choosing the default replica count is: if you have three or more nodes for storage, use 3; otherwise use 2. Using a single replica on a single node cluster is also OK, but the high availability functionality wouldn’t be available. You can still take snapshots or backups of the volume.

Default value: false

This flag is designed to prevent SUSE Storage from being accidentally uninstalled which will lead to data loss.

Default value: {"v1":"true"}

Allows engine controller and engine replica to disable revision counter file update for every data write. This improves the data path performance. See Revision Counter for details.

Default value: true

Upgrade Checker will check for a new SUSE Storage version periodically. When there is a new version available, it will notify the user in the SUSE Storage UI.

Default value: https://longhorn-upgrade-responder.rancher.io/v1/checkupgrade

The Upgrade Responder sends a notification whenever a new SUSE Storage version that you can upgrade to becomes available.

Default value: true

Enabling this setting will allow SUSE Storage to provide valuable usage metrics to https://metrics.longhorn.io/.

This information will help us gain insights how SUSE Storage is being used, which will ultimately contribute to future improvements.

Node Information collected from all cluster nodes includes:

Cluster Information collected from one of the cluster nodes includes:

The Upgrade Checker needs to be enabled to periodically send the collected data.

Default value: do-nothing

Defines the SUSE Storage action when a Volume is stuck with a StatefulSet or Deployment Pod on a node that is down.

The Kubernetes Secret name.

Default value: 600

When there is at least one failed replica volume in a degraded volume, this interval in seconds determines how long SUSE Storage will wait at most in order to reuse the existing data of the failed replicas rather than directly creating a new replica for this volume.

Default value: if-not-present

This setting defines the Image Pull Policy of Longhorn system managed pods, e.g. instance manager, engine image, CSI driver, etc.

Notice that the new Image Pull Policy will only apply after the system managed pods restart.

This setting definition is exactly the same as that of in Kubernetes. Here are the available options:

Default value: 60

This interval in minutes determines how long SUSE Storage will wait before cleaning up the backing image file when there is no replica in the disk using it.

Default value: 300

The interval in seconds determines how long SUSE Storage will wait before re-downloading the backing image file when all disk files of this backing image become failed or unknown.

Default value: 1

The default minimum number of backing image copies SUSE Storage maintains.

Default value: {"v1":"8","v2":"8"}

Number of seconds a V1 Data Engine waits for a replica to respond before marking it as failed. Values between 8 and 30 are allowed. This setting takes effect only when there are outstanding input or output requests.

This setting only applies to additional replicas. A V1 engine marks the last active replica as failed only after twice the configured number of seconds (timeout value x 2) have passed. This behavior is intended to balance volume responsiveness with volume availability.

The engine can quickly (after the configured timeout) ignore individual replicas that become unresponsive in favor of other available ones. This ensures future input or output will not be held up.

The engine waits on the last replica (until twice the configured timeout) to prevent unnecessarily crashing as a result of having no available backends.

Default value: 1

This setting specifies how many failed support bundles can exist in the cluster.

The retained failed support bundle is for analysis purposes and needs to clean up manually.

SUSE Storage blocks support bundle creation when reaching the upper bound of the limitation. You can set this value to 0 to have SUSE Storage automatically purge all failed support bundles.

Default value: 30

Number of minutes SUSE Storage allows for collection of node information and node logs for the support bundle.

If the collection process is not completed within the allotted time, SUSE Storage continues generating the support bundle without the uncollected node data.

Default value: {"v1":"true","v2":"true"}

The setting enables fast replica rebuilding feature. It relies on the checksums of snapshot disk files, so setting the snapshot-data-integrity to enable or fast-check is a prerequisite.

Default value: 30

The value in seconds specifies the timeout of the HTTP client to the replica’s file sync server used for replica rebuilding, volume cloning, snapshot cloning, etc.

Default value: false

Controls whether SUSE Storage automatically rebuilds degraded replicas while the volume is detached. This setting only takes effect if the volume-level setting is set to ignored or enabled.

Available options:

This setting allows SUSE Storage to automatically rebuild replicas for detached volumes when needed.

Default value: 86400

Number of seconds that SUSE Storage allows for the completion of replica rebuilding and snapshot cloning operations.

Default value: false

Enable improved ReadWriteMany volume HA by shortening the time it takes to recover from a node failure.

Default value: Log Level

Longhorn Manager uses the following log levels: Panic, Fatal, Error, Warn, Info, Debug, and Trace. The default log level is Info.

Default value: /var/lib/longhorn/logs/

This setting specifies the directory on the host where SUSE Storage stores log files for the instance manager pod. Currently, this is only used for instance manager pods in the v2 data engine.

Default value: {"v2":"Notice"}

Applies only to the V2 Data Engine. Specifies the log level for the Storage Performance Development Kit (SPDK) target daemon. Supported values: Error, Warning, Notice, Info, and Debug.

Default value: {"v2":""}

Applies only to the V2 Data Engine. Specifies the log flags for the Storage Performance Development Kit (SPDK) target daemon.

Default value: {"v2":"0"}

Applies only to the V2 Data Engine. Specifies the default write bandwidth limit, in megabytes per second (MB/s), for volume replica rebuilding.

Default value: {"v1":"fast-check","v2":"fast-check"}

This setting allows users to enable or disable snapshot hashing and data integrity checking. Available options are:

Default value: {"v1":"false","v2":"false"}

Hashing snapshot disk files impacts the performance of the system. The immediate snapshot hashing and checking can be disabled to minimize the impact after creating a snapshot.

Default value: {"v1":"0 0 */7 * *","v2":"0 0 */7 * *"}

Unix-cron string format. The setting specifies when SUSE Storage checks the data integrity of snapshot disk files.

Default value: 250

Maximum snapshot count for a volume. The value should be between 2 to 250.

Default value: {"v1":"false"}

This setting only applies to volumes with the Kubernetes volume mode Filesystem. When enabled, SUSE Storage freezes the volume’s filesystem immediately before creating a user-initiated snapshot. When disabled or when the Kubernetes volume mode is Block, SUSE Storage instead attempts a system sync before creating a user-initiated snapshot.

Snapshots created when this setting is enabled are more likely to be consistent because the filesystem is in a consistent state at the moment of creation. However, under very heavy input or output, freezing the filesystem may take a significant amount of time and may cause workload activity to pause.

When this setting is disabled, all data is flushed to disk just before the snapshot is created, but SUSE Storage cannot completely block write attempts during the brief interval between the system sync and snapshot creation. input or output is not paused during the system sync, so workloads likely do not notice that a snapshot is being created.

The default option for this setting is false because kernels with version v5.17 or earlier may not respond correctly when a volume crashes while a freeze is ongoing. This is not likely to happen but if it does, an affected kernel will not allow you to unmount the filesystem or stop processes using the filesystem without rebooting the node. Only enable this setting if you plan to use kernels with version 5.17 or later, and ext4 or XFS filesystems.

You can override this setting (using the field freezeFilesystemForSnapshot) for specific volumes through the SUSE Storage UI, a StorageClass, or direct changes to an existing volume. freezeFilesystemForSnapshot accepts the following values:

Default value: ignored

Example: replica-data;instance

This setting allows SUSE Storage to automatically delete orphan resources, which are typically Custom Resources (CRs) created by SUSE Storage to represent detected orphaned entities. The deletion of an orphan CR subsequently triggers the cleanup of the actual orphaned data or runtime instance it represents. However, orphan resources associated with nodes that are in a down or unknown state will not be automatically cleaned up by this setting.

You can list the resource types to be automatically deleted as a semicolon-separated string. Available types include:

Default value: 300 seconds

Number of seconds SUSE Storage waits before automatically deleting an orphaned custom resource (CR) and the actual orphaned data or runtime instance it represents.

Default value: false

If this setting is enabled, SUSE Storage automatically attaches the volume and takes snapshot or backup when it is the time to do recurring snapshot or backup.

Default value: 1440

The interval in minutes to keep the backup resource that was failed. Set to 0 to disable the auto-deletion.

Failed backups will be checked and cleaned up during backupstore polling which is controlled by Backupstore Poll Interval setting. Hence this value determines the minimal wait interval of the cleanup. And the actual cleanup interval is multiple of Backupstore Poll Interval. Disabling Backupstore Poll Interval also means to disable failed backup auto-deletion.

Default value: 1

This setting specifies how many failed backup or snapshot job histories should be retained.

History will not be retained if the value is 0.

Default value: 1

This setting specifies how many successful backup or snapshot job histories should be retained.

History will not be retained if the value is 0.

Default value: false

This setting allows restoring the recurring jobs of a backup volume from the backup target during a volume restoration if they do not exist on the cluster. This is also a volume-specific setting with the below options. Users can customize it for each volume to override the global setting.

Default value: ignored

Default value: lz4

This setting allows users to specify backup compression method.

Default value: 2

This setting controls how many worker threads per backup concurrently.

Default value: 2

This setting controls how many worker threads per restore concurrently.

Default value: 2

Specifies the default backup block size (in MiB), used when creating a new volume. Supported values are 2 or 16.

Default value: true

This setting allows user to create and attach a volume that doesn’t have all the replicas scheduled at the time of creation.

Default value: true

When this setting is checked, the Longhorn Manager will not schedule replicas on Kubernetes cordoned nodes.

When this setting is un-checked, the Longhorn Manager will schedule replicas on Kubernetes cordoned nodes.

Default value: false

When this setting is checked, the Longhorn Manager will allow scheduling on nodes with existing healthy replicas of the same volume.

When this setting is un-checked, Longhorn Manager will forbid scheduling on nodes with existing healthy replicas of the same volume.

Default value: true

When this setting is checked, the Longhorn Manager will allow scheduling new replicas of a volume to the nodes in the same zone as existing healthy replicas.

When this setting is un-checked, Longhorn Manager will forbid scheduling new replicas of a volume to the nodes in the same zone as existing healthy replicas.

Default value: true

When this setting is checked, the Longhorn Manager will allow scheduling new replicas of a volume to the same disks as existing healthy replicas.

When this setting is un-checked, Longhorn Manager will forbid scheduling new replicas of a volume to the same disks as existing healthy replicas.

Default value: disabled

Enable this setting automatically rebalances replicas when discovered an available node.

The available global options are:

SUSE Storage also supports customizing for individual volume. The setting can be specified in UI or with Kubernetes manifest volume.spec.replicaAutoBalance, this overrules the global setting. The available volume spec options are:

Default value: ignored

Default value: 90

Percentage of currently used storage that triggers automatic replica rebalancing.

When the threshold is reached, SUSE Storage automatically rebuilds replicas that are under disk pressure on another disk within the same node.

To disable this setting, set the value to 0.

This setting takes effect only when the following conditions are met:

This setting is not affected by Replica Node Level Soft Anti_Affinity, which can prevent SUSE Storage from rebuilding a replica on the same node. Regardless of that setting’s value, this setting still allows SUSE Storage to attempt replica rebuilding on a different disk on the same node for migration purposes.

Default value: 25

This setting controls the minimum free space that must remain on a disk, based on its Storage Maximum, before SUSE Storage can schedule a new replica.

By default, SUSE Storage ensures that at least 25% of the disk’s total capacity remains free. If adding a replica would reduce the available space below this limit, SUSE Storage temporarily marks the disk as unavailable for scheduling until sufficient space is freed.

This safeguard helps protect your disks from becoming too full, which can cause performance issues or storage failures. Maintaining a buffer of free space helps keep the system stable and ensures room for unexpected storage needs.

See Multiple Disks Support for details.

Default value: 100

The over-provisioning percentage defines the amount of storage that can be allocated relative to the hard drive’s capacity.

Adjusting this setting allows the Longhorn Manager to schedule new replicas on a disk as long as the combined size of all replicas remains within the permitted over-provisioning percentage of the usable disk space. The usable disk space is calculated as Storage Maximum minus Storage Reserved.

Default value: 30

The reserved percentage specifies the percentage of disk space that will not be allocated to the default disk on each new Longhorn node.

This setting only affects the default disk of a new adding node or nodes when installing SUSE Storage.

Default value: true

This setting allows replica of the volume without node selector to be scheduled on node with tags.

Default value: true

This setting allows replica of the volume without disk selector to be scheduled on disk with tags.

Default value: true

This setting lets you enable the V1 Data Engine.

Default value: false

This setting enable the V2 Data Engine, which is an experimental feature based on the Storage Performance Development Kit (SPDK). The V2 Data Engine is an experimental feature and should not be used in production environments. For more information, see V2 Data Engine (Experimental).

Default value: 5

This setting controls how many replicas on a node can be rebuilt simultaneously.

Typically, SUSE Storage can block the replica starting once the current rebuilding count on a node exceeds the limit. But when the value is 0, it means disabling the replica rebuilding.

Default value: 5

This setting controls how many backing image copies on a node can be replenished simultaneously.

Typically, SUSE Storage can block the backing image copy starting once the current replenishing count on a node exceeds the limit. But when the value is 0, it means disabling the backing image replenish.

Example: nodetype=storage:NoSchedule

If you want to dedicate nodes to just store SUSE Storage replicas and reject other general workloads, you can set tolerations for all SUSE Storage components and add taints to the nodes dedicated for storage.

Longhorn system contains user deployed components (e.g, Longhorn manager, Longhorn driver, Longhorn UI) and system managed components (e.g, instance manager, engine image, CSI driver, etc.) This setting only sets taint tolerations for system managed components. Depending on how you deployed Longhorn, you need to set taint tolerations for user deployed components in Helm chart or deployment YAML file.

To apply the modified toleration setting immediately, ensure that all Longhorn volumes are detached. When volumes are in use, SUSE Storage components are not restarted, and you need to reconfigure the settings after detaching the remaining volumes; otherwise, you can wait for the setting change to be reconciled in an hour. We recommend setting tolerations during SUSE Storage deployment because the Longhorn system cannot be operated during the update.

Multiple tolerations can be set here, and these tolerations are separated by semicolon. For example:

Default value: longhorn-critical

By default, SUSE Storage workloads run with the same priority as other pods in the cluster, meaning in cases of node pressure, such as a node running out of memory, SUSE Storage workloads will be at the same priority as other Pods for eviction.

The Priority Class setting will specify a Priority Class for the SUSE Storage workloads to run as. This can be used to set the priority for SUSE Storage workloads higher so that they will not be the first to be evicted when a node is under pressure.

Longhorn system contains user deployed components (e.g, Longhorn manager, Longhorn driver, Longhorn UI) and system managed components (e.g, instance manager, engine image, CSI driver, etc.).

Note that this setting only sets Priority Class for system managed components. Depending on how you deployed SUSE Storage, you need to set Priority Class for user deployed components in Helm chart or deployment YAML file.

See Priority Class for details.

Example: label-key1:label-value1;label-key2:label-value2

If you want to restrict SUSE Storage components to only run on a particular set of nodes, you can set node selector for all SUSE Storage components.

Longhorn system contains user deployed components (e.g, Longhorn manager, Longhorn driver, Longhorn UI) and system managed components (e.g, instance manager, engine image, CSI driver, etc.) You need to set node selector for both of them. This setting only sets node selector for system managed components. Follow the instruction at Node Selector to change node selector.

Default value: false

Setting the Kubernetes Cluster Autoscaler Enabled to true allows SUSE Storage to unblock the Kubernetes Cluster Autoscaler scaling.

See Kubernetes Cluster Autoscaler Support for details.

Example: kube-system/demo-192-168-0-0

The storage network uses Multus NetworkAttachmentDefinition to segregate the in-cluster data traffic from the default Kubernetes cluster network.

By default, the this setting applies only to RWO (Read-Write-Once) volumes. For RWX (Read-Write-Many) volumes, see Storage Network for RWX Volume Enabled setting.

See Storage Network for details.

Default value: false

This setting allows SUSE Storage to use the storage network for RWX volumes.

For more information, see Storage Network.

Example: false

This setting allows Longhorn filesystem trim feature to automatically mark the latest snapshot and its ancestors as removed and stops at the snapshot containing multiple children.

Since Longhorn filesystem trim feature can be applied to the volume head and the followed continuous removed or system snapshots only.

Notice that trying to trim a removed files from a valid snapshot will do nothing but the filesystem will discard this kind of in-memory trimmable file info. Later on if you mark the snapshot as removed and want to retry the trim, you may need to unmount and remount the filesystem so that the filesystem can recollect the trimmable file info.

See Trim Filesystem for details.

Default value: {"v1":"12","v2":"12"}

Percentage of the total allocatable CPU resources on each node to reserve for each instance manager pod. For example, a value of 10 means 10% of the total CPU on a node will be allocated to each instance manager pod on that node. This helps maintain engine and replica stability during periods of high node workload.

In order to prevent unexpected volume instance (engine/replica) crash as well as guarantee a relative acceptable IO performance, you can use the following formula to calculate a value for this setting:

The result of above calculation doesn’t mean that’s the maximum CPU resources the SUSE Storage workloads require. To fully exploit the Longhorn volume input or output performance, you can allocate or guarantee more CPU resources via this setting.

If it’s hard to estimate the usage now, you can leave it with the default value, which is 12%. Then you can tune it when there is no running workload using Longhorn volumes.

Default value: false

When set to true, temporarily prevent all attempts to purge volume snapshots.

SUSE Storage typically purges snapshots during replica rebuilding and user-initiated snapshot deletion. While purging, SUSE Storage coalesces unnecessary snapshots into their newer counterparts, freeing space consumed by historical data.

Allowing snapshot purging during normal operations is ideal, but this process temporarily consumes additional disk space. If insufficient disk space prevents the process from continuing, consider temporarily disabling purging while data is moved to other disks.

Default value: false

When set to true, the snapshot used by the backup will be automatically cleaned up when the backup is deleted.

Default value: 10 (in seconds)

The setting specifies the timeout for the instance manager pod liveness probe. The default value is 10 seconds.

Default value: {"v2":"true"}

This setting applies only to the V2 Data Engine. It enables hugepages for the Storage Performance Development Kit (SPDK) target daemon. If this setting is disabled, legacy memory is used. The allocation size for this memory is set through the Data Engine Memory Size setting.

Default value: {"v2":"2048"}

This setting applies only to the V2 Data Engine and specifies the memory size, in MiB, allocated to the Storage Performance Development Kit (SPDK) target daemon.

Default value: {"v2":"false"}

It applies only to the V2 Data Engine. It controls whether the Storage Performance Development Kit (SPDK) target daemon runs in interrupt mode or the default polling mode.

Default value: /var/lib/longhorn/logs/

Specifies the directory on the host where SUSE Storage stores log files for the instance manager pod. Currently, it is only used for instance manager pods in the v2 data engine.