This is unreleased documentation for SUSE® Storage 1.12 (Dev).

Create Volumes

This page explains how to create Kubernetes persistent storage resources of PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs) that correspond to Longhorn volumes. kubectl is used to dynamically provision V1 and V2 volumes for workloads using Longhorn storage classes.

For help creating volumes from the SUSE Storage UI, see the Creating Longhorn Volumes with the SUSE Storage UI section.

This section assumes that you understand how Kubernetes persistent storage works. For more information, see the Kubernetes documentation.

Access modes

SUSE Storage supports the following Kubernetes PersistentVolume access modes:

  • ReadWriteOnce (RWO): The volume can be mounted as read-write by a single node. Multiple pods on the same node can access the volume. This is the default and most common access mode.

  • ReadWriteOncePod (RWOP): The volume can be mounted as read-write by a single pod in the cluster. This mode provides the strongest isolation, ensuring that only one pod can access the volume at any time. It is recommended for stateful workloads that require single-writer access.

  • ReadWriteMany (RWX): The volume can be mounted as read-write by many nodes at the same time, enabling shared access across multiple pods. For more information, see ReadWriteMany (RWX) volumes.

ReadOnlyMany (ROX) is not supported. For read-only access from multiple pods, use ReadWriteMany with read-only mount options in the pod specification.

Create Longhorn Volumes

Creating V1 Longhorn Volumes with kubectl

Before creating a V1 volume, ensure that Longhorn has at least one available file system-type disk. V1 volumes are scheduled only to file system-type disks. For more information, see Add a Filesystem-Type Disk.

First, you need to create a Longhorn StorageClass. The Longhorn StorageClass contains the parameters to provision PVs.

Next, a PersistentVolumeClaim is created that references the StorageClass. Finally, the PersistentVolumeClaim is mounted as a volume within a Pod.

When the Pod is deployed, the Kubernetes master checks the PersistentVolumeClaim to make sure the resource request can be fulfilled. If storage is available, the Kubernetes master creates the Longhorn volume and bind it to the Pod.

  1. Use following command to create a StorageClass called longhorn:

    kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/v1.12.0/examples/storageclass.yaml

    The following example StorageClass is created:

    kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: longhorn
provisioner: driver.longhorn.io
allowVolumeExpansion: true
parameters:
  numberOfReplicas: "3"
  staleReplicaTimeout: "2880" # 48 hours in minutes
  fromBackup: ""
  fsType: "ext4"
#  backupTargetName: "default"
#  mkfsParams: "-I 256 -b 4096 -O ^metadata_csum,^64bit"
#  diskSelector: "ssd,fast"
#  nodeSelector: "storage,fast"
#  recurringJobSelector: '[
#   {
#     "name":"snap",
#     "isGroup":true,
#   },
#   {
#     "name":"backup",
#     "isGroup":false,
#   }
#  ]'

    The parameter mkfsParams can be used to specify file system format options for each StorageClass.

    The parameter backupTargetName can be used to specify the backup target. The name of the default backup target (default) is used if backupTargetName is not specified.

    Parameters may be omitted from the StorageClass specification. When the StorageClass is used to create a PV and a volume, parameters that are not specified are generally set using a default value taken from the global settings. For the full list of global settings, see StorageClass Parameters and Settings.

  2. Create a Pod that uses Longhorn volumes by running this command:

    kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/v1.12.0/examples/pod_with_pvc.yaml

    A Pod named volume-test is launched, along with a PersistentVolumeClaim named longhorn-volv-pvc. The PersistentVolumeClaim references the Longhorn StorageClass:

    apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: longhorn-volv-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: longhorn
  resources:
    requests:
      storage: 2Gi

    The PersistentVolumeClaim is mounted in the Pod as a volume:

    apiVersion: v1
  kind: Pod
  metadata:
    name: volume-test
    namespace: default
  spec:
    containers:
    - name: volume-test
      image: nginx:stable-alpine
      imagePullPolicy: IfNotPresent
      volumeMounts:
      - name: volv
        mountPath: /data
      ports:
      - containerPort: 80
    volumes:
    - name: volv
      persistentVolumeClaim:
        claimName: longhorn-volv-pvc

Creating V2 Longhorn Volumes with kubectl

Before creating a V2 volume, ensure that the V2 Data Engine is enabled and Longhorn has available block-type disks. V2 volumes are scheduled only to block-type disks. For more information, see V2 Data Engine Guide and Add a Block-Type Disk.

  1. Use the following command to create a StorageClass called longhorn-v2-data-engine:

    kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/v{patch-version}/examples/v2/storageclass.yaml

    The following example StorageClass is created:

    kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: longhorn-v2-data-engine
provisioner: driver.longhorn.io
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
  numberOfReplicas: "3"
  staleReplicaTimeout: "2880"
  fsType: "ext4"
  dataEngine: "v2"

    The dataEngine parameter must be set to v2 so Longhorn provisions a V2 volume. See Storage Class Parameters for the list of supported parameters.

  2. Create a Pod that uses a V2 Longhorn volume by running this command:

    kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/v{patch-version}/examples/v2/pod_with_pvc.yaml

    A Pod named volume-test is launched, along with a PersistentVolumeClaim named longhorn-volv-pvc. The PersistentVolumeClaim references the V2 StorageClass:

    apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: longhorn-volv-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: longhorn-v2-data-engine
  resources:
    requests:
      storage: 2Gi

    The PersistentVolumeClaim is mounted in the Pod as a volume:

    apiVersion: v1
kind: Pod
metadata:
  name: volume-test
  namespace: default
spec:
  restartPolicy: Always
  containers:
  - name: volume-test
    image: nginx:stable-alpine
    imagePullPolicy: IfNotPresent
    livenessProbe:
      exec:
        command:
          - ls
          - /data/lost+found
      initialDelaySeconds: 5
      periodSeconds: 5
    volumeMounts:
    - name: volv
      mountPath: /data
    persistentVolumeClaim:
      claimName: longhorn-volv-pvc

More examples are available examples section.

Binding Workloads to PVs without a Kubernetes StorageClass

It is possible to use a Longhorn StorageClass to bind a workload to a PV without creating a StorageClass object in Kubernetes.

Since the StorageClass is also a field used to match a PVC with a PV, which does not have to be created by a Provisioner, you can create a PV manually with a custom StorageClass name, then create a PVC asking for the same StorageClass name.

When a PVC requests a StorageClass that does not exist as a Kubernetes resource, Kubernetes tries to bind your PVC to a PV with the same StorageClass name. The StorageClass is used like a label to find the matching PV, and only existing PVs labeled with the StorageClass name are used.

If the PVC names a StorageClass, Kubernetes:

  1. Look for an existing PV that has the label matching the StorageClass

  2. Look for an existing StorageClass Kubernetes resource. If the StorageClass exists, it is used to create a PV.

Creating Longhorn Volumes with the UI

Since the Longhorn volume already exists while creating PV or PVC, a StorageClass is not needed for dynamically provisioning Longhorn volume. However, the field storageClassName should be set in PVC or PV, to be used for PVC binding purposes. It is not necessary for users to create the related StorageClass object.

By default the StorageClass for Longhorn created PV or PVC is longhorn-static. Users can modify it in Setting - General - Default Longhorn Static StorageClass Name as they need.

Users need to manually delete PVC and PV created by SUSE Storage.

PV/PVC Creation for Existing Longhorn Volume

Now users can create PV or PVC via our Longhorn UI for the existing Longhorn volumes.

Only detached volume can be used by a newly created pod.

The Failure of the Longhorn Volume Creation

Creating a Longhorn volume can fail for different reasons. The issues are categorized into:

  • insufficient storage

  • disk not found

  • disks are unavailable

  • tags not fulfilled

  • node not found

  • nodes are unavailable

  • none of the node candidates contains a ready engine image

  • hard affinity cannot be satisfied

  • replica scheduling failed

  • unused failed replica is not supported

  • replica already scheduled

  • longhorn client operation failed

  • incompatible volume size

The failure results in the workload failing to use the provisioned PV and showing a warning message

# kubectl describe pod workload-test

Events:
  Type     Reason              Age                From                     Message
  ----     ------              ----               ----                     -------
  Warning  FailedAttachVolume  14s (x8 over 82s)  attachdetach-controller  AttachVolume.Attach
  failed for volume "pvc-e130e369-274d-472d-98d1-f6074d2725e8" : rpc error: code = Aborted
  desc = volume pvc-e130e369-274d-472d-98d1-f6074d2725e8 is not ready for workloads

To help users understand the error causes, SUSE Storage summarizes them in the PV annotation, longhorn.io/volume-scheduling-error. Failures are combined in this annotation and separated by a semicolon, for example, longhorn.io/volume-scheduling-error: insufficient storage;disks are unavailable. The annotation can be checked by using kubectl describe pv <pvc name>.

# kubectl describe pv pvc-e130e369-274d-472d-98d1-f6074d2725e8
Name:            pvc-e130e369-274d-472d-98d1-f6074d2725e8
Labels:          <none>
Annotations:     longhorn.io/volume-scheduling-error: insufficient storage
                 pv.kubernetes.io/provisioned-by: driver.longhorn.io

...