Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to SUSE Linux Enterprise High Availability Extension 12 SP5

22 Cluster Multi-device (Cluster MD)

Abstract

The cluster multi-device (Cluster MD) is a software based RAID storage solution for a cluster. Cluster MD provides the redundancy of RAID1 mirroring to the cluster. Currently, only RAID1 is supported now. This chapter shows you how to create and use Cluster MD.

22.1 Conceptual Overview

The Cluster MD provides support for use of RAID1 across a cluster environment. The disks or devices used by Cluster MD are accessed by each node. If one device of the Cluster MD fails, it can be replaced at runtime by another device and it is re-synced to provide the same amount of redundancy. The Cluster MD requires Corosync and Distributed Lock Manager (DLM) for co-ordination and messaging.

A Cluster MD device is not automatically started on boot like the rest of the regular MD devices. A clustered device needs to be started using resource agents to ensure the DLM resource has been started.

22.2 Creating a Clustered MD RAID Device

Requirements
  • A running cluster with pacemaker.

  • A resource agent for DLM (see Procedure 17.1, “Configuring a Base Group for DLM” on how to configure DLM).

  • At least two shared disk devices. You can use an additional device as a spare which will fail over automatically in case of device failure.

  • An installed package cluster-md-kmp-default.

  1. Make sure the DLM resource is up and running on every node of the cluster and check the resource status with the command:

    root # crm_resource -r dlm -W
  2. Create the Cluster MD device:

    • If you do not have an existing normal RAID device, create the Cluster MD device on the node running the DLM resource with the following command:

      root # mdadm --create /dev/md0 --bitmap=clustered \
         --metadata=1.2 --raid-devices=2 --level=mirror /dev/sda /dev/sdb

      As Cluster MD only works with version 1.2 of the metadata, it is recommended to specify the version using the --metadata option. For other useful options, refer to the man page of mdadm. Monitor the progress of the re-sync in /proc/mdstat.

    • If you already have an existing normal RAID, first clear the existing bitmap and then create the clustered bitmap:

      root # mdadm --grow /dev/mdX --bitmap=none
      root # mdadm --grow /dev/mdX --bitmap=clustered
    • Optionally, to create a Cluster MD device with a spare device for automatic failover, run the following command on one cluster node:

      root # mdadm --create /dev/md0 --bitmap=clustered --raid-devices=2 \
            --level=mirror --spare-devices=1 /dev/sda /dev/sdb /dev/sdc --metadata=1.2
  3. Get the UUID and the related md path:

    root # mdadm --detail --scan

    The UUID must match the UUID stored in the superblock. For details on the UUID, refer to the mdadm.conf man page.

  4. Open /etc/mdadm.conf and add the md device name and the devices associated with it. Use the UUID from the previous step:

    DEVICE /dev/sda /dev/sdb
    ARRAY /dev/md0 UUID=1d70f103:49740ef1:af2afce5:fcf6a489
  5. Open Csync2's configuration file /etc/csync2/csync2.cfg and add /etc/mdadm.conf:

    group ha_group
    {
       # ... list of files pruned ...
       include /etc/mdadm.conf
    }

22.3 Configuring a Resource Agent

Configure a CRM resource as follows:

  1. Create a Raid1 primitive:

    crm(live)configure# primitive raider Raid1 \
      params raidconf="/etc/mdadm.conf" raiddev=/dev/md0 \
      force_clones=true \
      op monitor timeout=20s interval=10 \
      op start timeout=20s interval=0 \
      op stop timeout=20s interval=0
  2. Add the raider resource to the base group for storage that you have created for DLM:

    crm(live)configure# modgroup g-storage add raider

    The add sub-command appends the new group member by default.

    If not already done, clone the g-storage group so that it runs on all nodes:

    crm(live)configure# clone cl-storage g-storage \
        meta interleave=true target-role=Started
  3. Review your changes with show.

  4. If everything seems correct, submit your changes with commit.

22.4 Adding a Device

To add a device to an existing, active Cluster MD device, first ensure that the device is visible on each node with the command cat /proc/mdstat. If the device is not visible, the command will fail.

Use the following command on one cluster node:

root # mdadm --manage /dev/md0 --add /dev/sdc

The behavior of the new device added depends on the state of the Cluster MD device:

  • If only one of the mirrored devices is active, the new device becomes the second device of the mirrored devices and a recovery is initiated.

  • If both devices of the Cluster MD device are active, the new added device becomes a spare device.

22.5 Re-adding a Temporarily Failed Device

Quite often the failures are transient and limited to a single node. If any of the nodes encounters a failure during an I/O operation, the device will be marked as failed for the entire cluster.

This could happen, for example, because of a cable failure on one of the nodes. After correcting the problem, you can re-add the device. Only the outdated parts will be synchronized as opposed to synchronizing the entire device by adding a new one.

To re-add the device, run the following command on one cluster node:

root # mdadm --manage /dev/md0 --re-add /dev/sdb

22.6 Removing a Device

Before removing a device at runtime for replacement, do the following:

  1. Make sure the device is failed by introspecting /proc/mdstat. Look for an (F) before the device.

  2. Run the following command on one cluster node to make a device fail:

    root # mdadm --manage /dev/md0 --fail /dev/sda
  3. Remove the failed device using the command on one cluster node:

    root # mdadm --manage /dev/md0 --remove /dev/sda
Print this page