19 Cluster Logical Volume Manager (cLVM) #

Start a shell and log in as root.

Check the current configuration of the cluster resources:

# crm configure show

If you have already configured a DLM resource (and a corresponding base group and base clone), continue with Procedure 19.2, “Configuring DLM, CLVM, and STONITH”.

Otherwise, configure a DLM resource and a corresponding base group and base clone as described in Procedure 15.1, “Configuring a Base Group for DLM”.

Leave the crm live configuration with exit.

Create a cluster with at least two nodes as described in Installation and Setup Quick Start.

Configure your cluster to run dlm, clvmd, and STONITH:

# crm configure
crm(live)configure# primitive clvmd ocf:heartbeat:clvm \
        params with_cmirrord=1 \
        op stop interval=0 timeout=100 \
	       op start interval=0 timeout=90 \
	       op monitor interval=20 timeout=20
crm(live)configure# primitive dlm ocf:pacemaker:controld \
        op start timeout="90" \
        op stop timeout="100" \
        op monitor interval="60" timeout="60"
crm(live)configure# primitive sbd_stonith stonith:external/sbd \
        params pcmk_delay_max=30
crm(live)configure# group g-storage dlm clvmd
crm(live)configure# clone cl-storage g-storage \
        meta interleave="true" ordered=true

Leave crmsh with exit and commit your changes.

Create a clustered volume group (VG):

# pvcreate /dev/sda /dev/sdb
# vgcreate -cy vg1 /dev/sda /dev/sdb

Create a mirrored-log logical volume (LV) in your cluster:

# lvcreate -n lv1 -m1 -l10%VG vg1 --mirrorlog mirrored

Use lvs to show the progress. If the percentage number has reached 100%, the mirrored disk is successfully synchronized.

To test the clustered volume /dev/vg1/lv1, use the following steps:

1. Read or write to /dev/vg1/lv1.

2. Deactivate your LV with lvchange -an.

3. Activate your LV with lvchange -ay.

4. Use lvconvert to convert a mirrored log to a disk log.

Create a mirrored-log LV in another cluster VG. This is a different volume group from the previous one.

Create a volume group (VG):

# pvcreate /dev/sda /dev/sdb /dev/sdc /dev/sdd
  Physical volume "/dev/sda" successfully created
  Physical volume "/dev/sdb" successfully created
  Physical volume "/dev/sdc" successfully created
  Physical volume "/dev/sdd" successfully created
# vgcreate vgtest /dev/sda /dev/sdb /dev/sdc /dev/sdd
  Clustered volume group "vgtest" successfully created

Open the file /etc/lvm/lvm.conf and go to the section allocation. Set the following line and save the file:

mirror_logs_require_separate_pvs = 1

Add your tags to your PVs:

# pvchange --addtag @a /dev/sda /dev/sdb
# pvchange --addtag @b /dev/sdc /dev/sdd

A tag is an unordered keyword or term assigned to the metadata of a storage object. Tagging allows you to classify collections of LVM2 storage objects in ways that you find useful by attaching an unordered list of tags to their metadata.

List your tags:

# pvs -o pv_name,vg_name,pv_tags /dev/sd{a,b,c,d}

You should receive this output:

PV        VG   PV Tags
/dev/sda  vgtest   a
/dev/sdb  vgtest   a
/dev/sdc  vgtest   b
/dev/sdd  vgtest   b

Run YaST and click Network Services › iSCSI LIO Target to start the iSCSI Server module.

If you want to start the iSCSI target whenever your computer is booted, choose When Booting, otherwise choose Manually.

If you have a firewall running, enable Open Port in Firewall.

Switch to the Global tab. If you need authentication enable incoming or outgoing authentication or both. In this example, we select No Authentication.

Add a new iSCSI target:

1. Switch to the Targets tab.

2. Click Add.

3. Enter a target name. The name needs to be formatted like this:

   iqn.DATE.DOMAIN

   For more information about the format, refer to Section 3.2.6.3.1. Type "iqn." (iSCSI Qualified Name) at http://www.ietf.org/rfc/rfc3720.txt.

4. If you want a more descriptive name, you can change it as long as your identifier is unique for your different targets.

5. Click Add.

6. Enter the device name in Path and use a Scsiid.

7. Click Next twice.

Confirm the warning box with Yes.

Open the configuration file /etc/iscsi/iscsid.conf and change the parameter node.startup to automatic.

Run YaST and click Network Services › iSCSI Initiator.

If you want to start the iSCSI initiator whenever your computer is booted, choose When Booting, otherwise set Manually.

Change to the Discovery tab and click the Discovery button.

Add your IP address and your port of your iSCSI target (see Procedure 19.4, “Configuring iSCSI Targets (SAN)”). Normally, you can leave the port as it is and use the default value.

If you use authentication, insert the incoming and outgoing user name and password, otherwise activate No Authentication.

Select Next. The found connections are displayed in the list.

Proceed with Finish.

Open a shell, log in as root.

Test if the iSCSI initiator has been started successfully:

# iscsiadm -m discovery -t st -p 192.168.3.100
192.168.3.100:3260,1 iqn.2010-03.de.jupiter:san1

Establish a session:

# iscsiadm -m node -l -p 192.168.3.100 -T iqn.2010-03.de.jupiter:san1
Logging in to [iface: default, target: iqn.2010-03.de.jupiter:san1, portal: 192.168.3.100,3260]
Login to [iface: default, target: iqn.2010-03.de.jupiter:san1, portal: 192.168.3.100,3260]: successful

See the device names with lsscsi:

...
[4:0:0:2]    disk    IET      ...     0     /dev/sdd
[5:0:0:1]    disk    IET      ...     0     /dev/sde

Look for entries with IET in their third column. In this case, the devices are /dev/sdd and /dev/sde.

Open a root shell on one of the nodes you have run the iSCSI initiator from Procedure 19.5, “Configuring iSCSI Initiators”.

Prepare the physical volume for LVM2 with the command pvcreate on the disks /dev/sdd and /dev/sde, using their stable device names (for example, in /dev/disk/by-id/):

# pvcreate /dev/disk/by-id/DEVICE_ID1
# pvcreate /dev/disk/by-id/DEVICE_ID2

Create the cluster-aware volume group on both disks:

# vgcreate --clustered y clustervg /dev/disk/by-id/DEVICE_ID1 /dev/disk/by-id/DEVICE_ID2

Create logical volumes as needed:

# lvcreate -m1 --name clusterlv --size 500M clustervg

Check the physical volume with pvdisplay:

  --- Physical volume ---
      PV Name               /dev/sdd
      VG Name               clustervg
      PV Size               509,88 MB / not usable 1,88 MB
      Allocatable           yes
      PE Size (KByte)       4096
      Total PE              127
      Free PE               127
      Allocated PE          0
      PV UUID               52okH4-nv3z-2AUL-GhAN-8DAZ-GMtU-Xrn9Kh

      --- Physical volume ---
      PV Name               /dev/sde
      VG Name               clustervg
      PV Size               509,84 MB / not usable 1,84 MB
      Allocatable           yes
      PE Size (KByte)       4096
      Total PE              127
      Free PE               127
      Allocated PE          0
      PV UUID               Ouj3Xm-AI58-lxB1-mWm2-xn51-agM2-0UuHFC

Check the volume group with vgdisplay:

  --- Volume group ---
      VG Name               clustervg
      System ID
      Format                lvm2
      Metadata Areas        2
      Metadata Sequence No  1
      VG Access             read/write
      VG Status             resizable
      Clustered             yes
      Shared                no
      MAX LV                0
      Cur LV                0
      Open LV               0
      Max PV                0
      Cur PV                2
      Act PV                2
      VG Size               1016,00 MB
      PE Size               4,00 MB
      Total PE              254
      Alloc PE / Size       0 / 0
      Free  PE / Size       254 / 1016,00 MB
      VG UUID               UCyWw8-2jqV-enuT-KH4d-NXQI-JhH3-J24anD

Create a primary/primary DRBD resource:

1. First, set up a DRBD device as primary/secondary as described in Procedure 18.2, “Manually Configuring DRBD”. Make sure the disk state is up-to-date on both nodes. Check this with drbdadm status.

2. Add the following options to your configuration file (usually something like /etc/drbd.d/r0.res):

   resource r0 {
     net {
        allow-two-primaries;
     }
     ...
   }

3. Copy the changed configuration file to the other node, for example:

   # scp /etc/drbd.d/r0.res venus:/etc/drbd.d/

4. Run the following commands on both nodes:

   # drbdadm disconnect r0
   # drbdadm connect r0
   # drbdadm primary r0

5. Check the status of your nodes:

   # drbdadm status r0

Include the clvmd resource as a clone in the pacemaker configuration, and make it depend on the DLM clone resource. See Procedure 19.1, “Creating a DLM Resource” for detailed instructions. Before proceeding, confirm that these resources have started successfully on your cluster. You may use crm status or the Web interface to check the running services.

Prepare the physical volume for LVM2 with the command pvcreate. For example, on the device /dev/drbd_r0 the command would look like this:

# pvcreate /dev/drbd_r0

Create a cluster-aware volume group:

# vgcreate --clustered y myclusterfs /dev/drbd_r0

Create logical volumes as needed. You may probably want to change the size of the logical volume. For example, create a 4 GB logical volume with the following command:

# lvcreate -m1 --name testlv -L 4G myclusterfs

The logical volumes within the VG are now available as file system mounts or raw usage. Ensure that services using them have proper dependencies to collocate them with and order them after the VG has been activated.