Configuring HA Cluster Resources

1 What are cluster resources? #

In a High Availability cluster, the applications and services that need to be highly available are called resources. Cluster resources can include Web sites, e-mail servers, databases, file systems, virtual machines or any other applications or services that need to be available to users at all times. You can start, stop, monitor and move resources as needed. You can also specify whether resources should run together on the same node, or start and stop in sequential order. If a node fails, the resources running on it fail over (move) to another node instead of being lost.

In SUSE Linux Enterprise High Availability, the cluster resource manager (CRM) is Pacemaker, which manages and coordinates all cluster services. Pacemaker uses resource agents (RAs) to start, stop and monitor resources. A resource agent abstracts the resource and presents its status to the cluster. This means that Pacemaker only interacts with the resource agent, not with the actual application or service. SUSE Linux Enterprise High Availability supports many different resource agents that are designed to manage specific types of resources.

1.1 Primitive resources #

Primitive resources represent the applications or services that need to be highly available. Configuring a primitive specifies the resource agent and any other information required for the cluster to manage a resource. To see the supported resource agent classes, run crm ra classes. To see the resource agents available within a class, run crm ra list CLASS.

Primitive resources can have the following settings:

Properties: Resource properties are the basic details required by the cluster: a unique name for the resource, and the resource agent that manages the application or service.
Parameters: Parameters are specific to each resource agent and determine the details and behavior of an application or service. To show all the parameters available for a resource agent, run crm ra info RESOURCE_AGENT.
Operations: Operations are actions that the cluster can perform on a resource, such as start, stop and monitor. To show the supported operations for a resource agent, run crm ra info RESOURCE_AGENT and scroll to the bottom of the output.
Meta attributes: Meta attributes change how the cluster treats a resource. To show all the meta attributes available for primitives, run crm_resource --list-options primitive.
Utilization attributes: If the cluster nodes have utilization attributes configured, you can add these attributes to cluster resources to help with load balancing. For more information, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/utilization.html.

1.2 Resource groups #

Cluster resources might depend on other resources, such as a Web server that requires an IP address and a file system. You can combine these resources into a resource group. Groups contain multiple primitives that need to be located together, started sequentially and stopped in the reverse order.

This diagram shows a box representing a node. The box contains four individual resources and a smaller box containing three more resources. The smaller box represents a resource group. A zoomed-in view of the resource group shows that the resources are an IP address, an XFS file system and an Apache Web server. The resources are numbered to show the start order in one direction and the reversed stop order in the other direction.

Figure 1: Resource group #

Groups have the following properties:

Contents: Groups can only contain primitive resources, not clones or other groups.
Starting and stopping: Resources start in the order they appear and stop in the reverse order.
Location: Resources in the group must all run on the same node.
Dependency: If one of the resources can't run anywhere, then the resources that appear after it in the group can't run anywhere either.
Meta attributes: The meta attributes target-role, priority, maintenance and is-managed can be added directly to the group. All other attributes are inherited from the group's resources.
Resource stickiness: Stickiness in groups is additive. Every active resource in the group contributes its stickiness value to the group's total.

1.3 Resource clones #

Resource clones can run simultaneously on multiple nodes. This is required for cluster file systems like GFS2, for example. You can clone a primitive or a group if the resource agent supports it. Clones can also have additional clone-specific meta attributes to change their behavior.

Clones can be one of the following types:

Anonymous clones

Anonymous clones are the simplest type. All instances of the clone behave identically. Therefore, only one instance of an anonymous clone can be active on each node.

Globally unique clones

Globally unique clones are distinct entities. An instance of the clone running on one node is not necessarily equivalent to another instance on another node.

Promotable clones

Promotable clones allow the instances to be in one of two operating modes: promoted or unpromoted (also known as primary and secondary, or active and passive). Promotable clones can be either anonymous or globally unique.

To make a clone promotable, the resource agent must support the promote and demote operations. The cluster determines which instance of the clone to promote using one of the following methods (or a combination of both):

If a resource agent supports promotable clones, it might automatically set promotion scores based on its own preference for which instance to promote.
You can manually set promotion rules or preferences by creating location or colocation constraints.

For more information, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html#determining-which-instance-is-promoted

1.4 For more information #

Primitive resources: https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/resources.html
Groups and clones: https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html
Resource operations: https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/operations.html
Utilization attributes: https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/utilization.html

2 Configuring primitive resources #

Primitive resources represent the applications or services that need to be highly available. Configuring a primitive specifies the resource agent and any other information required for the cluster to manage a resource. To see the supported resource agent classes, run crm ra classes. To see the resource agents available within a class, run crm ra list CLASS.

You can create primitive resources with the CRM Shell (crmsh) or with the Hawk Web interface.

Requirements #

The application or service is already set up. For example, you must install and configure an Apache server before you can configure a cluster resource to manage it.
If an application or service has specific environment requirements, make sure the required settings are identical on all cluster nodes. Alternatively, you can add resource constraints to make sure the resource can only run on nodes that meet the requirements.

CRM Shell
Hawk

You can perform this procedure on any node in the cluster.

Log in either as the root user or as a user with sudo privileges.
Show the required and optional parameters for a resource agent:
```
> sudo crm ra info RESOURCE_AGENT
```
Configure a primitive resource. The basic command requires at least a name, a resource agent, and any parameters required by the resource agent:
```
> sudo crm configure primitive RESOURCE_NAME RESOURCE_AGENT \
  params ATTRIBUTE=VALUE
```
This example shows a basic cluster resource for managing an IP address:
```
> sudo crm configure primitive myIP IPaddr2 params ip=192.168.1.10
```
Additional options #
Operations
To show the supported operations for a resource agent, run crm ra info RESOURCE_AGENT and scroll to the bottom of the output. You can configure the operations as advised or change them as required. For example, to clone a resource, the primitive must have two monitor operations (one for each role):
> sudo crm configure primitive db1 mysql \ op monitor timeout=30 interval=10 role=Promoted \ op monitor timeout=30 interval=20 role=Unpromoted
Before configuring a primitive in a production environment, we recommend testing it to make sure the start and stop timeouts are long enough under load. Timeouts that are too short can lead to resource failure and potentially node fencing.
Meta attributes
Meta attributes change how the cluster treats a resource. To show all the meta attributes available for primitives, run crm_resource --list-options primitive. For example, allow-migrate enables live migration for a VM resource:
> sudo crm configure primitive VM1 VirtualDomain \ params config="/mnt/shared/VM1.xml" \ meta allow-migrate=true
Utilization attributes
If the cluster nodes have utilization attributes configured, you can add these attributes to cluster resources to help with load balancing. For example, if the nodes have the attribute memory, you can add the same attribute to a resource so that it can only run on nodes with enough capacity:
> sudo crm configure primitive web1 apache \ utilization memory=1024
Check the status of the resource:
```
> sudo crm status
```

Important: Don't manually touch cluster-controlled resources

After you configure a resource, the cluster controls all start or stop actions for that resource. The application or service that the resource represents must not be started or stopped from outside the cluster.

If you need to perform testing or maintenance tasks after a resource is already running under cluster control, put the resource into maintenance mode first.

For more information #

3 Configuring resource groups #

Cluster resources might depend on other resources, such as a Web server that requires an IP address and a file system. You can combine these resources into a resource group. Groups contain multiple primitives that need to be located together, started sequentially and stopped in the reverse order.

You can configure resource groups with the CRM Shell (crmsh) or with the Hawk Web interface.

Requirements #

The primitive resources already exist.
The primitive resources aren't already cloned or included in another group.

CRM Shell
Hawk

You can perform this procedure on any node in the cluster.

Log in either as the root user or as a user with sudo privileges.
Configure a group. Add the resources in the order you want them to start:
```
> sudo crm configure group GROUP_NAME RESOURCE1 RESOURCE2
```
Additional options #
- Groups inherit most meta attributes from their primitive resources, but you can also add the target-role, priority, maintenance and is-managed meta attributes directly to the group. For example:
```
> sudo crm configure group res-group res1 res2 res3 \
  meta target-role=Started priority=100
```
Check the status of the group:
```
> sudo crm status
```

For more information #

crm configure help group
https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html#groups-a-syntactic-shortcut

4 Configuring resource clones #

Resource clones can run simultaneously on multiple nodes. This is required for cluster file systems like GFS2, for example. You can clone a primitive or a group if the resource agent supports it. Clones can also have additional clone-specific meta attributes to change their behavior.

You can configure resource clones with the CRM Shell (crmsh) or with the Hawk Web interface.

Requirements #

The primitive or group already exists.
For promotable clones: the primitive has two monitor operations: one for when the resource is in the unpromoted role, and one with a different interval for when the resource is in the promoted role.
For promotable clones: the resource agent supports the promote and demote operations.

CRM Shell
Hawk

You can perform this procedure on any node in the cluster.

Log in either as the root user or as a user with sudo privileges.
Configure a clone. The basic command requires a name and one primitive or group:
```
> sudo crm configure clone CLONE_NAME RESOURCE
```
Additional options #
Meta attributes
Clones can have additional clone-specific meta attributes, such as clone-max or globally-unique. For more information about the meta attributes for clones, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html#clone-options
Promotable clones
To configure a promotable clone, add the meta attribute promotable=true:
> sudo crm configure clone clone1 res1 meta promotable=true
Promotable clones can also have additional meta attributes: promoted-max and promoted-node-max.
Check the cluster status to make sure the clone set is running on all nodes:
```
> sudo crm status
```

For more information #

crm configure help clone
https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html#clones-resources-that-can-have-multiple-active-instances

5 Adding resource tags #

You can use tags to refer to multiple resources at once without creating any constraints between the resources. This can be useful for grouping conceptually related resources and allows you to stop or start them all with a single command. A tag can include primitives, groups and clones, and can also be used in resource constraints.

You can create tags with the CRM Shell (crmsh) or with the Hawk Web interface.

CRM Shell
Hawk

You can perform this procedure on any node in the cluster.

Log in either as the root user or as a user with sudo privileges.
Create a tag and specify the resources to refer to with that tag:
```
> sudo crm configure tag TAG_NAME: RESOURCE1 RESOURCE2
```
For example, if you have multiple resources related to a database, you could create a tag called databases and add all resources related to the database to this tag:
```
> sudo crm configure tag databases: db1 db2 db3
```
To view all tags and their resources, use the following command:
```
> sudo crm configure show type:tag
```

This allows you to start or stop the resources with a single command:

> sudo crm resource start TAG_NAME

> sudo crm resource stop TAG_NAME

For more information #

crm configure help tag

6 Next steps #

To manage the resources after creating them, see the following articles:

Adding Constraints to HA Cluster Resources: This article explains how to add resource constraints to specify where, and in what order, cluster resources should run.
Managing HA Cluster Resources: This article explains how to manage cluster resources, including tasks such as starting, stopping, editing and deleting.

7 Legal Notice #

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”.

For SUSE trademarks, see https://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.

All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors, nor the translators shall be held liable for possible errors or the consequences thereof.