|Index|Configuring High Availability Cluster Resources
SUSE Linux Enterprise High Availability 16.0

Configuring High Availability Cluster Resources

Publication Date: 22 Oct 2026

Configure High Availability cluster resources to manage your critical applications and services. Cluster resources can include Web sites, e-mail servers, databases, file systems, virtual machines or any other applications or services that need to be available to users at all times. If one node fails, the resources that were running on it move to another node with little or no downtime. This helps to protect data integrity and reduce unplanned downtime for critical workloads.

Prerequisites

  • An existing SUSE Linux Enterprise High Availability cluster

  • Existing applications or services to add to the cluster

1 What are cluster resources?

 

In a High Availability cluster, the applications and services that need to be highly available are called resources. Cluster resources can include Web sites, e-mail servers, databases, file systems, virtual machines or any other applications or services that need to be available to users at all times. You can start, stop, monitor and move resources as needed. You can also specify whether resources should run together on the same node, or start and stop in sequential order. If a node fails, the resources running on it fail over (move) to another node instead of being lost.

In SUSE Linux Enterprise High Availability, the cluster resource manager (CRM) is Pacemaker, which manages and coordinates all cluster services. Pacemaker uses resource agents (RAs) to start, stop and monitor resources. A resource agent abstracts the resource and presents its status to the cluster. This means that Pacemaker only interacts with the resource agent, not with the actual application or service. SUSE Linux Enterprise High Availability supports many different resource agents that are designed to manage specific types of resources.

1.1 Primitive resources

 

Primitive resources represent the applications or services that need to be highly available. Configuring a primitive specifies the resource agent and any other information required for the cluster to manage a resource. To see the supported resource agent classes, run crm ra classes. To see the resource agents available within a class, run crm ra list CLASS.

Primitive resources can have the following settings:

Properties

Resource properties are the basic details required by the cluster: a unique name for the resource, and the resource agent that manages the application or service.

Parameters

Parameters are specific to each resource agent and determine the details and behavior of an application or service. To show all the parameters available for a resource agent, run crm ra info RESOURCE_AGENT.

Operations

Operations are actions that the cluster can perform on a resource, such as start, stop and monitor. To show the supported operations for a resource agent, run crm ra info RESOURCE_AGENT and scroll to the bottom of the output.

Meta attributes

Meta attributes change how the cluster treats a resource. To show all the meta attributes available for primitives, run crm_resource --list-options primitive.

Utilization attributes

If the cluster nodes have utilization attributes configured, you can add these attributes to cluster resources to help with load balancing. For more information, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/utilization.html.

1.2 Resource groups

 

Cluster resources might depend on other resources, such as a Web server that requires an IP address and a file system. You can combine these resources into a resource group. Groups contain multiple primitives that need to be located together, started sequentially and stopped in the reverse order.

This diagram shows a box representing a node. The box contains four individual resources and a smaller box containing three more resources. The smaller box represents a resource group. A zoomed-in view of the resource group shows that the resources are an IP address, an XFS file system and an Apache Web server. The resources are numbered to show the start order in one direction and the reversed stop order in the other direction.
Figure 1: Resource group
 

Groups have the following properties:

Contents

Groups can only contain primitive resources, not clones or other groups.

Starting and stopping

Resources start in the order they appear and stop in the reverse order.

Location

Resources in the group must all run on the same node.

Dependency

If one of the resources can't run anywhere, then the resources that appear after it in the group can't run anywhere either.

Meta attributes

The meta attributes target-role, priority, maintenance and is-managed can be added directly to the group. All other attributes are inherited from the group's resources.

Resource stickiness

Stickiness in groups is additive. Every active resource in the group contributes its stickiness value to the group's total.

1.3 Resource clones

 

Resource clones can run simultaneously on multiple nodes. This is required for cluster file systems like GFS2, for example. You can clone a primitive or a group if the resource agent supports it. Clones can also have additional clone-specific meta attributes to change their behavior.

Clones can be one of the following types:

Anonymous clones

Anonymous clones are the simplest type. All instances of the clone behave identically. Therefore, only one instance of an anonymous clone can be active on each node.

Globally unique clones

Globally unique clones are distinct entities. An instance of the clone running on one node is not necessarily equivalent to another instance on another node.

Promotable clones

Promotable clones allow the instances to be in one of two operating modes: promoted or unpromoted (also known as primary and secondary, or active and passive). Promotable clones can be either anonymous or globally unique.

To make a clone promotable, the resource agent must support the promote and demote operations. The cluster determines which instance of the clone to promote using one of the following methods (or a combination of both):

  • If a resource agent supports promotable clones, it might automatically set promotion scores based on its own preference for which instance to promote.

  • You can manually set promotion rules or preferences by creating location or colocation constraints.

For more information, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html#determining-which-instance-is-promoted

1.4 For more information

 

2 Configuring primitive resources

 

Primitive resources represent the applications or services that need to be highly available. Configuring a primitive specifies the resource agent and any other information required for the cluster to manage a resource. To see the supported resource agent classes, run crm ra classes. To see the resource agents available within a class, run crm ra list CLASS.

You can create primitive resources with the CRM Shell (crmsh) or with the Hawk Web interface.

Prerequisites
 

  • The application or service is already set up. For example, you must install and configure an Apache server before you can configure a cluster resource to manage it.

  • If an application or service has specific environment requirements, make sure the required settings are identical on all cluster nodes. Alternatively, you can add resource constraints to make sure the resource can only run on nodes that meet the requirements.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Show the required and optional parameters for a resource agent: 

    > sudo crm ra info RESOURCE_AGENT

  3. Configure a primitive resource. The basic command requires at least a name, a resource agent, and any parameters required by the resource agent: 

    > sudo crm configure primitive RESOURCE_NAME RESOURCE_AGENT \
  params ATTRIBUTE=VALUE

    This example shows a basic cluster resource for managing an IP address: 

    > sudo crm configure primitive myIP IPaddr2 params ip=192.168.1.10
    Additional options
     
    Operations

    To show the supported operations for a resource agent, run crm ra info RESOURCE_AGENT and scroll to the bottom of the output. You can configure the operations as advised or change them as required. For example, to clone a resource, the primitive must have two monitor operations (one for each role): 

    > sudo crm configure primitive db1 mysql \
  op monitor timeout=30 interval=10 role=Promoted \
  op monitor timeout=30 interval=20 role=Unpromoted
    Important

    Before configuring a primitive in a production environment, we recommend testing it to make sure the start and stop timeouts are long enough under load. Timeouts that are too short can lead to resource failure and potentially node fencing.

    Meta attributes

    Meta attributes change how the cluster treats a resource. To show all the meta attributes available for primitives, run crm_resource --list-options primitive. For example, allow-migrate enables live migration for a VM resource: 

    > sudo crm configure primitive VM1 VirtualDomain \
  params config="/mnt/shared/VM1.xml" \
  meta allow-migrate=true
    Utilization attributes

    If the cluster nodes have utilization attributes configured, you can add these attributes to cluster resources to help with load balancing. For example, if the nodes have the attribute memory, you can add the same attribute to a resource so that it can only run on nodes with enough capacity: 

    > sudo crm configure primitive web1 apache \
  utilization memory=1024

  4. Check the status of the resource: 

    > sudo crm status
Important
Important: Don't manually touch cluster-controlled resources

After you configure a resource, the cluster controls all start or stop actions for that resource. The application or service that the resource represents must not be started or stopped from outside the cluster.

If you need to perform testing or maintenance tasks after a resource is already running under cluster control, put the resource into maintenance mode first.

For more information
 

3 Configuring resource groups

 

Cluster resources might depend on other resources, such as a Web server that requires an IP address and a file system. You can combine these resources into a resource group. Groups contain multiple primitives that need to be located together, started sequentially and stopped in the reverse order.

You can configure resource groups with the CRM Shell (crmsh) or with the Hawk Web interface.

Prerequisites
 

  • The primitive resources already exist.

  • The primitive resources aren't already cloned or included in another group.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Configure a group. Add the resources in the order you want them to start: 

    > sudo crm configure group GROUP_NAME RESOURCE1 RESOURCE2
    Additional options
     

    • Groups inherit most meta attributes from their primitive resources, but you can also add the target-role, priority, maintenance and is-managed meta attributes directly to the group. For example: 

      > sudo crm configure group res-group res1 res2 res3 \
  meta target-role=Started priority=100

  3. Check the status of the group: 

    > sudo crm status
For more information
 

4 Configuring resource clones

 

Resource clones can run simultaneously on multiple nodes. This is required for cluster file systems like GFS2, for example. You can clone a primitive or a group if the resource agent supports it. Clones can also have additional clone-specific meta attributes to change their behavior.

You can configure resource clones with the CRM Shell (crmsh) or with the Hawk Web interface.

Prerequisites
 

  • The primitive or group already exists.

  • For promotable clones: the primitive has two monitor operations: one for when the resource is in the unpromoted role, and one with a different interval for when the resource is in the promoted role.

  • For promotable clones: the resource agent supports the promote and demote operations.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Configure a clone. The basic command requires a name and one primitive or group: 

    > sudo crm configure clone CLONE_NAME RESOURCE
    Additional options
     
    Meta attributes

    Clones can have additional clone-specific meta attributes, such as clone-max or globally-unique. For more information about the meta attributes for clones, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html#clone-options

    Promotable clones

    To configure a promotable clone, add the meta attribute promotable=true: 

    > sudo crm configure clone clone1 res1 meta promotable=true

    Promotable clones can also have additional meta attributes: promoted-max and promoted-node-max.

  3. Check the cluster status to make sure the clone set is running on all nodes: 

    > sudo crm status
For more information
 

5 Adding resource tags

 

You can use tags to refer to multiple resources at once without creating any constraints between the resources. This can be useful for grouping conceptually related resources and allows you to stop or start them all with a single command. A tag can include primitives, groups and clones, and can also be used in resource constraints.

You can create tags with the CRM Shell (crmsh) or with the Hawk Web interface.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Create a tag and specify the resources to refer to with that tag: 

    > sudo crm configure tag TAG_NAME: RESOURCE1 RESOURCE2

    For example, if you have multiple resources related to a database, you could create a tag called databases and add all resources related to the database to this tag: 

    > sudo crm configure tag databases: db1 db2 db3

  3. To view all tags and their resources, use the following command: 

    > sudo crm configure show type:tag

This allows you to start or stop the resources with a single command: 

> sudo crm resource start TAG_NAME
> sudo crm resource stop TAG_NAME
For more information
 

  • crm configure help tag

6 Applying multiple changes at once

 

SUSE Linux Enterprise High Availability provides different ways to apply multiple cluster changes simultaneously. This is sometimes necessary when you make changes that are dependent on each other. For example, a group of related resources might need to have colocation or order constraints in place before the resources start.

You can use one of the following methods to make batch changes:

6.1 Using the CRM Shell in interactive mode

 

The CRM Shell's interactive mode is a dedicated, internal command shell that allows you to run multiple commands without needing to repeatedly type the crm prefix. You can also move down a level to a subcommand, such as crm configure, to run multiple commands at the same level. Changes you make in interactive mode are only applied to the live cluster after you commit them, which can be useful when configuring resources that are dependent on one another and require constraints. Interactive mode also supports tab completion.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Start interactive mode: 

    > sudo crm

    The prompt changes to crm(live/HOSTNAME)#. However, for better readability we omit the host name in our documentation: 

    crm(live)#
    Tip
    Tip: Starting a subcommand directly

    You can also start interactive mode directly at the subcommand level. For example: 

    > sudo crm configure
crm(live)configure#

  3. You can navigate interactive mode with the following options:

    • List the available subcommands: 

      crm(live)# ls

    • Enter the name of a subcommand to switch to that level. For example: 

      crm(live)# resource
crm(live)resource#

    • Switch between subcommands with cd. For example: 

      crm(live)resource# cd ../node
crm(live)node#

    • Go back one level with up or cd: 

      crm(live)node# up
crm(live)#

  4. Make your changes. This example shows commands for configuring a virtual IP address and adding a location constraint to keep the IP address running on a node named alice. Because both commands are under the crm configure subcommand, you can switch to the configure level to shorten the actual commands you need to type.

    Example 1: Adding a virtual IP address with a location constraint for node alice
     
    crm(live)# configure
crm(live)configure# primitive vip1 IPaddr2 params ip=192.168.1.5
crm(live)configure# location loc-vip1-on-alice vip1 inf: alice
crm(live)configure# show changed

  5. Commit your changes to apply them to the live cluster: 

    crm(live)configure# commit

    You must commit your changes before you exit interactive mode, switch back to the crm level, or switch to another subcommand. The changes will be lost if you don't commit them, but the CRM Shell will warn you before this happens.

  6. Exit interactive mode: 

    crm(live)configure# quit
For more information
 

6.2 Using Hawk in batch mode

 

Hawk's batch mode allows you to stage configuration changes and apply them all as a single transaction, instead of having each change take effect immediately. This can be useful when configuring resources that are dependent on one another and require constraints.

Batch mode also includes a cluster simulator, which shows the expected outcome of each change without affecting the live cluster. For example, if you stop a resource while in batch mode, the user interface shows the resource as stopped, but in the live cluster the resource is still running.

You can also inject cluster events, such as node changes and resource operations, to explore potential failure scenarios. These changes are only simulations, and won't be applied to the live cluster along with the configuration changes.

Important
Important: Using configuration wizards in batch mode

Certain wizards include actions beyond cluster configuration, such as installing packages. When using those wizards in batch mode, any changes that go beyond cluster configuration would be applied to the live system immediately.

Therefore, wizards that require root permission can't be used while in batch mode.

  1. In a Web browser, go to https://HAWKSERVER:7630/ and log in.

  2. In the top-level row, click Batch.

    An additional bar appears below the top-level row to indicate that batch mode is active. This bar also includes the options to show, discard or apply your changes.

    At the top of the screen there is an orange banner that says “Batch mode active”. The right side of the banner has three options: Show, Discard and Apply.
    Figure 2: Hawk batch mode bar
     

  3. While batch mode is active, perform any changes to your cluster, like adding or editing resources and constraints, or editing the cluster configuration. The changes will be simulated but are not applied to the live cluster yet.

  4. To view details of your changes, select Show from the batch mode bar.

    The Batch Mode window opens and shows the difference between the live cluster and the simulated changes in crmsh syntax. Lines starting with a - character represent the current state, whereas lines starting with + show the proposed state.

    This example of the “Batch Mode” window shows that the user has changed the state of a resource clone from Started to Stopped. The top of the window also has two additional options: Simulator and Inject.
    Figure 3: Hawk Batch Mode window
     

  5. (Optional) To test potential failures or to see how the cluster behaves in a specific scenario, you can simulate node changes and resource operations:

    • Simulate a node failure or status change:

      1. Click Inject › Node Event.

      2. Select a Node and the node State to simulate.

      3. Click OK.

    • Simulate a resource operation:

      1. Click Inject › Resource Event.

      2. Select a Resource and the Operation to simulate.

      3. If necessary, define an Interval.

      4. Select the Node on which to run the operation.

      5. Select the operation Result to simulate.

      6. Click OK.

    The Batch Mode window shows a new line per injected event. Any event listed here is simulated immediately and is reflected on the Status screen.

    To remove an injected event, click the x icon next to it. Hawk updates the Status screen accordingly.

    This example of the “Batch Mode” window shows that the user has changed the state of a resource clone from Started to Stopped. The user has also injected a node event and a resource event.
    Figure 4: Hawk Batch Mode window with injected events
     

  6. (Optional) To view more details about the changes, click Simulator and choose one of the following:

    • Summary: Shows a detailed summary.

    • CIB (in): Shows the initial CIB state.

    • CIB (out): Shows what the CIB would look like after the transition.

    • Transition Graph: Shows a graphical representation of the transition.

    • Transition: Shows an XML representation of the transition.

  7. After reviewing your changes, click Close to exit the Batch Mode window.

  8. Choose whether to keep your changes or discard them:

    • To keep your changes, click Apply in the batch mode bar. The Apply Configuration window opens and shows a list of the changes. Only configuration changes are applied; injected events are not. Click Apply to confirm.

    • To discard your changes, click Discard in the batch mode bar, then click OK to confirm.

    Both options deactivate batch mode and take you back to normal mode.

6.3 Loading configuration changes from a file

 

The crm configure load command allows you to load configuration changes from a local file or a network URL. This command is useful if you prefer to prepare your configuration changes in advance. The CRM Shell validates the changes and warns you if the new configuration is invalid.

You can load the new configuration with one of the following methods:

  • update: Imports the new configuration into the current configuration by adding new items or updating existing items. This method can't remove any existing items.

  • replace: Replaces the current configuration with the new configuration. If any resources are running, this method still adds new items and updates existing items, but doesn't remove any existing items.

  • push: Imports the new configuration into the current configuration by adding new items or updating existing items, and also removes existing items that are not present in the new configuration. This method can remove existing items even when resources are running, but only if the removed item isn't a running resource. If you try to remove a running resource, the command exits with an error.

You can perform this procedure on any node in the cluster:

  1. Log in either as the root user or as a user with sudo privileges.

  2. Show the cluster's current configuration:

    • Show the current configuration in CLI format: 

      > sudo crm configure show

    • Show the current configuration in XML format: 

      > sudo crm configure show xml

  3. Copy the current configuration into a new file, for example, hacluster-config.xml. You can copy the full configuration or just the portion you want to change.

  4. Make your changes to the new configuration.

  5. Load the new configuration from the file to the live cluster:

    • Load the new configuration in CLI format: 

      > sudo crm configure load METHOD FILE

    • Load the new configuration in XML format: 

      > sudo crm configure load xml METHOD FILE

    METHOD can be one of replace, update or push. FILE can be a path to a local file or a URL to a remote file.

    The CRM Shell checks the new configuration. If your changes would break the cluster configuration, you'll see the following output: 

    error: CIB did not pass schema validation
Configuration invalid
Do you still want to commit (y/n)?

    If this happens, enter n to discard your changes.

  6. Check the live cluster configuration for any errors: 

    > sudo crm configure show

  7. Check the status of the cluster to make sure everything is still running as expected: 

    > sudo crm status
For more information
 

  • crm configure help load

7 Next steps

 

To manage the resources after creating them, see the following articles:

Adding Constraints to High Availability Cluster Resources

This article explains how to add resource constraints to specify where, and in what order, cluster resources should run.

Managing High Availability Cluster Resources

This article explains how to manage cluster resources, including tasks such as starting, stopping, editing and deleting.

Manually Moving High Availability Cluster Resources

This article explains how to manually move a resource, either to a specific node or just away from the current node.

Performing Maintenance Tasks on Cluster Resources

This article explains how to safely perform manual maintenance tasks on cluster resources.