|Index|Managing High Availability Cluster Resources
SUSE Linux Enterprise High Availability 16.0

Managing High Availability Cluster Resources

Publication Date: 22 Oct 2026

After configuring High Availability cluster resources, you might need to start, stop, edit or delete them for various reasons. This article explains how to manage cluster resources safely.

Prerequisites

  • An existing SUSE Linux Enterprise High Availability cluster

  • Existing cluster resources

Tip

If you still need to configure cluster resources, see Configuring High Availability Cluster Resources.

1 What are cluster resources?

 

In a High Availability cluster, the applications and services that need to be highly available are called resources. Cluster resources can include Web sites, e-mail servers, databases, file systems, virtual machines or any other applications or services that need to be available to users at all times. You can start, stop, monitor and move resources as needed. You can also specify whether resources should run together on the same node, or start and stop in sequential order. If a node fails, the resources running on it fail over (move) to another node instead of being lost.

In SUSE Linux Enterprise High Availability, the cluster resource manager (CRM) is Pacemaker, which manages and coordinates all cluster services. Pacemaker uses resource agents (RAs) to start, stop and monitor resources. A resource agent abstracts the resource and presents its status to the cluster. This means that Pacemaker only interacts with the resource agent, not with the actual application or service. SUSE Linux Enterprise High Availability supports many different resource agents that are designed to manage specific types of resources.

1.1 Primitive resources

 

Primitive resources represent the applications or services that need to be highly available. Configuring a primitive specifies the resource agent and any other information required for the cluster to manage a resource. To see the supported resource agent classes, run crm ra classes. To see the resource agents available within a class, run crm ra list CLASS.

Primitive resources can have the following settings:

Properties

Resource properties are the basic details required by the cluster: a unique name for the resource, and the resource agent that manages the application or service.

Parameters

Parameters are specific to each resource agent and determine the details and behavior of an application or service. To show all the parameters available for a resource agent, run crm ra info RESOURCE_AGENT.

Operations

Operations are actions that the cluster can perform on a resource, such as start, stop and monitor. To show the supported operations for a resource agent, run crm ra info RESOURCE_AGENT and scroll to the bottom of the output.

Meta attributes

Meta attributes change how the cluster treats a resource. To show all the meta attributes available for primitives, run crm_resource --list-options primitive.

Utilization attributes

If the cluster nodes have utilization attributes configured, you can add these attributes to cluster resources to help with load balancing. For more information, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/utilization.html.

1.2 Resource groups

 

Cluster resources might depend on other resources, such as a Web server that requires an IP address and a file system. You can combine these resources into a resource group. Groups contain multiple primitives that need to be located together, started sequentially and stopped in the reverse order.

This diagram shows a box representing a node. The box contains four individual resources and a smaller box containing three more resources. The smaller box represents a resource group. A zoomed-in view of the resource group shows that the resources are an IP address, an XFS file system and an Apache Web server. The resources are numbered to show the start order in one direction and the reversed stop order in the other direction.
Figure 1: Resource group
 

Groups have the following properties:

Contents

Groups can only contain primitive resources, not clones or other groups.

Starting and stopping

Resources start in the order they appear and stop in the reverse order.

Location

Resources in the group must all run on the same node.

Dependency

If one of the resources can't run anywhere, then the resources that appear after it in the group can't run anywhere either.

Meta attributes

The meta attributes target-role, priority, maintenance and is-managed can be added directly to the group. All other attributes are inherited from the group's resources.

Resource stickiness

Stickiness in groups is additive. Every active resource in the group contributes its stickiness value to the group's total.

1.3 Resource clones

 

Resource clones can run simultaneously on multiple nodes. This is required for cluster file systems like GFS2, for example. You can clone a primitive or a group if the resource agent supports it. Clones can also have additional clone-specific meta attributes to change their behavior.

Clones can be one of the following types:

Anonymous clones

Anonymous clones are the simplest type. All instances of the clone behave identically. Therefore, only one instance of an anonymous clone can be active on each node.

Globally unique clones

Globally unique clones are distinct entities. An instance of the clone running on one node is not necessarily equivalent to another instance on another node.

Promotable clones

Promotable clones allow the instances to be in one of two operating modes: promoted or unpromoted (also known as primary and secondary, or active and passive). Promotable clones can be either anonymous or globally unique.

To make a clone promotable, the resource agent must support the promote and demote operations. The cluster determines which instance of the clone to promote using one of the following methods (or a combination of both):

  • If a resource agent supports promotable clones, it might automatically set promotion scores based on its own preference for which instance to promote.

  • You can manually set promotion rules or preferences by creating location or colocation constraints.

For more information, see https://clusterlabs.org/projects/pacemaker/doc/3.0/Pacemaker_Explained/html/collective.html#determining-which-instance-is-promoted

1.4 For more information

 

2 Showing cluster resources

 

You can check the status and configuration of existing cluster resources with the CRM Shell (crmsh) or with the Hawk Web interface.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Show the status of the resources: 

    > sudo crm resource status

  3. Show the configuration of the resources. The basic command shows the full CIB: 

    > sudo crm configure show

    You can limit the output by using modifiers such as an object's name, type or tag. For more information about the available modifiers, run the following command: 

    > sudo crm configure help show

3 Starting cluster resources

 

You can start cluster resources with the CRM Shell (crmsh) or with the Hawk Web interface.

Prerequisites
 

  • The application or service that the cluster resource manages must be set up correctly. For example, if you have a resource managing an Apache server, the Apache configuration must be complete before you start the resource, or the start action will fail.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Check the current status of the resource: 

    > sudo crm resource status

  3. Start the resource. If necessary, you can start multiple resources at once: 

    > sudo crm resource start RESOURCE1 RESOURCE2

  4. Check the status again to make sure the resource started successfully: 

    > sudo crm resource status

4 Stopping cluster resources

 

You can stop cluster resources with the CRM Shell (crmsh) or with the Hawk Web interface.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Check the current status of the resource: 

    > sudo crm resource status

  3. Stop the resource. If necessary, you can stop multiple resources at once: 

    > sudo crm resource stop RESOURCE1 RESOURCE2

  4. Check the status again to make sure the resource stopped successfully: 

    > sudo crm resource status

5 Editing primitive resources

 

You can edit primitive resources with the CRM Shell (crmsh) or with the Hawk Web interface.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. To add a new monitor operation, use the following command: 

    > sudo crm configure monitor RESOURCE INTERVAL:TIMEOUT

  3. To add a new parameter, meta attribute or utilization attribute, use one of the following commands: 

    > sudo crm resource param RESOURCE set PARAMETER VALUE
> sudo crm resource meta RESOURCE set ATTRIBUTE VALUE
> sudo crm resource utilization RESOURCE set ATTRIBUTE VALUE

  4. To delete a parameter, meta attribute or utilization attribute, use one of the following commands: 

    > sudo crm resource param RESOURCE delete PARAMETER
> sudo crm resource meta RESOURCE delete ATTRIBUTE
> sudo crm resource utilization RESOURCE delete ATTRIBUTE

  5. To change multiple attributes at once, use the following command: 

    > sudo crm configure edit RESOURCE

    This command opens the CIB representation of the resource in a text editor. You can make multiple changes to the resource's configuration, then save and exit.

    Important

    Don't use this command to rename a resource. The change won't apply to related objects.

  6. To rename a resource, first stop the resource, then use the following command: 

    > sudo crm configure rename OLD_NAME NEW_NAME

    This also changes the resource's name in related objects such as constraints, groups and clones.

For more information
 

  • crm configure help edit

  • crm configure help monitor

  • crm configure help rename

  • crm resource help meta

  • crm resource help param

  • crm resource help util

6 Editing resource groups

 

You can edit resource groups with the CRM Shell (crmsh) or with the Hawk Web interface.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. To remove a resource from a group, use the following command: 

    > sudo crm configure modgroup GROUP remove RESOURCE

  3. To add a resource to a group, use the following command: 

    > sudo crm configure modgroup GROUP add RESOURCE

    The resource is appended to the end of the group, meaning it starts last and stops first.

    Alternatively, you can use before or after to add the resource to a specific position in the group's order: 

    > sudo crm configure modgroup GROUP add RESOURCE0 before RESOURCE1
    > sudo crm configure modgroup GROUP add RESOURCE2 after RESOURCE1

  4. To change the order of resources that are already in the group, use the following command: 

    > sudo crm configure edit GROUP

    This command opens the CIB representation of the group in a text editor. You can make multiple changes to the group's configuration, then save and exit.

    Important

    Don't use this command to rename a group. The change won't apply to related objects.

  5. To rename a group, use the following command: 

    > sudo crm configure rename OLD_NAME NEW_NAME

    This also changes the group's name in related objects such as constraints and clones.

For more information
 

  • crm configure help edit

  • crm configure help modgroup

  • crm configure help rename

7 Cleaning up failed resources

 

The cluster automatically restarts resources if they fail, but each failure increases the resource's fail count. If the fail count reaches the resource's migration threshold, the current node becomes ineligible to run the resource. The cluster either moves the resource to an eligible node or stops the resource if no eligible node is available.

By default, fail counts are not automatically reset. This procedure explains how to reset the fail count manually.

You can clean up failed resources with the CRM Shell (crmsh) or with the Hawk Web interface. If the resource is part of a group, the other resources in the group are also cleaned up.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Show the status of the cluster, including fail counts: 

    > sudo crm status failcounts

    If a resource has a fail count, it appears in the Migration Summary section.

  3. Clean up the resource on all nodes: 

    > sudo crm resource cleanup RESOURCE

    Alternatively, you can clean up the resource on a specific node: 

    > sudo crm resource cleanup RESOURCE NODE
Tip
Tip: Automatically resetting fail counts

You can configure a cluster resource's fail count to reset automatically by adding the failure-timeout meta attribute to the resource.

For more information
 

  • crm resource help cleanup

8 Deleting cluster resources

 

You can delete a resource from the cluster configuration with the CRM Shell (crmsh) or with the Hawk Web interface.

Prerequisites
 

  • If the resource has the status FAILED, you must clean it up before you can delete it.

  • CRM Shell
  • Hawk

You can perform this procedure on any node in the cluster.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Check the current status of the resource: 

    > sudo crm resource status

  3. If the resource is Started, you can stop it with the following command: 

    > sudo crm resource stop RESOURCE

  4. Delete the resource. If necessary, you can delete multiple resources at once: 

    > sudo crm configure delete RESOURCE1 RESOURCE2
For more information
 

  • crm configure help delete

9 Applying multiple changes at once

 

SUSE Linux Enterprise High Availability provides different ways to apply multiple cluster changes simultaneously. This is sometimes necessary when you make changes that are dependent on each other. For example, a group of related resources might need to have colocation or order constraints in place before the resources start.

You can use one of the following methods to make batch changes:

9.1 Using the CRM Shell in interactive mode

 

The CRM Shell's interactive mode is a dedicated, internal command shell that allows you to run multiple commands without needing to repeatedly type the crm prefix. You can also move down a level to a subcommand, such as crm configure, to run multiple commands at the same level. Changes you make in interactive mode are only applied to the live cluster after you commit them, which can be useful when configuring resources that are dependent on one another and require constraints. Interactive mode also supports tab completion.

  1. Log in either as the root user or as a user with sudo privileges.

  2. Start interactive mode: 

    > sudo crm

    The prompt changes to crm(live/HOSTNAME)#. However, for better readability we omit the host name in our documentation: 

    crm(live)#
    Tip
    Tip: Starting a subcommand directly

    You can also start interactive mode directly at the subcommand level. For example: 

    > sudo crm configure
crm(live)configure#

  3. You can navigate interactive mode with the following options:

    • List the available subcommands: 

      crm(live)# ls

    • Enter the name of a subcommand to switch to that level. For example: 

      crm(live)# resource
crm(live)resource#

    • Switch between subcommands with cd. For example: 

      crm(live)resource# cd ../node
crm(live)node#

    • Go back one level with up or cd: 

      crm(live)node# up
crm(live)#

  4. Make your changes. This example shows commands for configuring a virtual IP address and adding a location constraint to keep the IP address running on a node named alice. Because both commands are under the crm configure subcommand, you can switch to the configure level to shorten the actual commands you need to type.

    Example 1: Adding a virtual IP address with a location constraint for node alice
     
    crm(live)# configure
crm(live)configure# primitive vip1 IPaddr2 params ip=192.168.1.5
crm(live)configure# location loc-vip1-on-alice vip1 inf: alice
crm(live)configure# show changed

  5. Commit your changes to apply them to the live cluster: 

    crm(live)configure# commit

    You must commit your changes before you exit interactive mode, switch back to the crm level, or switch to another subcommand. The changes will be lost if you don't commit them, but the CRM Shell will warn you before this happens.

  6. Exit interactive mode: 

    crm(live)configure# quit
For more information
 

9.2 Using Hawk in batch mode

 

Hawk's batch mode allows you to stage configuration changes and apply them all as a single transaction, instead of having each change take effect immediately. This can be useful when configuring resources that are dependent on one another and require constraints.

Batch mode also includes a cluster simulator, which shows the expected outcome of each change without affecting the live cluster. For example, if you stop a resource while in batch mode, the user interface shows the resource as stopped, but in the live cluster the resource is still running.

You can also inject cluster events, such as node changes and resource operations, to explore potential failure scenarios. These changes are only simulations, and won't be applied to the live cluster along with the configuration changes.

Important
Important: Using configuration wizards in batch mode

Certain wizards include actions beyond cluster configuration, such as installing packages. When using those wizards in batch mode, any changes that go beyond cluster configuration would be applied to the live system immediately.

Therefore, wizards that require root permission can't be used while in batch mode.

  1. In a Web browser, go to https://HAWKSERVER:7630/ and log in.

  2. In the top-level row, click Batch.

    An additional bar appears below the top-level row to indicate that batch mode is active. This bar also includes the options to show, discard or apply your changes.

    At the top of the screen there is an orange banner that says “Batch mode active”. The right side of the banner has three options: Show, Discard and Apply.
    Figure 2: Hawk batch mode bar
     

  3. While batch mode is active, perform any changes to your cluster, like adding or editing resources and constraints, or editing the cluster configuration. The changes will be simulated but are not applied to the live cluster yet.

  4. To view details of your changes, select Show from the batch mode bar.

    The Batch Mode window opens and shows the difference between the live cluster and the simulated changes in crmsh syntax. Lines starting with a - character represent the current state, whereas lines starting with + show the proposed state.

    This example of the “Batch Mode” window shows that the user has changed the state of a resource clone from Started to Stopped. The top of the window also has two additional options: Simulator and Inject.
    Figure 3: Hawk Batch Mode window
     

  5. (Optional) To test potential failures or to see how the cluster behaves in a specific scenario, you can simulate node changes and resource operations:

    • Simulate a node failure or status change:

      1. Click Inject › Node Event.

      2. Select a Node and the node State to simulate.

      3. Click OK.

    • Simulate a resource operation:

      1. Click Inject › Resource Event.

      2. Select a Resource and the Operation to simulate.

      3. If necessary, define an Interval.

      4. Select the Node on which to run the operation.

      5. Select the operation Result to simulate.

      6. Click OK.

    The Batch Mode window shows a new line per injected event. Any event listed here is simulated immediately and is reflected on the Status screen.

    To remove an injected event, click the x icon next to it. Hawk updates the Status screen accordingly.

    This example of the “Batch Mode” window shows that the user has changed the state of a resource clone from Started to Stopped. The user has also injected a node event and a resource event.
    Figure 4: Hawk Batch Mode window with injected events
     

  6. (Optional) To view more details about the changes, click Simulator and choose one of the following:

    • Summary: Shows a detailed summary.

    • CIB (in): Shows the initial CIB state.

    • CIB (out): Shows what the CIB would look like after the transition.

    • Transition Graph: Shows a graphical representation of the transition.

    • Transition: Shows an XML representation of the transition.

  7. After reviewing your changes, click Close to exit the Batch Mode window.

  8. Choose whether to keep your changes or discard them:

    • To keep your changes, click Apply in the batch mode bar. The Apply Configuration window opens and shows a list of the changes. Only configuration changes are applied; injected events are not. Click Apply to confirm.

    • To discard your changes, click Discard in the batch mode bar, then click OK to confirm.

    Both options deactivate batch mode and take you back to normal mode.

9.3 Loading configuration changes from a file

 

The crm configure load command allows you to load configuration changes from a local file or a network URL. This command is useful if you prefer to prepare your configuration changes in advance. The CRM Shell validates the changes and warns you if the new configuration is invalid.

You can load the new configuration with one of the following methods:

  • update: Imports the new configuration into the current configuration by adding new items or updating existing items. This method can't remove any existing items.

  • replace: Replaces the current configuration with the new configuration. If any resources are running, this method still adds new items and updates existing items, but doesn't remove any existing items.

  • push: Imports the new configuration into the current configuration by adding new items or updating existing items, and also removes existing items that are not present in the new configuration. This method can remove existing items even when resources are running, but only if the removed item isn't a running resource. If you try to remove a running resource, the command exits with an error.

You can perform this procedure on any node in the cluster:

  1. Log in either as the root user or as a user with sudo privileges.

  2. Show the cluster's current configuration:

    • Show the current configuration in CLI format: 

      > sudo crm configure show

    • Show the current configuration in XML format: 

      > sudo crm configure show xml

  3. Copy the current configuration into a new file, for example, hacluster-config.xml. You can copy the full configuration or just the portion you want to change.

  4. Make your changes to the new configuration.

  5. Load the new configuration from the file to the live cluster:

    • Load the new configuration in CLI format: 

      > sudo crm configure load METHOD FILE

    • Load the new configuration in XML format: 

      > sudo crm configure load xml METHOD FILE

    METHOD can be one of replace, update or push. FILE can be a path to a local file or a URL to a remote file.

    The CRM Shell checks the new configuration. If your changes would break the cluster configuration, you'll see the following output: 

    error: CIB did not pass schema validation
Configuration invalid
Do you still want to commit (y/n)?

    If this happens, enter n to discard your changes.

  6. Check the live cluster configuration for any errors: 

    > sudo crm configure show

  7. Check the status of the cluster to make sure everything is still running as expected: 

    > sudo crm status
For more information
 

  • crm configure help load