8 Managing cluster resources #
After configuring the resources in the cluster, use the cluster management tools to start, stop, clean up, remove or migrate the resources. This chapter describes how to use Hawk2 or crmsh for resource management tasks.
8.1 Showing cluster resources #
8.1.1 Showing cluster resources with crmsh #
When administering a cluster the command crm configure show
lists the current CIB objects like cluster configuration, global options,
primitives, and others:
#
crm configure show
node 178326192: alice node 178326448: bob primitive admin_addr IPaddr2 \ params ip=192.168.2.1 \ op monitor interval=10 timeout=20 primitive stonith-sbd stonith:external/sbd \ params pcmk_delay_max=30 property cib-bootstrap-options: \ have-watchdog=true \ dc-version=1.1.15-17.1-e174ec8 \ cluster-infrastructure=corosync \ cluster-name=hacluster \ stonith-enabled=true \ placement-strategy=balanced \ standby-mode=true rsc_defaults rsc-options: \ resource-stickiness=1 \ migration-threshold=3 op_defaults op-options: \ timeout=600 \ record-pending=true
If you have lots of resources, the output of show
is too verbose. To restrict the output, use the name of the resource.
For example, to list the properties of the primitive
admin_addr
only, append the resource name to
show
:
#
crm configure show admin_addr
primitive admin_addr IPaddr2 \ params ip=192.168.2.1 \ op monitor interval=10 timeout=20
However, in some cases, you want to limit the output of specific resources
even more. This can be achieved with filters. Filters
limit the output to specific components. For example, to list the
nodes only, use type:node
:
#
crm configure show type:node
node 178326192: alice node 178326448: bob
If you are also interested in primitives, use the
or
operator:
#
crm configure show type:node or type:primitive
node 178326192: alice node 178326448: bob primitive admin_addr IPaddr2 \ params ip=192.168.2.1 \ op monitor interval=10 timeout=20 primitive stonith-sbd stonith:external/sbd \ params pcmk_delay_max=30
Furthermore, to search for an object that starts with a certain string, use this notation:
#
crm configure show type:primitive and 'admin*'
primitive admin_addr IPaddr2 \ params ip=192.168.2.1 \ op monitor interval=10 timeout=20
To list all available types, enter crm configure show type:
and press the →| key. The Bash completion gives
you a list of all types.
8.2 Editing resources and groups #
You can edit resources or groups using either Hawk2 or crmsh.
8.2.1 Editing resources and groups with Hawk2 #
If you have created a resource, you can edit its configuration at any time by adjusting parameters, operations or meta attributes as needed.
Log in to Hawk2:
https://HAWKSERVER:7630/
On the Hawk2
screen, go to the list.In the
column, click the arrow down icon next to the resource or group you want to modify and select .The resource configuration screen opens.
Figure 8.1: Hawk2—editing a primitive resource #At the top of the configuration screen, you can select operations to perform.
If you edit a primitive resource, the following operations are available:
Copying the resource
Renaming the resource (changing its ID)
Deleting the resource
If you edit a group, the following operations are available:
Creating a new primitive to be added to this group
Renaming the group (changing its ID)
Dragging and dropping group members into a new order
To add a new parameter, operation or meta attribute, select an entry from the empty drop-down box.
To edit any values in the
category, click the icon of the respective entry, enter a different value for the operation, and click .When you are finished, click the
button in the resource configuration screen to confirm your changes to the parameters, operations or meta attributes.A message at the top of the screen shows if the action has been successful.
8.2.2 Editing groups with crmsh #
To change the order of a group member, use the modgroup
command
from the configure
subcommand. For example, use the following
command to move the primitive Email
before Public-IP
:
crm(live)configure#
modgroup g-mailsvc add Email before Public-IP
To remove a resource from a group (for example, Email
), use this command:
crm(live)configure#
modgroup g-mailsvc remove Email
8.3 Starting cluster resources #
Before you start a cluster resource, make sure it is set up correctly. For example, if you use an Apache server as a cluster resource, set up the Apache server first. Complete the Apache configuration before starting the respective resource in your cluster.
When managing a resource via the High Availability software, the resource must not be started or stopped otherwise (outside the cluster, for example manually or on boot or reboot). The High Availability software is responsible for all service start or stop actions.
However, if you want to check if the service is configured properly, start it manually, but make sure that it is stopped again before the High Availability software takes over.
For interventions in resources that are currently managed by the cluster,
set the resource to maintenance mode
first. For details,
see Procedure 28.5, “Putting a resource into maintenance mode with Hawk2”.
You can start a cluster resource using either Hawk2 or crmsh.
8.3.1 Starting cluster resources with Hawk2 #
When creating a resource with Hawk2, you can set its initial state with
the target-role
meta attribute. If you set its value to
stopped
, the resource does not start automatically after
being created.
Log in to Hawk2:
https://HAWKSERVER:7630/
From the left navigation bar, select
› . The list of also shows the .Select the resource to start. In its
column, click the icon. To continue, confirm the message that appears.
When the resource has started, Hawk2 changes the resource's
to green and shows on which node it is running.8.3.2 Starting cluster resources with crmsh #
To start a new cluster resource, you need the respective identifier.
Log in as
root
and start thecrm
interactive shell:#
crm
Switch to the resource level:
crm(live)#
resource
Start the resource with
start
and press the →| key to show all known resources:crm(live)resource#
start ID
8.4 Stopping cluster resources #
8.4.1 Stopping cluster resources with crmsh #
To stop one or more existing cluster resources, you need the respective identifiers.
Log in as
root
and start thecrm
interactive shell:#
crm
Switch to the resource level:
crm(live)#
resource
Stop the resource with
stop
and press the →| key to show all known resources:crm(live)resource#
stop ID
You can stop multiple resources at once:
crm(live)resource#
stop ID1 ID2
...
8.5 Cleaning up cluster resources #
A resource is automatically restarted if it fails, but each failure increases the resource's fail count.
If a migration-threshold
has been set for the resource,
the node can no longer run the resource when the number of failures reaches
the migration threshold.
By default, fail counts are not automatically reset. You can configure a fail count
to be reset automatically by setting a failure-timeout
option for the
resource, or you can manually reset the fail count using either Hawk2 or crmsh.
8.5.1 Cleaning up cluster resources with Hawk2 #
Log in to Hawk2:
https://HAWKSERVER:7630/
From the left navigation bar, select
. The list of also shows the .Go to the resource to clean up. In the
column, click the arrow down button and select . To continue, confirm the message that appears.This executes the command
crm resource cleanup
and cleans up the resource on all nodes.
8.5.2 Cleaning up cluster resources with crmsh #
Open a shell and log in as user
root
.Get a list of all your resources:
#
crm resource status
Full List of Resources * admin-ip (ocf:heartbeat:IPaddr2): Started * stonith-sbd (stonith:external/sbd): Started * Resource Group: dlm-clvm: * dlm: (ocf:pacemaker:controld) Started * clvm: (ocf:heartbeat:lvmlockd) StartedShow the fail count of a resource:
#
crm resource failcount RESOURCE show NODE
For example, to show the fail count of the resource
dlm
on nodealice
:#
crm resource failcount dlm show alice
scope=status name=fail-count-dlm value=2Clean up the resource:
#
crm resource cleanup RESOURCE
This command cleans up the resource on all nodes. If the resource is part of a group, crmsh also cleans up the other resources in the group.
8.6 Removing cluster resources #
To remove a resource from the cluster, follow either the Hawk2 or crmsh procedure below to avoid configuration errors.
8.6.1 Removing cluster resources with Hawk2 #
Log in to Hawk2:
https://HAWKSERVER:7630/
Clean up the resource on all nodes as described in Procedure 8.5, “Cleaning up a resource”.
Stop the resource:
From the left navigation bar, select
› . The list of also shows the .In the
column, click the button next to the resource.To continue, confirm the message that appears.
The
column reflects the change when the resource is stopped.
Delete the resource:
From the left navigation bar, select
› .In the list of
, go to the respective resource. From the column, click the icon next to the resource.To continue, confirm the message that appears.
8.6.2 Removing cluster resources with crmsh #
Log in as
root
and start thecrm
interactive shell:#
crm
Get a list of your resources:
crm(live)#
resource status
Full List of Resources: * admin-ip (ocf:heartbeat:IPaddr2): Started * stonith-sbd (stonith:external/sbd): Started * nfsserver (ocf:heartbeat:nfsserver): StartedStop the resource you want to remove:
crm(live)#
resource stop RESOURCE
Delete the resource:
crm(live)#
configure delete RESOURCE
8.7 Migrating cluster resources #
The cluster fails over (migrates) resources automatically during software or hardware failures, according to certain parameters you can define (for example, migration threshold or resource stickiness). You can also manually migrate a resource to another node in the cluster, or move the resource away from the current node and let the cluster decide where to put it.
You can migrate a cluster resource using either Hawk2 or crmsh.
8.7.1 Migrating cluster resources with Hawk2 #
Log in to Hawk2:
https://HAWKSERVER:7630/
From the left navigation bar, select
› . The list of also shows the .In the list of
, select the respective resource.In the
column, click the arrow down button and select .In the window that opens, you have the following choices:
-INFINITY
score for the current node.Alternatively, you can move the resource to another node. This creates a location constraint with an
INFINITY
score for the destination node.
Confirm your choice.
To allow a resource to move back again, proceed as follows:
Log in to Hawk2:
https://HAWKSERVER:7630/
From the left navigation bar, select
› . The list of also shows the .In the list of
, go to the respective resource.In the
column, click the arrow down button and select . To continue, confirm the message that appears.Hawk2 uses the
crm_resource
--clear
command. The resource can move back to its original location or it may stay where it is (depending on resource stickiness).
For more information, see Pacemaker Explained, available from https://www.clusterlabs.org/pacemaker/doc/. Refer to section Resource Migration.
8.7.2 Migrating cluster resources with crmsh #
Use the move
command for this task. For example,
to migrate the resource ipaddress1
to a cluster node
named bob
, use these
commands:
#
crm resource
crm(live)resource#
move ipaddress1 bob
8.8 Grouping resources by using tags #
Tags are a way to refer to multiple resources at once, without creating any
colocation or ordering relationship between them. This can be useful for
grouping conceptually related resources. For example, if you have
several resources related to a database, create a tag called
databases
and add all resources related to the
database to this tag. This allows you to stop or start them all with a
single command.
Tags can also be used in constraints. For example, the following
location constraint loc-db-prefer
applies to the set
of resources tagged with databases
:
location loc-db-prefer databases 100: alice
You can create tags using either Hawk2 or crmsh.
8.8.1 Grouping resources by using tags with Hawk2 #
Log in to Hawk2:
https://HAWKSERVER:7630/
From the left navigation bar, select
› › .Enter a unique
.From the
list, select the resources you want to refer to with the tag.Click
to finish the configuration. A message at the top of the screen shows if the action has been successful.
8.8.2 Grouping resources by using tags with crmsh #
For example, if you have
several resources related to a database, create a tag called
databases
and add all resources related to the
database to this tag:
#
crm configure tag databases: db1 db2 db3
This allows you to start them all with a single command:
#
crm resource start databases
Similarly, you can stop them all too:
#
crm resource stop databases