Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / SUSE Linux Enterprise High Availability Extension Documentation / Administration Guide / Configuration and Administration / Configuring and Managing Cluster Resources with Hawk2
Applies to SUSE Linux Enterprise High Availability Extension 15 SP2

6 Configuring and Managing Cluster Resources with Hawk2

To configure and manage cluster resources, either use Hawk2, or the crm shell (crmsh) command line utility. If you upgrade from an earlier version of SUSE® Linux Enterprise High Availability Extension where Hawk was installed, the package will be replaced with the current version, Hawk2.

Hawk2's user-friendly Web interface allows you to monitor and administer your High Availability clusters from Linux or non-Linux machines alike. Hawk2 can be accessed from any machine inside or outside of the cluster by using a (graphical) Web browser.

6.1 Hawk2 Requirements

Before users can log in to Hawk2, the following requirements need to be fulfilled:

hawk2 Package

The hawk2 package must be installed on all cluster nodes you want to connect to with Hawk2.

Web Browser

On the machine from which to access a cluster node using Hawk2, you need a (graphical) Web browser (with JavaScript and cookies enabled) to establish the connection.

Hawk2 Service

To use Hawk2, the respective Web service must be started on the node that you want to connect to via the Web interface. See Procedure 6.1, “Starting Hawk2 Services”.

If you have set up your cluster with the scripts from the ha-cluster-bootstrap package, the Hawk2 service is already enabled.

Username, Group and Password on Each Cluster Node

Hawk2 users must be members of the haclient group. The installation creates a Linux user named hacluster, who is added to the haclient group.

When using the ha-cluster-init script for setup, a default password is set for the hacluster user. Before starting Hawk2, change it to a secure password. If you did not use the ha-cluster-init script, either set a password for the hacluster first or create a new user which is a member of the haclient group. Do this on every node you will connect to with Hawk2.

Wildcard certificate handling

A wildcard certificate is a public key certificate which is valid for multiple sub-domains. For example, a wildcard certificate for *.example.com secures the domains www.example.com, login.example.com, and possibly more.

Hawk2 supports wildcard certificates as well as conventional certificates. A self-signed default private key and certificate is generated by /srv/www/hawk/bin/generate-ssl-cert.

To use your own certificate (conventional or wildcard), replace the generated certificate at /etc/ssl/certs/hawk.pem with your own.

Procedure 6.1: Starting Hawk2 Services
  1. On the node you want to connect to, open a shell and log in as root.

  2. Check the status of the service by entering

    # systemctl status hawk
  3. If the service is not running, start it with

    # systemctl start hawk

    If you want Hawk2 to start automatically at boot time, execute the following command:

    # systemctl enable hawk

6.2 Logging In

The Hawk2 Web interface uses the HTTPS protocol and port 7630.

Instead of logging in to an individual cluster node with Hawk2, you can configure a floating, virtual IP address (IPaddr or IPaddr2) as a cluster resource. It does not need any special configuration. It allows clients to connect to the Hawk service no matter which physical node the service is running on.

When setting up the cluster with the ha-cluster-bootstrap scripts, you will be asked whether to configure a virtual IP for cluster administration.

Procedure 6.2: Logging In to the Hawk2 Web Interface
  1. On any machine, start a Web browser and enter the following URL:

    https://HAWKSERVER:7630/

    Replace HAWKSERVER with the IP address or host name of any cluster node running the Hawk Web service. If a virtual IP address has been configured for cluster administration with Hawk2, replace HAWKSERVER with the virtual IP address.

    Note
    Note: Certificate Warning

    If a certificate warning appears when you try to access the URL for the first time, a self-signed certificate is in use. Self-signed certificates are not considered trustworthy by default.

    To verify the certificate, ask your cluster operator for the certificate details.

    To proceed anyway, you can add an exception in the browser to bypass the warning.

    For information on how to replace the self-signed certificate with a certificate signed by an official Certificate Authority, refer to Replacing the Self-Signed Certificate.

  2. On the Hawk2 login screen, enter the Username and Password of the hacluster user (or of any other user that is a member of the haclient group).

  3. Click Log In.

6.3 Hawk2 Overview: Main Elements

After logging in to Hawk2, you will see a navigation bar on the left-hand side and a top-level row with several links on the right-hand side.

Note
Note: Available Functions in Hawk2

By default, users logged in as root or hacluster have full read-write access to all cluster configuration tasks. However, Access Control Lists (ACLs) can be used to define fine-grained access permissions.

If ACLs are enabled in the CRM, the available functions in Hawk2 depend on the user role and their assigned access permissions. The History Explorer in Hawk2 can only be executed by the user hacluster.

6.3.1 Left Navigation Bar

Monitoring
  • Status: Displays the current cluster status at a glance (similar to crm status on the crmsh). For details, see Section 11.1.1, “Monitoring a Single Cluster”. If your cluster includes guest nodes (nodes that run the pacemaker_remote daemon), they are displayed, too. The screen refreshes in near real-time: any status changes for nodes or resources are visible almost immediately.

  • Dashboard: Allows you to monitor multiple clusters (also located on different sites, in case you have a Geo cluster setup). For details, see Section 11.1.2, “Monitoring Multiple Clusters”. If your cluster includes guest nodes (nodes that run the pacemaker_remote daemon), they are displayed, too. The screen refreshes in near real-time: any status changes for nodes or resources are visible almost immediately.

Troubleshooting
Configuration

6.3.2 Top-Level Row

Hawk2's top-level row shows the following entries:

  • Batch: Click to switch to batch mode. This allows you to simulate and stage changes and to apply them as a single transaction. For details, see Section 6.7, “Using the Batch Mode”.

  • USERNAME: Allows you to set preferences for Hawk2 (for example, the language for the Web interface, or whether to display a warning if STONITH is disabled).

  • Help: Access the SUSE Linux Enterprise High Availability Extension documentation, read the release notes or report a bug.

  • Logout: Click to log out.

6.4 Configuring Global Cluster Options

Global cluster options control how the cluster behaves when confronted with certain situations. They are grouped into sets and can be viewed and modified with cluster management tools like Hawk2 and crmsh. The predefined values can usually be kept. However, to ensure the key functions of your cluster work correctly, you need to adjust the following parameters after basic cluster setup:

Procedure 6.3: Modifying Global Cluster Options
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Cluster Configuration.

    The Cluster Configuration screen opens. It displays the global cluster options and their current values.

    To display a short description of the parameter on the right-hand side of the screen, hover your mouse over a parameter.

    Hawk2—Cluster Configuration
    Figure 6.1: Hawk2—Cluster Configuration
  3. Check the values for no-quorum-policy and stonith-enabled and adjust them, if necessary:

    1. Set no-quorum-policy to the appropriate value. See Section 5.2.2, “Global Option no-quorum-policy for more details.

    2. If you need to disable fencing for any reason, set stonith-enabled to no. By default, it is set to true, because using STONITH devices is necessary for normal cluster operation. According to the default value, the cluster will refuse to start any resources if no STONITH resources have been configured.

      Important
      Important: No Support Without STONITH
      • You must have a node fencing mechanism for your cluster.

      • The global cluster options stonith-enabled and startup-fencing must be set to true. When you change them, you lose support.

    3. To remove a parameter from the cluster configuration, click the Minus icon next to the parameter. If a parameter is deleted, the cluster will behave as if that parameter had the default value.

    4. To add a new parameter to the cluster configuration, choose one from the drop-down box.

  4. If you need to change Resource Defaults or Operation Defaults, proceed as follows:

    1. To adjust a value, either select a different value from the drop-down box or edit the value directly.

    2. To add a new resource default or operation default, choose one from the empty drop-down box and enter a value. If there are default values, Hawk2 proposes them automatically.

    3. To remove a parameter, click the Minus icon next to it. If no values are specified for Resource Defaults and Operation Defaults, the cluster uses the default values that are documented in Section 5.3.6, “Resource Options (Meta Attributes)” and Section 5.3.8, “Resource Operations”.

  5. Confirm your changes.

6.5 Configuring Cluster Resources

A cluster administrator needs to create cluster resources for every resource or application that runs on the servers in your cluster. Cluster resources can include Web sites, mail servers, databases, file systems, virtual machines, and any other server-based applications or services you want to make available to users at all times.

For an overview of the resource types you can create, refer to Section 5.3.3, “Types of Resources”. After you have specified the resource basics (ID, class, provider, and type), Hawk2 shows the following categories:

Parameters (Instance Attributes)

Determines which instance of a service the resource controls. For more information, refer to Section 5.3.7, “Instance Attributes (Parameters)”.

When creating a resource, Hawk2 automatically shows any required parameters. Edit them to get a valid resource configuration.

Operations

Needed for resource monitoring. For more information, refer to Section 5.3.8, “Resource Operations”.

When creating a resource, Hawk2 displays the most important resource operations (monitor, start, and stop).

Meta Attributes

Tells the CRM how to treat a specific resource. For more information, refer to Section 5.3.6, “Resource Options (Meta Attributes)”.

When creating a resource, Hawk2 automatically lists the important meta attributes for that resource (for example, the target-role attribute that defines the initial state of a resource. By default, it is set to Stopped, so the resource will not start immediately).

Utilization

Tells the CRM what capacity a certain resource requires from a node. For more information, refer to Section 8.10.1, “Placing Resources Based on Their Load Impact with Hawk2”.

You can adjust the entries and values in those categories either during resource creation or later.

6.5.1 Showing the Current Cluster Configuration (CIB)

Sometimes a cluster administrator needs to know the cluster configuration. Hawk2 can show the current configuration in crm shell syntax, as XML and as a graph. To view the cluster configuration in crm shell syntax, from the left navigation bar select Configuration › Edit Configuration and click Show. To show the configuration in raw XML instead, click XML. Click Graph for a graphical representation of the nodes and resources configured in the CIB. It also shows the relationships between resources.

6.5.2 Adding Resources with the Wizard

The Hawk2 wizard is a convenient way of setting up simple resources like a virtual IP address or an SBD STONITH resource, for example. It is also useful for complex configurations that include multiple resources, like the resource configuration for a DRBD block device or an Apache Web server. The wizard guides you through the configuration steps and provides information about the parameters you need to enter.

Procedure 6.4: Using the Resource Wizard
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Wizards.

  3. Expand the individual categories by clicking the arrow down icon next to them and select the desired wizard.

  4. Follow the instructions on the screen. After the last configuration step, Verify the values you have entered.

    Hawk2 shows which actions it is going to perform and what the configuration looks like. Depending on the configuration, you might be prompted for the root password before you can Apply the configuration.

Hawk2—Wizard for Apache Web Server
Figure 6.2: Hawk2—Wizard for Apache Web Server

6.5.3 Adding Simple Resources

To create the most basic type of resource, proceed as follows:

Procedure 6.5: Adding a Primitive Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Add Resource › Primitive.

  3. Enter a unique Resource ID.

  4. In case a resource template exists on which you want to base the resource configuration, select the respective Template. For details about configuring templates, see Procedure 6.6, “Adding a Resource Template”.

  5. Select the resource agent Class you want to use: lsb, ocf, service, stonith, or systemd. For more information, see Section 5.3.2, “Supported Resource Agent Classes”.

  6. If you selected ocf as class, specify the Provider of your OCF resource agent. The OCF specification allows multiple vendors to supply the same resource agent.

  7. From the Type list, select the resource agent you want to use (for example, IPaddr or Filesystem). A short description for this resource agent is displayed.

    With that, you have specified the resource basics.

    Note
    Note

    The selection you get in the Type list depends on the Class (and for OCF resources also on the Provider) you have chosen.

    Hawk2—Primitive Resource
    Figure 6.3: Hawk2—Primitive Resource
  8. To keep the Parameters, Operations, and Meta Attributes as suggested by Hawk2, click Create to finish the configuration. A message at the top of the screen shows if the action has been successful.

    To adjust the parameters, operations, or meta attributes, refer to Section 6.5.5, “Modifying Resources”. To configure Utilization attributes for the resource, see Procedure 8.8, “Configuring the Capacity a Resource Requires”.

6.5.4 Adding Resource Templates

To create lots of resources with similar configurations, defining a resource template is the easiest way. After being defined, it can be referenced in primitives or in certain types of constraints. For detailed information about function and use of resource templates, refer to Section 8.3, “Resource Templates and Constraints”.

Procedure 6.6: Adding a Resource Template

Resource templates are configured like primitive resources.

  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Add Resource › Template.

  3. Enter a unique Resource ID.

  4. Follow the instructions in Procedure 6.5, “Adding a Primitive Resource”, starting from Step 5.

6.5.5 Modifying Resources

If you have created a resource, you can edit its configuration at any time by adjusting parameters, operations, or meta attributes as needed.

Procedure 6.7: Modifying Parameters, Operations, or Meta Attributes for a Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. On the Hawk2 Status screen, go to the Resources list.

  3. In the Operations column, click the arrow down icon next to the resource or group you want to modify and select Edit.

    The resource configuration screen opens.

    Hawk2—Editing A Primitive Resource
    Figure 6.4: Hawk2—Editing A Primitive Resource
  4. To add a new parameter, operation, or meta attribute, select an entry from the empty drop-down box.

  5. To edit any values in the Operations category, click the Edit icon of the respective entry, enter a different value for the operation, and click Apply.

  6. When you are finished, click the Apply button in the resource configuration screen to confirm your changes to the parameters, operations, or meta attributes.

    A message at the top of the screen shows if the action has been successful.

6.5.6 Adding STONITH Resources

Important
Important: No Support Without STONITH
  • You must have a node fencing mechanism for your cluster.

  • The global cluster options stonith-enabled and startup-fencing must be set to true. When you change them, you lose support.

By default, the global cluster option stonith-enabled is set to true. If no STONITH resources have been defined, the cluster will refuse to start any resources. Configure one or more STONITH resources to complete the STONITH setup. To add a STONITH resource for SBD, for libvirt (KVM/Xen) or for vCenter/ESX Server, the easiest way is to use the Hawk2 wizard (see Section 6.5.2, “Adding Resources with the Wizard”). While STONITH resources are configured similarly to other resources, their behavior is different in some respects. For details refer to Section 12.3, “STONITH Resources and Configuration”.

Procedure 6.8: Adding a STONITH Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Add Resource › Primitive.

  3. Enter a unique Resource ID.

  4. From the Class list, select the resource agent class stonith.

  5. From the Type list, select the STONITH plug-in to control your STONITH device. A short description for this plug-in is displayed.

  6. Hawk2 automatically shows the required Parameters for the resource. Enter values for each parameter.

  7. Hawk2 displays the most important resource Operations and proposes default values. If you do not modify any settings here, Hawk2 adds the proposed operations and their default values when you confirm.

  8. If there is no reason to change them, keep the default Meta Attributes settings.

    Hawk2—STONITH Resource
    Figure 6.5: Hawk2—STONITH Resource
  9. Confirm your changes to create the STONITH resource.

    A message at the top of the screen shows if the action has been successful.

To complete your fencing configuration, add constraints. For more details, refer to Chapter 12, Fencing and STONITH.

6.5.7 Adding Cluster Resource Groups

Some cluster resources depend on other components or resources. They require that each component or resource starts in a specific order and runs on the same server. To simplify this configuration SUSE Linux Enterprise High Availability Extension supports the concept of groups.

Resource groups contain a set of resources that need to be located together, be started sequentially and stopped in the reverse order. For an example of a resource group and more information about groups and their properties, refer to Section 5.3.5.1, “Groups”.

Note
Note: Empty Groups

Groups must contain at least one resource, otherwise the configuration is not valid. While creating a group, Hawk2 allows you to create more primitives and add them to the group. For details, see Section 6.6.1, “Editing Resources and Groups”.

Procedure 6.9: Adding a Resource Group
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Add Resource › Group.

  3. Enter a unique Group ID.

  4. To define the group members, select one or multiple entries in the list of Children. Re-sort group members by dragging and dropping them into the order you want by using the handle icon on the right.

  5. If needed, modify or add Meta Attributes.

  6. Click Create to finish the configuration. A message at the top of the screen shows if the action has been successful.

Hawk2—Resource Group
Figure 6.6: Hawk2—Resource Group

6.5.8 Adding Clone Resources

If you want certain resources to run simultaneously on multiple nodes in your cluster, configure these resources as clones. An example of a resource that can be configured as a clone is ocf:pacemaker:controld for cluster file systems like OCFS2. Any regular resources or resource groups can be cloned. Instances of cloned resources may behave identically. However, they may also be configured differently, depending on which node they are hosted on.

For an overview of the available types of resource clones, refer to Section 5.3.5.2, “Clones”.

Note
Note: Child Resources for Clones

Clones can either contain a primitive or a group as child resources. In Hawk2, child resources cannot be created or modified while creating a clone. Before adding a clone, create child resources and configure them as desired. For details, refer to Section 6.5.3, “Adding Simple Resources” or Section 6.5.7, “Adding Cluster Resource Groups”.

Procedure 6.10: Adding a Clone Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Add Resource › Clone.

  3. Enter a unique Clone ID.

  4. From the Child Resource list, select the primitive or group to use as a sub-resource for the clone.

  5. If needed, modify or add Meta Attributes.

  6. Click Create to finish the configuration. A message at the top of the screen shows if the action has been successful.

Hawk2—Clone Resource
Figure 6.7: Hawk2—Clone Resource

6.5.9 Adding Multi-state Resources

Multi-state resources are a specialization of clones. They allow the instances to be in one of two operating modes (called active/passive, primary/secondary, or master/slave). Multi-state resources must contain exactly one group or one regular resource.

When configuring resource monitoring or constraints, multi-state resources have different requirements than simple resources. For details, see Pacemaker Explained, available from http://www.clusterlabs.org/pacemaker/doc/. Refer to section Multi-state - Resources That Have Multiple Modes.

Note
Note: Child Resources for Multi-state Resources

Multi-state resources can either contain a primitive or a group as child resources. In Hawk2, child resources cannot be created or modified while creating a multi-state resource. Before adding a multi-state resource, create child resources and configure them as desired. For details, refer to Section 6.5.3, “Adding Simple Resources” or Section 6.5.7, “Adding Cluster Resource Groups”.

Procedure 6.11: Adding a Multi-state Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Add Resource › Multi-state.

  3. Enter a unique Multi-state ID.

  4. From the Child Resource list, select the primitive or group to use as a sub-resource for the multi-state resource.

  5. If needed, modify or add Meta Attributes.

  6. Click Create to finish the configuration. A message at the top of the screen shows if the action has been successful.

Hawk2—Multi-state Resource
Figure 6.8: Hawk2—Multi-state Resource

6.5.10 Grouping Resources by Using Tags

Tags are a way to refer to multiple resources at once, without creating any colocation or ordering relationship between them. You can use tags for grouping conceptually related resources. For example, if you have several resources related to a database, you can add all related resources to a tag named database.

All resources belonging to a tag can be started or stopped with a single command.

Procedure 6.12: Adding a Tag
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Configuration › Add Resource › Tag.

  3. Enter a unique Tag ID.

  4. From the Objects list, select the resources you want to refer to with the tag.

  5. Click Create to finish the configuration. A message at the top of the screen shows if the action has been successful.

Hawk2—Tag
Figure 6.9: Hawk2—Tag

6.5.11 Configuring Resource Monitoring

The High Availability Extension does not only detect node failures, but also when an individual resource on a node has failed. If you want to ensure that a resource is running, configure resource monitoring for it. Usually, resources are only monitored by the cluster while they are running. However, to detect concurrency violations, also configure monitoring for resources which are stopped. For resource monitoring, specify a timeout and/or start delay value, and an interval. The interval tells the CRM how often it should check the resource status. You can also set particular parameters such as timeout for start or stop operations.

Procedure 6.13: Adding and Modifying an Operation
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. Add a resource as described in Procedure 6.5, “Adding a Primitive Resource” or select an existing primitive to edit.

    Hawk2 automatically shows the most important Operations (start, stop, monitor) and proposes default values.

    To see the attributes belonging to each proposed value, hover the mouse pointer over the respective value.

    Image
  3. To change the suggested timeout values for the start or stop operation:

    1. Click the pen icon next to the operation.

    2. In the dialog that opens, enter a different value for the timeout parameter, for example 10, and confirm your change.

  4. To change the suggested interval value for the monitor operation:

    1. Click the pen icon next to the operation.

    2. In the dialog that opens, enter a different value for the monitoring interval.

    3. To configure resource monitoring in the case that the resource is stopped:

      1. Select the role entry from the empty drop-down box below.

      2. From the role drop-down box, select Stopped.

      3. Click Apply to confirm your changes and to close the dialog for the operation.

  5. Confirm your changes in the resource configuration screen. A message at the top of the screen shows if the action has been successful.

For the processes that take place if the resource monitor detects a failure, refer to Section 5.4, “Resource Monitoring”.

To view resource failures, switch to the Status screen in Hawk2 and select the resource you are interested in. In the Operations column click the arrow down icon and select Recent Events. The dialog that opens lists recent actions performed for the resource. Failures are displayed in red. To view the resource details, click the magnifier icon in the Operations column.

Hawk2—Resource Details
Figure 6.10: Hawk2—Resource Details

6.6 Managing Cluster Resources

In addition to configuring your cluster resources, Hawk2 allows you to manage existing resources from the Status screen. For a general overview of the screen refer to Section 11.1.1, “Monitoring a Single Cluster”.

6.6.1 Editing Resources and Groups

In case you need to edit existing resources, go to the Status screen. In the Operations column, click the arrow down icon next to the resource or group you want to modify and select Edit.

The editing screen appears. If you edit a primitive resource, the following operations are available:

Operations for Primitives
  • Copying the resource.

  • Renaming the resource (changing its ID).

  • Deleting the resource.

If you edit a group, the following operations are available:

Operations for Groups
  • Creating a new primitive which will be added to this group.

  • Renaming the group (changing its ID).

  • Re-sort group members by dragging and dropping them into the order you want using the handle icon on the right.

6.6.2 Starting Resources

Before you start a cluster resource, make sure it is set up correctly. For example, if you use an Apache server as a cluster resource, set up the Apache server first. Complete the Apache configuration before starting the respective resource in your cluster.

Note
Note: Do Not Touch Services Managed by the Cluster

When managing a resource via the High Availability Extension, the resource must not be started or stopped otherwise (outside of the cluster, for example manually or on boot or reboot). The High Availability Extension software is responsible for all service start or stop actions.

However, if you want to check if the service is configured properly, start it manually, but make sure that it is stopped again before the High Availability Extension takes over.

For interventions in resources that are currently managed by the cluster, set the resource to maintenance mode first. For details, see Procedure 27.5, “Putting a Resource into Maintenance Mode with Hawk2”.

When creating a resource with Hawk2, you can set its initial state with the target-role meta attribute. If you set its value to stopped, the resource does not start automatically after being created.

Procedure 6.14: Starting A New Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Monitoring › Status. The list of Resources also shows the Status.

  3. Select the resource to start. In its Operations column click the Start icon. To continue, confirm the message that appears.

    When the resource has started, Hawk2 changes the resource's Status to green and shows on which node it is running.

6.6.3 Cleaning Up Resources

A resource will be automatically restarted if it fails, but each failure increases the resource's failcount.

If a migration-threshold has been set for the resource, the node will no longer run the resource when the number of failures reaches the migration threshold.

A resource's failcount can either be reset automatically (by setting a failure-timeout option for the resource) or it can be reset manually as described below.

Procedure 6.15: Cleaning Up A Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Status. The list of Resources also shows the Status.

  3. Go to the resource to clean up. In the Operations column click the arrow down button and select Cleanup. To continue, confirm the message that appears.

    This executes the command crm resource cleanup and cleans up the resource on all nodes.

6.6.4 Removing Cluster Resources

If you need to remove a resource from the cluster, follow the procedure below to avoid configuration errors:

Procedure 6.16: Removing a Cluster Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. Clean up the resource on all nodes as described in Procedure 6.15, “Cleaning Up A Resource”.

  3. Stop the resource:

    1. From the left navigation bar, select Monitoring › Status. The list of Resources also shows the Status.

    2. In the Operations column click the Stop button next to the resource.

    3. To continue, confirm the message that appears.

      The Status column will reflect the change when the resource is stopped.

  4. Delete the resource:

    1. From the left navigation bar, select Configuration › Edit Configuration.

    2. In the list of Resources, go to the respective resource. From the Operations column click the Delete icon next to the resource.

    3. To continue, confirm the message that appears.

6.6.5 Migrating Cluster Resources

As mentioned in Section 8.8.1, “Specifying Resource Failover Nodes with Hawk2”, the cluster will fail over (migrate) resources automatically in case of software or hardware failures—according to certain parameters you can define (for example, migration threshold or resource stickiness). You can also manually migrate a resource to another node in the cluster. Or you decide to move the resource away from the current node and let the cluster decide where to put it.

Procedure 6.17: Manually Migrating a Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Monitoring › Status. The list of Resources also shows the Status.

  3. In the list of Resources, select the respective resource.

  4. In the Operations column click the arrow down button and select Migrate.

  5. In the window that opens you have the following choices:

    • Away from current node: This creates a location constraint with a -INFINITY score for the current node.

    • Alternatively, you can move the resource to another node. This creates a location constraint with an INFINITY score for the destination node.

  6. Confirm your choice.

To allow a resource to move back again, proceed as follows:

Procedure 6.18: Unmigrating a Resource
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. From the left navigation bar, select Monitoring › Status. The list of Resources also shows the Status.

  3. In the list of Resources, go to the respective resource.

  4. In the Operations column click the arrow down button and select Clear. To continue, confirm the message that appears.

    Hawk2 uses the crm_resource  --clear command. The resource can move back to its original location or it may stay where it is (depending on resource stickiness).

For more information, see Pacemaker Explained, available from http://www.clusterlabs.org/pacemaker/doc/. Refer to section Resource Migration.

6.7 Using the Batch Mode

Hawk2 provides a Batch Mode, including a cluster simulator. It can be used for the following:

  • Staging changes to the cluster and applying them as a single transaction, instead of having each change take effect immediately.

  • Simulating changes and cluster events, for example, to explore potential failure scenarios.

For example, batch mode can be used when creating groups of resources that depend on each other. Using batch mode, you can avoid applying intermediate or incomplete configurations to the cluster.

While batch mode is enabled, you can add or edit resources and constraints or change the cluster configuration. It is also possible to simulate events in the cluster, including nodes going online or offline, resource operations and tickets being granted or revoked. See Procedure 6.20, “Injecting Node, Resource or Ticket Events” for details.

The cluster simulator runs automatically after every change and shows the expected outcome in the user interface. For example, this also means: If you stop a resource while in batch mode, the user interface shows the resource as stopped—while actually, the resource is still running.

Important
Important: Wizards and Changes to the Live System

Some wizards include actions beyond mere cluster configuration. When using those wizards in batch mode, any changes that go beyond cluster configuration would be applied to the live system immediately.

Therefore wizards that require root permission cannot be executed in batch mode.

Procedure 6.19: Working with the Batch Mode
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. To activate the batch mode, select Batch from the top-level row.

    An additional bar appears below the top-level row. It indicates that batch mode is active and contains links to actions that you can execute in batch mode.

    Hawk2 Batch Mode Activated
    Figure 6.11: Hawk2 Batch Mode Activated
  3. While batch mode is active, perform any changes to your cluster, like adding or editing resources and constraints or editing the cluster configuration.

    The changes will be simulated and shown in all screens.

  4. To view details of the changes you have made, select Show from the batch mode bar. The Batch Mode window opens.

    For any configuration changes it shows the difference between the live state and the simulated changes in crmsh syntax: Lines starting with a - character represent the current state whereas lines starting with + show the proposed state.

  5. To inject events or view even more details, see Procedure 6.20. Otherwise Close the window.

  6. Choose to either Discard or Apply the simulated changes and confirm your choice. This also deactivates batch mode and takes you back to normal mode.

When running in batch mode, Hawk2 also allows you to inject Node Events and Resource Events.

Node Events

Let you change the state of a node. Available states are online, offline, and unclean.

Resource Events

Let you change some properties of a resource. For example, you can set an operation (like start, stop, monitor), the node it applies to, and the expected result to be simulated.

Ticket Events

Let you test the impact of granting and revoking tickets (used for Geo clusters).

Procedure 6.20: Injecting Node, Resource or Ticket Events
  1. Log in to Hawk2:

    https://HAWKSERVER:7630/
  2. If batch mode is not active yet, click Batch at the top-level row to switch to batch mode.

  3. In the batch mode bar, click Show to open the Batch Mode window.

  4. To simulate a status change of a node:

    1. Click Inject › Node Event.

    2. Select the Node you want to manipulate and select its target State.

    3. Confirm your changes. Your event is added to the queue of events listed in the Batch Mode dialog.

  5. To simulate a resource operation:

    1. Click Inject › Resource Event.

    2. Select the Resource you want to manipulate and select the Operation to simulate.

    3. If necessary, define an Interval.

    4. Select the Node on which to run the operation and the targeted Result. Your event is added to the queue of events listed in the Batch Mode dialog.

    5. Confirm your changes.

  6. To simulate a ticket action:

    1. Click Inject › Ticket Event.

    2. Select the Ticket you want to manipulate and select the Action to simulate.

    3. Confirm your changes. Your event is added to the queue of events listed in the Batch Mode dialog.

  7. The Batch Mode dialog (Figure 6.12) shows a new line per injected event. Any event listed here is simulated immediately and is reflected on the Status screen.

    If you have made any configuration changes, too, the difference between the live state and the simulated changes is shown below the injected events.

    Hawk2 Batch Mode—Injected Invents and Configuration Changes
    Figure 6.12: Hawk2 Batch Mode—Injected Invents and Configuration Changes
  8. To remove an injected event, click the Remove icon next to it. Hawk2 updates the Status screen accordingly.

  9. To view more details about the simulation run, click Simulator and choose one of the following:

    Summary

    Shows a detailed summary.

    CIB (in)/CIB (out)

    CIB (in) shows the initial CIB state. CIB (out) shows what the CIB would look like after the transition.

    Transition Graph

    Shows a graphical representation of the transition.

    Transition

    Shows an XML representation of the transition.

  10. If you have reviewed the simulated changes, close the Batch Mode window.

  11. To leave the batch mode, either Apply or Discard the simulated changes.