Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to SUSE OpenStack Cloud Monitoring

1 Introduction to SUSE OpenStack Cloud Monitoring

As more and more applications are deployed on cloud systems and cloud systems are growing in complexity, managing the cloud infrastructure is becoming increasingly difficult. SUSE OpenStack Cloud Monitoring helps mastering this challenge by providing a sophisticated Monitoring as a Service solution that is operated on top of OpenStack-based cloud computing platforms.

The component architecture of OpenStack provides for high flexibility, yet it increases the burden of system operation because multiple services must be handled. SUSE OpenStack Cloud Monitoring offers an integrated view of all services and assembles and presents related metrics and log data in one convenient access point. While being flexible and scalable to instantly reflect changes in the OpenStack platform, SUSE OpenStack Cloud Monitoring provides the ways and means required to ensure multi-tenancy, high availability, and data security.

SUSE OpenStack Cloud Monitoring covers all aspects of a Monitoring as a Service solution:

  • Central management of monitoring and log data from medium and large-size OpenStack deployments.

  • Storage of metrics and log data in a resilient way.

  • Multi-tenancy architecture to ensure the secure isolation of metrics and log data.

  • Horizontal and vertical scalability to support constantly evolving cloud infrastructures. When physical and virtual servers are scaled up or down to varying loads, the monitoring and log management solution can be adapted accordingly.

1.1 Basic Usage Scenario

The basic usage scenario of setting up and using the monitoring features of SUSE OpenStack Cloud Monitoring looks as follows:

MetricsLogs.png

An application operator acts as a service provider in the OpenStack environment. He books virtual machines to provide services to end users or to host services that he needs for his own development activities. SUSE OpenStack Cloud Monitoring helps application operators ensure that their services and the servers on which they are provided are configured and working as required.

As an OpenStack operator, you are a special application operator who is responsible for administrating and maintaining the underlying OpenStack platform. The monitoring and log management services of SUSE OpenStack Cloud Monitoring enable you to ensure the availability and quality of your platform. You use SUSE OpenStack Cloud Monitoring for:

  • Monitoring physical and virtual servers, hypervisors, and OpenStack services.

  • Monitoring middleware components, for example, database services.

  • Retrieving and analyzing the log data of the OpenStack services and servers, the middleware components, and the operating system.

The Monitoring Service operator is responsible for providing the monitoring and log management features to the application operators and the OpenStack operator. This enables them to focus on operation and the quality of their services and servers without having to carry out the tedious tasks implied by setting up and administrating their own monitoring software. The Monitoring Service operator uses the monitoring services himself for ensuring the quality of SUSE OpenStack Cloud Monitoring.

1.2 The OpenStack Operator's Tasks

In order to use SUSE OpenStack Cloud Monitoring for monitoring your OpenStack services and servers, so-called agents must be installed and configured:

  • Metrics Agent is required for monitoring your services and servers.

  • Log Agent is required for collecting the log data that is generated for your services and servers.

Monitoring

The Metrics agent is responsible for querying metrics and sending them to the Monitoring Service for further processing.

Metrics are self-describing data structures that are uniquely identified by a name and a set of dimensions. Each dimension consists of a key/value pair that allows for a flexible and concise description of the data to be monitored, for example, region, availability zone, service tier, or resource ID.

The Metrics Agent supports various types of metrics including the following:

  • System metrics, for example, CPU usage, consumed disk space, or network traffic.

  • Host alive checks. The agent can perform active checks on a host to determine whether it is alive using ping (ICMP) or SSH.

  • Process checks. The agent can check and monitor a process, for example, the number of instances, memory size, or number of threads.

  • HTTP endpoint checks. The agent can perform up/down checks on HTTP endpoints by sending an HTTP request and reporting success or failure to the Monitoring Service.

  • Service checks. The agent can check middleware services, for example, MySQL, Kafka, or RabbitMQ.

  • OpenStack services. The agent can perform specific checks on each process that is part of an OpenStack service.

  • Log metrics. The agent can check and monitor the number of critical log entries in the log data retrieved from the cloud resources.

Your individual agent configuration determines which metrics are available for monitoring your services and servers. For details on installing and configuring a Metrics Agent, refer to https://documentation.suse.com/soc/8/html/suse-openstack-cloud-crowbar-all/cha-depl-ostack.html#sec-depl-ostack-monasca.

As soon as an agent is available, you have access to the SUSE OpenStack Cloud Monitoring monitoring features. You work with a graphical user interface that is seamlessly integrated into your cloud infrastructure. Based on OpenStack Horizon, the user interface enables access to all monitoring functionality and the resulting large-scale monitoring data. A comfortable dashboard visualizes the health and status of your cloud resources.

SUSE OpenStack Cloud Monitoring provides functions for alarm and notification management. Template-based alarm definitions allow for monitoring a dynamically changing set of resources without the need for reconfiguration. While the number of underlying virtual machines is changing, for example, this ensures the efficient monitoring of scalable cloud services. Alarm definitions allow you to specify expressions that are evaluated based on the metrics data that is received. Alarm definitions can be combined to form compound alarms. Compound alarms allow you to track and process even more complex events. Notifications can be configured in order to inform SUSE OpenStack Cloud Monitoring users when an alarm is triggered.

For details on the monitoring functions, refer to Chapter 3, Monitoring.

Log Management

The Log Agent collects the log data from the services and servers and sends them to the Monitoring Service for further processing. For details on installing and configuring a Log Agent, refer to https://documentation.suse.com/soc/8/html/suse-openstack-cloud-crowbar-all/cha-depl-ostack.html#sec-depl-ostack-monasca.

SUSE OpenStack Cloud Monitoring stores the log data in a central database. This forms the basis for visualizing the log data for the SUSE OpenStack Cloud Monitoring users. Advanced data analysis and visualization of the log data is supported in a variety of charts, tables, and maps. Visualizations can easily be combined in dynamic dashboards that display changes to search queries in real time.

Based on OpenStack Horizon, the customizable dashboards are seamlessly integrated into your cloud infrastructure. They enable user access to all log management functionality.

GUI-based alarm and notification management is also supported for log data. Based on a template mechanism, you can configure alarms and notifications to monitor the number of critical log events over time. Compound alarms can be created to analyze more complex log events. This automation of log handling guarantees that you can identify problems in your their infrastructure early and find the root cause quickly.

For details on the log management functions, refer to Chapter 4, Log Management.

1.3 Components

The following illustration provides an overview of the main components of SUSE OpenStack Cloud Monitoring:

structure_new.png
OpenStack

SUSE OpenStack Cloud Monitoring relies on OpenStack as technology for building cloud computing platforms for public and private clouds. OpenStack consists of a series of interrelated projects delivering various components for a cloud infrastructure solution and allowing for the deployment and management of Infrastructure as a Service (IaaS) platforms.

Monitoring Service

The Monitoring Service is the central SUSE OpenStack Cloud Monitoring component. It is responsible for receiving, persisting, and processing metrics and log data, as well as providing the data to the users.

The Monitoring Service relies on Monasca. It uses Monasca for high-speed metrics querying and integrates the streaming alarm engine and the notification engine of Monasca. For details, refer to the Monasca Wiki (https://wiki.openstack.org/wiki/Monasca).

Horizon Plugin

SUSE OpenStack Cloud Monitoring comes with a plugin for the OpenStack Horizon dashboard. The plugin extends the main dashboard in OpenStack with a view for monitoring. This enables SUSE OpenStack Cloud Monitoring users to access the monitoring functions from a central Web-based graphical user interface. For details, refer to the OpenStack Horizon documentation (http://docs.openstack.org/developer/horizon/).

Based on OpenStack Horizon, the monitoring data is visualized on a comfortable and easy-to-use dashboard which fully integrates with the following applications:

  • Grafana (for metrics data). An open source application for visualizing large-scale measurement data.

  • Kibana (for log data). An open source analytics and visualization platform designed to work with Elasticsearch.

Metrics Agent

A Metrics Agent is required for retrieving metrics data from the host on which it runs and sending the metrics data to the Monitoring Service. The agent supports metrics from a variety of sources as well as a number of built-in system and service checks.

A Metrics Agent can be installed on each virtual or physical server to be monitored.

The agent functionality is fully integrated into the source code base of the Monasca project. For details, refer to the Monasca Wiki (https://wiki.openstack.org/wiki/Monasca).

Log Agent

A Log Agent is needed for collecting log data from the host on which it runs and forwarding the log data to the Monitoring Service for further processing. It can be installed on each virtual or physical server from which log data is to be retrieved.

The agent functionality is fully integrated into the source code base of the Monasca project. For details, refer to the Monasca Wiki (https://wiki.openstack.org/wiki/Monasca).

1.4 User Management

SUSE OpenStack Cloud Monitoring is fully integrated with Keystone, the identity service which serves as the common authentication and authorization system in OpenStack.

The SUSE OpenStack Cloud Monitoring integration with Keystone requires any SUSE OpenStack Cloud Monitoring user to be registered as an OpenStack user. All authentication and authorization in SUSE OpenStack Cloud Monitoring is done through Keystone. If a user requests monitoring data, for example, SUSE OpenStack Cloud Monitoring verifies that the user is a valid user in OpenStack and allowed to access the requested metrics.

SUSE OpenStack Cloud Monitoring users are created and administrated in OpenStack:

  • Each user assumes a role in OpenStack to perform a specific set of operations. The OpenStack role specifies a set of rights and privileges.

  • Each user is assigned to at least one project in OpenStack. A project is an organizational unit that defines a set of resources which can be accessed by the assigned users.

    Application operators in SUSE OpenStack Cloud Monitoring can monitor the set of resources that is defined for the projects to which they are assigned.

For details on user management, refer to the OpenStack documentation (http://docs.openstack.org/newton/).

Print this page