7 Prometheus integration #

Prometheus Server is a Trento Server component responsible for retrieving host metrics from the agent hosts and serving them to the web component. The web component renders some of these metrics as graphical charts in the details view of the registered hosts. Particularly: memory, CPU, filesystem and swap utilization. Note that metrics are stored in UTC in Prometheus Server; the web UI converts and renders chart axes and hover timestamps using each user’s preferred timezone.

7.1 Metrics collection overview #

Trento supports two methods for collecting system metrics, depending on the operating system version running on your monitored hosts:

SUSE Linux Enterprise Server for SAP applications 15: Uses node_exporter in pull mode by default. Prometheus Server periodically scrapes metrics from each host’s node_exporter endpoint.
SUSE Linux Enterprise Server for SAP applications 16: Uses Grafana Alloy in push mode. Alloy collects metrics locally and pushes them to the Prometheus remote write endpoint.

Note

SUSE Linux Enterprise Server for SAP applications 15 SP7 also provides Grafana Alloy as an optional package. Users who prefer to use Alloy on SUSE Linux Enterprise Server for SAP applications 15 SP7 can manually install and configure it following the same procedure described for SUSE Linux Enterprise Server for SAP applications 16.

Note

In environments where at least one host uses Grafana Alloy (either SUSE Linux Enterprise Server for SAP applications 16 or SUSE Linux Enterprise Server for SAP applications 15 SP7 with Alloy manually configured), you must enable the Prometheus remote write endpoint. See Section 7.6, “Enabling Prometheus remote write for Grafana Alloy hosts” for details.

7.2 Requirements for SUSE Linux Enterprise Server for SAP applications 15 hosts (pull mode) #

The node_exporter must be installed and running in the agent hosts and Prometheus Server must be able to reach the agent hosts at the IP address and port that the node_exporter is bound to.

The IP address and port that Prometheus Server uses to reach the node_exporter can be changed by setting parameter prometheus-node-exporter-target with value <ip_address>:<port> in the agent configuration file. The legacy parameter name node-exporter-target is still supported for backward compatibility.

If the parameter is not set, Prometheus Server uses the lowest IPv4 address discovered in the host with default port 9100.

7.3 Requirements for SUSE Linux Enterprise Server for SAP applications 16 hosts (push mode) #

SUSE Linux Enterprise Server for SAP applications 16 uses Grafana Alloy instead of node_exporter for metrics collection. In this mode:

Grafana Alloy must be installed and running on the agent hosts.
The Prometheus remote write endpoint must be enabled on Trento Server.
The Trento Agent generates the Alloy configuration using the trento-agent generate alloy command.

See Section 7.7, “Configuring Grafana Alloy” for detailed setup instructions.

7.4 Kubernetes deployment #

To enable Prometheus alongside Trento Server in a Kubernetes cluster, see Section 7.6, “Enabling Prometheus remote write for Grafana Alloy hosts”.

In a Kubernetes cluster with multiple nodes, the user can select on which node to deploy Prometheus Server by adding the following flag to the Helm installation command:

--set prometheus.server.nodeSelector.LABEL=<value>

Where <value> is the label assigned to the node where the user wants Prometheus Server to be deployed.

7.5 systemd deployment #

In a systemd deployment of Trento Server, you can choose between using an existing installation of Prometheus Server, installing a dedicated Prometheus Server instance, or not using Prometheus Server at all.

7.5.1 Use an existing installation #

If you already have an existing Prometheus Server that you want to use with Trento Server, you must set CHARTS_ENABLED=true and PROMETHEUS_URL pointing to the right address and port in the Trento Web configuration file. You must restart restart the Trento Web service to enable the changes.

The lowest required Prometheus Server version is 2.28.0.

Use the section Install Prometheus on SUSE Linux Enterprise Server for SAP applications 16.0 as a reference to adjust the Prometheus Server configuration.

7.5.2 Install Prometheus Server from SUSE Package Hub on SUSE Linux Enterprise Server for SAP applications 15.x #

SUSE Package Hub packages are tested by SUSE but are not officially supported as part of the SUSE Linux Enterprise Server for SAP applications base product. Users should assess the suitability of these packages based on their own risk tolerance and support needs.

Enable the SUSE Package Hub repository (replace 15.x with the version of your operating system, for example 15.7):

SUSEConnect --product PackageHub/15.x/x86_64
zypper refresh

Now proceed with the same steps as in Install Prometheus on SUSE Linux Enterprise Server for SAP applications 16.0.

7.5.3 Install Prometheus on SUSE Linux Enterprise Server for SAP applications 16.0 #

Create the Prometheus user and group:

groupadd --system prometheus
useradd -s /sbin/nologin --system -g prometheus prometheus

Install Prometheus using Zypper:

zypper in golang-github-prometheus-prometheus

Configure Prometheus for Trento by replacing or updating the existing configuration at /etc/prometheus/prometheus.yml with:

global:
  scrape_interval: 30s
  evaluation_interval: 10s

scrape_configs:
  - job_name: "http_sd_hosts"
    honor_timestamps: true
    scrape_interval: 30s
    scrape_timeout: 30s
    scheme: http
    follow_redirects: true
    http_sd_configs:
      - follow_redirects: true
        refresh_interval: 1m
        url: http://localhost:4000/api/prometheus/targets

Note: the value of the url parameter above assumes that the Trento Web service is running in the same host as Prometheus Server.

Enable and start the Prometheus service:
```
systemctl enable --now prometheus
```
If firewalld is running, allow Prometheus to be accessible and add an exception to firewalld:
```
firewall-cmd --zone=public --add-port=9090/tcp --permanent
firewall-cmd --reload
```
Set CHARTS_ENABLED=true and PROMETHEUS_URL=http://localhost:9090 in the Trento Web configuration file and restart the Trento Web service.
```
systemctl restart trento-web
```
Note: the value of the PROMETHEUS_URL parameter above assumes that the Trento Web service is running in the same host as Prometheus Server.

7.5.4 Not using Prometheus Server #

If you decide not to use Prometheus Server in your Trento installation, you must disable graphical charts in the UI by setting CHARTS_ENABLED=false in the Trento Web configuration file.

7.6 Enabling Prometheus remote write for Grafana Alloy hosts #

If your environment includes at least one host using Grafana Alloy for metrics collection (either SUSE Linux Enterprise Server for SAP applications 16 or SUSE Linux Enterprise Server for SAP applications 15 SP7 with Alloy manually configured), you must enable the Prometheus remote write endpoint to receive pushed metrics.

The authentication type must be explicitly configured during installation or upgrade. Both the Helm chart and the Ansible playbook fail if the authentication type is left unconfigured, preventing an unprotected remote write endpoint from being silently exposed.

7.6.1 Kubernetes deployment #

The Helm chart enables the Prometheus remote write receiver and deploys an NGINX sidecar container in the Prometheus pod to handle authentication. The Kubernetes Ingress routes traffic to the NGINX sidecar (port 8081) instead of directly to Prometheus (port 9090).

The NGINX sidecar protects the /prometheus/api/v1/write endpoint with HTTP Basic Auth. Other Prometheus routes (UI and query API) remain accessible without authentication.

To enable Prometheus with basic authentication, replace flag --set prometheus.enabled=false in the Helm installation command with the following:

--set prometheus.enabled=true \
--set prometheus.server.auth.type=basic \
--set prometheus.server.auth.basic.username=<username> \
--set prometheus.server.auth.basic.password=<password>

The auth.type field must be explicitly set to basic or none. An empty value causes the chart to fail validation.

To explicitly disable authentication (not recommended for production):

--set prometheus.server.auth.type=none

Warning

Running without authentication allows anyone with network access to push arbitrary metrics to Prometheus.

7.6.2 Ansible deployment #

The Ansible playbook configures basic authentication on the existing NGINX reverse proxy. The reverse proxy terminates TLS and enforces basic auth on the Prometheus write endpoint.

Set the following inventory variables:

all:
  vars:
    trento_prometheus_auth: "basic"
    trento_prometheus_auth_username: <username>
    trento_prometheus_auth_password: <password>

The trento_prometheus_auth variable must be explicitly set to basic or none. The playbook fails if Prometheus is enabled and this variable is not set.

When set to basic, the playbook:

Validates that both username and password are provided
Generates an htpasswd file at /etc/nginx/prometheus.htpasswd
Configures the NGINX reverse proxy to enforce basic authentication on the /prometheus/api/v1/write endpoint

These credentials propagate automatically to the agent role, so agents deployed by the same playbook are configured to authenticate against the Prometheus endpoint without additional setup.

To explicitly disable authentication (not recommended for production):

all:
  vars:
    trento_prometheus_auth: "none"

Warning

Running without authentication allows anyone with network access to push arbitrary metrics to Prometheus.

7.6.3 Custom deployments #

For custom deployments not using the official Helm chart or Ansible playbook, the Prometheus remote write endpoint must be enabled and exposed manually. The procedure below walks through how to add an authenticated /prometheus/ endpoint to an existing Trento Server installation that already has NGINX running as a TLS-terminating reverse proxy in front of the Trento Web and Trento Wanda services.

Grafana Alloy supports basic authentication, bearer token, and mutual TLS (mTLS) when pushing metrics. Configure your reverse proxy to validate credentials using the method of your choice. The example below uses HTTP basic authentication.

7.6.3.1 Enable the Prometheus remote write receiver #

Prometheus must be started with the --web.enable-remote-write-receiver flag. How to set this flag depends on how Prometheus is managed on your platform (for example, via a sysconfig file, a systemd drop-in, or a command-line argument). In a sysconfig file scenario, add ARGS="--web.enable-remote-write-receiver" to the /etc/sysconfig/prometheus configuration file.

After enabling the flag, restart Prometheus and confirm it is healthy:

systemctl restart prometheus
systemctl status prometheus --no-pager

7.6.3.2 Create the htpasswd file for basic authentication #

Generate the password hash with openssl passwd -apr1:

PROM_USER='prometheus'
PROM_PASS='<choose a strong password>'

printf '%s:%s\n' "$PROM_USER" \
  "$(openssl passwd -apr1 "$PROM_PASS")" \
  > /etc/nginx/prometheus.htpasswd

chown root:nginx /etc/nginx/prometheus.htpasswd
chmod 0640       /etc/nginx/prometheus.htpasswd

Verify that the file contains exactly one line of the form prometheus:$apr1$<salt>$<hash>.

7.6.3.3 Patch the NGINX vhost #

Edit /etc/nginx/conf.d/trento.conf and add a prometheus upstream alongside the existing web and wanda upstreams. Adjust the address and port to match your Prometheus installation:
```
upstream prometheus {
  server 127.0.0.1:9090 max_fails=5 fail_timeout=60s;
}
```

Inside the existing server { listen 443 ssl; … } block, add a /prometheus/ location before the catch-all location / { … } block:

# Prometheus API
location /prometheus/ {
  auth_basic           "Prometheus";
  auth_basic_user_file /etc/nginx/prometheus.htpasswd;

  # Sized for remote_write batches and longer queries
  client_max_body_size 16m;
  proxy_read_timeout   120s;
  proxy_send_timeout   60s;

  proxy_http_version 1.1;
  proxy_set_header Host                 $http_host;
  proxy_set_header X-Forwarded-For      $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto    $scheme;
  proxy_set_header X-Cluster-Client-Ip  $remote_addr;

  # Trailing slash strips the /prometheus/ prefix:
  # /prometheus/api/v1/write -> /api/v1/write upstream
  proxy_pass http://prometheus/;
}

Validate and reload NGINX:
```
nginx -t && systemctl reload nginx
```

7.7 Configuring Grafana Alloy #

On SUSE Linux Enterprise Server for SAP applications 16, Grafana Alloy replaces node_exporter for collecting system metrics. On SUSE Linux Enterprise Server for SAP applications 15 SP7, Alloy is available as an optional package for users who prefer push-based metrics collection.

The Trento Agent provides a command to generate the required Alloy configuration.

7.7.1 Prerequisites #

Grafana Alloy must be installed on the host:
- On SUSE Linux Enterprise Server for SAP applications 16, it is available in the default repositories.
- On SUSE Linux Enterprise Server for SAP applications 15 SP7, it is available as an optional package.
The Prometheus remote write endpoint must be enabled on Trento Server (see Section 7.6, “Enabling Prometheus remote write for Grafana Alloy hosts”).

7.7.2 Generate and deploy the Alloy configuration #

Install Grafana Alloy:
```
zypper install alloy
```
Configure the Trento Agent with the required parameters for push mode in /etc/trento/agent.yaml. At minimum, you must set:
```
prometheus-mode: push
prometheus-url: https://<trento-server>/prometheus/api/v1/write
prometheus-auth: basic
prometheus-auth-username: <username>
prometheus-auth-password: <password>
```
Replace <username> and <password> with the credentials configured on Trento Server.
Note
When using the Ansible playbook, the agent configuration is set automatically from the inventory variables.
The Trento Agent also supports bearer, mtls, and none authentication methods. See the Trento Agent configuration documentation for the complete list of available parameters.

Generate the Alloy configuration and deploy it:

The generated configuration enables the following collectors by default:

Collector	Description
`cpu`	CPU usage statistics
`cpufreq`	CPU frequency scaling
`loadavg`	System load averages
`meminfo`	Memory usage statistics
`filesystem`	Filesystem usage and capacity
`netdev`	Network device statistics
`uname`	System information (kernel version, architecture)

These collectors cover the metrics used by Trento to render CPU and memory charts in the host detail view. If you need additional system metrics, you can edit the generated Alloy configuration to add more collectors from the full list of available collectors.

Generate the Trento-specific Alloy configuration using the Trento Agent:

trento-agent generate alloy > /etc/alloy/trento.alloy
chown alloy:alloy /etc/alloy/trento.alloy
chmod 640 /etc/alloy/trento.alloy

Grafana Alloy supports loading a single configuration file or multiple files from a directory. SUSE Linux Enterprise Server for SAP applications 16 ships Alloy with a default configuration file at /etc/alloy/config.alloy. Choose one of the following approaches to integrate the Trento configuration:

Single configuration file: Append the content of /etc/alloy/trento.alloy to the existing /etc/alloy/config.alloy file.
Import configuration file: Keep /etc/alloy/trento.alloy as a separate file and include it in the main configuration using the import directive. See import.file.
Configuration directory: Keep /etc/alloy/trento.alloy as a separate file and configure Alloy to automatically load all *.alloy files from the /etc/alloy/ directory. This can be set up by overriding the CONFIG_FILE environment variable in the alloy.service systemd unit using a systemd drop-in file. See systemd.unit.

Important

Regardless of the chosen approach, all components defined in the Trento configuration must have unique names across all configuration files to avoid conflicts.

Enable and start the Alloy service:
```
systemctl enable --now alloy
```
If Alloy is already running, reload the configuration:
```
systemctl reload alloy
```
Start or restart the Trento Agent:
```
systemctl enable --now trento-agent
```

7.7.3 Verifying the configuration #

After completing the setup, verify that metrics are being pushed successfully:

Check that Alloy is running without errors:

systemctl status alloy
journalctl -u alloy -f

In the Trento web console, navigate to the host details view. The CPU and memory charts should display data after a short time.