Applies to SUSE Linux Enterprise Server 15 SP4

1 General notes on system tuning #

This manual discusses how to find the reasons for performance problems and provides means to solve these problems. Before you start tuning your system, you should make sure you have ruled out common problems and have found the cause for the problem. You should also have a detailed plan on how to tune the system, because applying random tuning tips often will not help and could make things worse.

Procedure 1.1: General approach when tuning a system #

Specify the problem that needs to be solved.
In case the degradation is new, identify any recent changes to the system.
Identify why the issue is considered a performance problem.
Specify a metric that can be used to analyze performance. This metric could for example be latency, throughput, the maximum number of users that are simultaneously logged in, or the maximum number of active users.
Measure current performance using the metric from the previous step.
Identify the subsystem(s) where the application is spending the most time.
1. Monitor the system and/or the application.
2. Analyze the data, categorize where time is being spent.
Tune the subsystem identified in the previous step.
Remeasure the current performance without monitoring using the same metric as before.
If performance is still not acceptable, start over with Step 3.

1.1 Be sure what problem to solve #

Before starting to tuning a system, try to describe the problem as exactly as possible. A statement like “The system is slow!” is not a helpful problem description. For example, it could make a difference whether the system speed needs to be improved in general or only at peak times.

Furthermore, make sure you can apply a measurement to your problem, otherwise you cannot verify if the tuning was a success or not. You should always be able to compare “before” and “after”. Which metrics to use depends on the scenario or application you are looking into. Relevant Web server metrics, for example, could be expressed in terms of:

Latency: The time to deliver a page
Throughput: Number of pages served per second or megabytes transferred per second
Active users: The maximum number of users that can be downloading pages while still receiving pages within an acceptable latency

1.2 Rule out common problems #

A performance problem often is caused by network or hardware problems, bugs, or configuration issues. Make sure to rule out problems such as the ones listed below before attempting to tune your system:

Check the output of the systemd journal (see Chapter 21, journalctl: Query the systemd journal) for unusual entries.
Check (using top or ps) whether a certain process misbehaves by eating up unusual amounts of CPU time or memory.
Check for network problems by inspecting /proc/net/dev.
In case of I/O problems with physical disks, make sure it is not caused by hardware problems (check the disk with the smartmontools) or by a full disk.
Ensure that background jobs are scheduled to be carried out in times the server load is low. Those jobs should also run with low priority (set via nice).
If the machine runs several services using the same resources, consider moving services to another server.
Last, make sure your software is up-to-date.

1.3 Finding the bottleneck #

Finding the bottleneck very often is the hardest part when tuning a system. SUSE Linux Enterprise Server offers many tools to help you with this task. See Part II, “System monitoring” for detailed information on general system monitoring applications and log file analysis. If the problem requires a long-time in-depth analysis, the Linux kernel offers means to perform such analysis. See Part III, “Kernel monitoring” for coverage.

Once you have collected the data, it needs to be analyzed. First, inspect if the server's hardware (memory, CPU, bus) and its I/O capacities (disk, network) are sufficient. If these basic conditions are met, the system might benefit from tuning.

1.4 Step-by-step tuning #

Make sure to carefully plan the tuning itself. It is of vital importance to only do one step at a time. Only by doing so can you measure whether the change made an improvement or even had a negative impact. Each tuning activity should be measured over a sufficient time period to ensure you can do an analysis based on significant data. If you cannot measure a positive effect, do not make the change permanent. Chances are, that it might have a negative effect in the future.