Applies to SUSE Linux Enterprise Server for SAP Applications 15 SP6

9 Tuning Workload Memory Protection #

Keeping SAP applications in physical memory is essential for their performance. In older product versions, the Page Cache Limit prevented a swap out to disk by a growing page cache (in SUSE Linux Enterprise Server for SAP Applications 11 SP1 onwards and in SUSE Linux Enterprise Server for SAP Applications 12). In SUSE Linux Enterprise Server for SAP Applications 15, the Page Cache Limit has been replaced by the more advanced Workload Memory Protection.

Workload Memory Protection puts SAP instances into a dedicated cgroup (v2) and tells the kernel, by the memory.low parameter, the amount of memory to keep in physical memory. This protects the processes in this cgroup against any form of memory pressure outside that cgroup, including a growing page cache. Workload Memory Protection cannot protect against memory pressure inside this cgroup. It covers the memory of all instances together on one host.

The value for memory.low depends on the kind of SAP instance and the workload and needs to be configured manually. If the system is under extreme pressure, the Linux kernel will ignore the memory.low value and try to stabilize the whole system, even by swapping or invoking the OOM killer.

For more information about cgroups, see https://documentation.suse.com/sles-15/html/SLES-all/cha-tuning-cgroups.html.

9.1 Architecture #

Workload Memory Protection relies on two components:

cgroup2 memory controller (Linux kernel)

The cgroup2 memory controller parameter memory.low allows defining an amount of memory, which the Linux kernel will keep in physical memory. This amount of memory will be excluded from the reclaiming process unless the entire system is in a critical memory situation.

Workload Memory Protection uses memory.low to prevent memory from SAP processes from being paged or swapped out to disk. Apart from the memory controller, cgroup1 controllers are still available, but are not mounted any more.

systemd

systemd provides the infrastructure to create and maintain the cgroup hierarchy and allows the configuration of cgroup parameters.

9.2 Support for Workload Memory Protection #

Workload Memory Protection is supported for SUSE Linux Enterprise Server for SAP Applications 15 SP6 on AMD64/Intel 64 and POWER for one or multiple SAP systems on one host, such as App Server (SAP NetWeaver, SAP S/4HANA). SUSE High Availability cluster solutions are supported.

Workload Memory Protection does not cover databases other than SAP HANA. Depending on their start method, the processes might run inside or outside the dedicated cgroup. If they run inside, the memory consumption needs to be taken into account when determining memory.low.

Important: Restrictions of Workload Memory Protection

Using Workload Memory Protection comes with benefits, but you should be aware of certain restrictions:

Workload Memory Protection cannot protect against memory pressure inside the dedicated cgroup.
Workload Memory Protection cannot protect SAP systems or their instances from each other. All SAP processes share the same memory limit. If you have multiple SAP systems (for example, SAP NetWeaver and SAP S/4HANA), Workload Memory Protection cannot shield one SAP application from the other.

To use Workload Memory Protection, the SAP system must use systemd. Details about the systemd integration can be found in SAP Notes: 139184 - Linux: systemd integration for sapstartsrv and SAP Host Agent and 3189534 - Linux: systemd integration for sapstartsrv and SAP HANA.

Important

Starting with SUSE Linux Enterprise Server for SAP Applications 15 SP5, the package sapwmp is deprecated. For infomation about migration, see Section 9.5, “Migration from Workload Memory Protection using sapwmp (SLES for SUSE Linux Enterprise Server for SAP Applications 15 SP4 and earlier)”.

9.3 Setting up Workload Memory Protection #

9.3.1 Preparing for Workload Memory Protection #

The SAP Start Service puts SAP instances into the dedicated SAP.slice cgroup. To use Workload Memory Protection, switch to the unified cgroup2 hierarchy and set MemoryLow= correctly.

Stop all SAP instances and the SAP Host Agent.
The service can be enabled, but all SAP processes need to be terminated.
Add systemd.unified_cgroup_hierarchy=true to the kernel command line by adding it to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub like:
```
GRUB_CMDLINE_LINUX_DEFAULT="... systemd.unified_cgroup_hierarchy=true swapaccount=1"
```
With this change, only cgroup2 controllers are be mounted on /sys/fs/cgroup. Cgroup1 controllers, except the memory controller, are still available and can be used though. Tools using cgroup1 may not work out of the box any more and may need reconfiguration. Also, the required mount structure for cgroup1 needs to be provided.
The parameter swapaccount=1 is not needed for Workload Memory Protection to work, but it aids the analysis in support cases to show the amount of swapped out memory for each cgroup.
Rewrite the GRUB2 configuration:
```
> sudo grub2-mkconfig -o /boot/grub2/grub.cfg
```
After reboot (which is performed later), the cgroup hierarchy is switched to v2 (unified hierarchy) only.
Configure MemoryLow for the SAP.slice:
```
> sudo systemctl set-property SAP.slice MemoryLow=...
```
This command creates a drop-in in /etc/systemd/system.control/SAP.slice.d/ to set MemoryLow.
SAP.slice is the name of the cgroup that contains the SAP processes. MemoryLow is the systemd equivalent of the cgroup parameter memory.low mentioned in the introduction. The value for MemoryLow depends on the type of the SAP application and the workload.
For SAP HANA
Since SAP HANA has a Global Allocation Limit, its value can be used directly.
SAP Application Server (SAP NetWeaver, SAP S/4HANA)
For the Application Server, the sizing for the workload should indicate the value for MemoryLow.
Keep in mind the following.
- All SAP instances on one host are inside the SAP.slice. MemoryLow must cover the amount of memory of all instances together on that host. You cannot protect SAP systems or their instances from each other.
- If you are using a database other than SAP HANA, some database processes can be part of SAP.slice. Their memory consumption needs to be taken into account when determining the MemoryLow value.
- Never choose a value for MemoryLow very close to or larger than your physical memory. System services and additional installed software require memory too. If they are forced to use swap too extensively, at the expense of the SAP application, your system can become unresponsive.
Note: Correctly calculate MemoryLow value
MemoryLow takes the memory size in bytes. If the value is suffixed with K, M, G, or T, the specified memory size is parsed as Kibibytes, Mebibytes, Gibibytes, or Tebibytes (with the base 1024 instead of 1000, see https://en.wikipedia.org/wiki/Binary_prefix), respectively. Alternatively, a percentage value may be specified, which is taken relative to the installed physical memory on the system.
The underlying cgroup memory controller rounds up the value to a multiple of the page size. To avoid confusion, set the value for MemoryLow to a multiple of the page size.

Now the system is ready for a reboot.

9.3.2 Reboot and verification #

Reboot the system.

After rebooting, verify that cgroups v2 has indeed been used:

# grep  cgroup /proc/mounts
cgroup /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime 0 0

Verify that the low memory value has been set:
```
> systemctl show -p MemoryLow SAP.slice
MemoryLow=18487889920    <- Should be your chosen value (always in bytes)!

# cat /sys/fs/cgroup/SAP.slice/memory.low
18487889920    <- Should be your chosen value!
```
The variable MemoryLow can be set to any value, but the content of the variable is always a multiple of the page size. Keep this in mind if you notice a slight difference between the values.

9.4 Changing the value of MemoryLow #

To change the value of MemoryLow run:

# systemctl set-property SAP.slice MemoryLow=...

The changes takes effect immediately.

The underlying cgroup memory controller rounds up the value to a multiple of the page size. To avoid confusion, set the value of MemoryLow to a multiple of the page size.

Important: Value of MemoryLow

Never set MemoryLow to a value lower than the memory already accounted in SAP.slice. To check, run:

# systemctl show -p MemoryCurrent SAP.slice

9.5 Migration from Workload Memory Protection using sapwmp (SLES for SUSE Linux Enterprise Server for SAP Applications 15 SP4 and earlier) #

It is necessary to perform a migration when you are upgrading from a Service Pack SP4 or earlier, and you have not yet switched to systemd-enabled instances.

Procedure 9.1: Migration procedure #

Switch to a systemd-enabled SAP system. For more info, refer to 139184 - Linux: systemd integration for sapstartsrv and SAP Host Agent and 3189534 - Linux: systemd integration for sapstartsrv and SAP HANA.
Remove the lines calling sapwmp-capture from all instance profiles (for example: Execute_20 = local /usr/lib/sapwmp/sapwmp-capture -a. All SAP services require a restart after the change.
Monitor and readjust MemoryLow=. A systemd-enabled SAP Host Agent now has a cgroup (saphostagent.service) below SAP.slice, and it is accounted for protection.
Note
It is not necessary to set MemoryLow=infinity for each SAP service or the SAP Host Agent service, if the cgroup2 mount has the option memory_recursiveprot set (it is set by default). To check that, run the following command:
```
>  mount | grep cgroup2
      cgroup2 on /sys/fs/cgroup type cgroup2
      (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
```