Applies to SUSE Linux Enterprise Server for SAP Applications 15 SP3

9 Tuning Workload Memory Protection #

Important

If you have systemd-based SAP instances, read Section 9.9, “Systems running both systemd-based and regular SAP instances” and Section 9.10, “Systems running only systemd-based instances” before setting up Workload Memory Protection.

Keeping SAP applications in physical memory is essential for their performance. In older product versions, the Page Cache Limit prevented a swap out to disk by a growing page cache (in SUSE Linux Enterprise Server for SAP Applications 11 SP1 onwards and in SUSE Linux Enterprise Server for SAP Applications 12). In SUSE Linux Enterprise Server for SAP Applications 15, the Page Cache Limit has been replaced by the more advanced Workload Memory Protection.

Workload Memory Protection puts SAP instances into a dedicated cgroup (v2) and tells the kernel, by the memory.low parameter, the amount of memory to keep in physical memory. This protects the processes in this cgroup against any form of memory pressure outside that cgroup, including a growing page cache. Workload Memory Protection cannot protect against memory pressure inside this cgroup. It covers the memory of all instances together on one host.

The value for memory.low depends on the kind of SAP instance and the workload and needs to be configured manually. If the system is under extreme pressure, the Linux kernel will ignore the memory.low value and try to stabilize the whole system, even by swapping or invoking the OOM killer.

For more information about cgroups, see https://documentation.suse.com/sles-15/html/SLES-all/cha-tuning-cgroups.html.

9.1 Architecture #

WMP relies on three components:

cgroup2 memory controller (Linux kernel)

The cgroup2 memory controller parameter memory.low allows defining an amount of memory, which the Linux kernel will keep in physical memory. This amount of memory will be excluded from the reclaiming process unless the entire system is in a critical memory situation.

WMP uses memory.low to prevent memory from SAP processes from being paged or swapped out to disk. Apart from the memory controller, cgroup1 controllers are still available, but are not mounted any more.

systemd

systemd provides the infrastructure to create and maintain the cgroup hierarchy and allows the configuration of cgroup parameters. WMP ships systemd configuration files to allow easy configuration of memory.low via systemd methods.

SAP start service

The SAP start service manages the starting and stopping of SAP instances. An important feature for WMP is the configurable execution of programs before the instance itself gets started in the instance profile. WMP uses this method to call a program to move the sapstart process into a designated cgroup, so the SAP instance will be started inside that cgroup.

9.2 Support for Workload Memory Protection #

WMP is supported for SUSE Linux Enterprise Server for SAP Applications 15 SP3 on AMD64/Intel 64 and POWER for one or multiple SAP systems on one host, such as:

App Server (SAP NetWeaver, SAP S/4HANA) or
SAP HANA 1.0/2.0

Workload Memory Protection does not cover databases other than SAP HANA. Depending on their start method, the processes might run inside or outside the dedicated cgroup. If they run inside, the memory consumption needs to be taken into account when determining memory.low.

Important: Restrictions of WMP

Using WMP comes with benefits, but you should be aware of some restrictions:

WMP cannot protect against memory pressure inside the dedicated cgroup.
WMP cannot protect SAP systems or their instances from each other. All SAP processes share the same memory limit. If you have multiple SAP systems (for example, SAP NetWeaver and SAP S/4HANA), WMP cannot shield one SAP application from the other.
Support for SUSE’s HA cluster solution is not yet available.

9.3 Setting up Workload Memory Protection #

9.3.1 Preparing for Workload Memory Protection #

Check if your SAP software (SAP HANA, SAP NetWeaver etc) is installed. The group sapsys is needed during the package installation of sapwmp later. If you skip that part, you will get a warning message (see Important: Watch out for order of packages).
Stop the SAP system:
```
# systemctl stop sapinit
```
The service can be enabled, but all SAP processes need to be terminated.
Install the package sapwmp:
```
> sudo zypper install sapwmp
```
Important: Watch out for order of packages
The following message should only appear if no SAP software has been installed on the system:
```
Warning: sapsys group not found warning: group sapsys does not exist - using root
```
Remove the package sapwmp and install the SAP software first before installing sapwmp again.
As an alternative, you can fix ownership and permission after installing the SAP software with:
```
> sudo chgrp sapsys /usr/lib/sapwmp/sapwmp-capture && \
chmod +s /usr/lib/sapwmp/sapwmp-capture
```
The following message can be ignored:
```
Warning: Found memory controller on v1 hierarchy. Make sure unified hierarchy only is used.
```
Switching to unified hierarchy is done in the next step.
Add systemd.unified_cgroup_hierarchy=true to the kernel command line by adding it to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub like:
```
GRUB_CMDLINE_LINUX_DEFAULT="... systemd.unified_cgroup_hierarchy=true swapaccount=1"
```
With this change, only cgroup2 controllers will be mounted on /sys/fs/cgroup. Cgroup1 controllers, except the memory controller, are still available and can be used though. Tools using cgroup1 might not work out of the box any more and might need reconfiguration. Also, the required mount structure for cgroup1 needs to be provided.
The parameter swapaccount=1 is not needed for WMP to work, but it aids the analysis in support cases to show the amount of swapped out memory for each cgroup.
Rewrite the GRUB2 configuration:
```
> sudo grub2-mkconfig -o /boot/grub2/grub.cfg
```
After reboot (will be done later), the cgroup hierarchy is switched to v2 (unified hierarchy) only.
Configure MemoryLow for the SAP.slice:
```
> sudo systemctl set-property SAP.slice MemoryLow=...
```
This command creates a drop-in in /etc/systemd/system.control/SAP.slice.d/ to set MemoryLow.
The sapwmp package includes the systemd configuration SAP.slice which creates the cgroup of the same name for the SAP instances. MemoryLow is the systemd equivalent of the cgroup parameter memory.low mentioned in the introduction. The value for MemoryLow depends on the type of the SAP application and the workload.
For SAP HANA
Since SAP HANA has a Global Allocation Limit, its value can be used directly.
SAP Application Server (SAP NetWeaver, SAP S/4HANA)
For the Application Server, the sizing for the workload should indicate the value for MemoryLow. The sapwmp package contains a monitoring part which might be useful to determine MemoryLow. See Section 9.6, “Monitoring memory usage”.
Keep in mind:
- All SAP instances on one host are inside the SAP.slice. MemoryLow must cover the amount of memory of all instances together on that host. You cannot protect SAP systems or their instances from each other.
- If you are using a database other than SAP HANA, some database processes might be part of SAP.slice. Their memory consumption needs to be taken into account when determining the MemoryLow value.
- Never choose a value for MemoryLow very close to or larger than your physical memory. System services and additional installed software require memory too. If they are forced to use swap too extensively, at the expense of the SAP application, your system can become unresponsive.
Note: Correctly calculate MemoryLow value
MemoryLow takes the memory size in bytes. If the value is suffixed with K, M, G, or T, the specified memory size is parsed as Kibibytes, Mebibytes, Gibibytes, or Tebibytes (with the base 1024 instead of 1000, see https://en.wikipedia.org/wiki/Binary_prefix), respectively. Alternatively, a percentage value may be specified, which is taken relative to the installed physical memory on the system.
The underlying cgroup memory controller will round up the value to a multiple of the page size. To avoid confusion, set the value for MemoryLow to a multiple of the page size.
Create a backup of each SAP instance profile. Errors in a profile can prevent a SAP system from starting.
For each SAP instance, add the following line to the instance profile (usually located in /usr/sap/SID/SYS/profile/) after the last Execute_ line:
```
Execute_20 = local /usr/lib/sapwmp/sapwmp-capture -a
```
If necessary, increase the number of the Execute statement so that it is the highest one, which means that that line is executed last.
Important: Editing instance profiles
Edit the instance profiles directly only if you do not have the profiles imported into the database to manage them via the SAP GUI (transaction RZ11). If you have imported them, use the SAP GUI to add the lines. Profile files located in the file system are overwritten and any manual changes would be lost!

Now the system is ready for a reboot.

9.3.2 Reboot and verification #

Reboot the system.

After rebooting, verify that cgroups v2 has indeed been used:

# grep  cgroup /proc/mounts
cgroup /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime 0 0

Verify that the cgroup was created successfully and the low memory value has been set:
```
> systemctl show -p MemoryLow SAP.slice
MemoryLow=18487889920    <- Should be your chosen value (always in bytes)!

# cat /sys/fs/cgroup/SAP.slice/memory.low
18487889920    <- Should be your chosen value!
```
The variable MemoryLow can be set to any value, but the content of the variable is always a multiple of the page size. Keep this in mind if you notice a slight difference between the values.
Check that all SAP instance processes are in the correct system slices/cgroup.
If you have not enabled sapinit.service start the service now. If autostart is not enabled in the instance profiles, start the instances before you check.
Example:
```
# systemd-cgls -a /sys/fs/cgroup/SAP.slice
Directory /sys/fs/cgroup/SAP.slice:
|-wmp-rd91fd6b3ca0d4c1183659ef4f9a092fa.scope
| |-3349 sapstart pf=/usr/sap/HA0/ERS10/profile/HA0_ERS10_sapha0er
| `-3375 er.sapHA0_ERS10 pf=/usr/sap/HA0/ERS10/profile/HA0_ERS10_sapha0er N...
|-wmp-r360ebfe09bcd4df4873ef69898576199.scope
| |-3572 sapstart pf=/usr/sap/HA0/SYS/profile/HA0_D01_sapha0ci
| |-3624 dw.sapHA0_D01 pf=/usr/sap/HA0/SYS/profile/HA0_D01_sapha0ci
...
```
The sapstartsrv process of an instance always remains in the user slice of SIDadm. Only the sapstart process and its children will be moved to the target cgroup.
For each instance, a directory wmp-rSCOPEID.scope exists with all processes of this instance. The SCOPEID is a random 128-bit value in hexadecimal.
The SAP HostAgent is not covered by WMP and remains partly in sapinit.slice and partly in the user slice of sapadm.

If the processes are not in the cgroup, check if the Execute lines in the instance profiles are correct. Also each instance start should now be logged in the system log /var/log/messages:

...
2020-06-16T18:41:28.317233+02:00 server-03 sapwmp-capture: Found PIDs:
2020-06-16T18:41:28.317624+02:00 server-03 sapwmp-capture:      17001
2020-06-16T18:41:28.317813+02:00 server-03 sapwmp-capture:      16994
2020-06-16T18:41:28.317959+02:00 server-03 sapwmp-capture:      16551
2020-06-16T18:41:28.319423+02:00 server-03 sapwmp-capture: Successful capture into SAP.slice/wmp-r07a27e12d7f2491f8ccb9aeb0e080aaa.scope
2020-06-16T18:41:28.319672+02:00 server-03 systemd[1]: Started wmp-r07a27e12d7f2491f8ccb9aeb0e080aaa.scope.
...

To verify the correct setup, run wmp-check. The script checks the setup of Workload Memory Protection:

Correct setup of cgroup2.
Ownership and permission of the capture program.
WMP entries of SAP instance profiles.
Correct cgrop of running SAP instance processes.
Correct setup of SAP.slice.
Sane configuration of MemoryLow. However, it cannot determine if the MemoryLow value has been chosen wisely.
Setup of the optional memory sampler.
Setup of optional swap accounting.

It assumes SAP instances profiles can be found beneath /usr/sap/SID/SYS/profile/.

9.4 Configuring Workload Memory Protection #

To configure WMP, edit /etc/sapwmp.conf:

# NOTE: Local changes may be reverted after update of WMP package. Check for
#      .rpmsave file to restore & merge changes.

## Description: Slice unit name where workload is put into
## Type:        string
## Default:     "SAP.slice"
DEFAULT_SLICE="SAP.slice"

## Description: Comma-separated list of command names to which capture is
##              applied (matching against /proc/$PID/stat)
## Type:        string
## Default:     sapstart
PARENT_COMMANDS=sapstart

After any change, restart all SAP instances.

Warning

Altering /etc/sapwmp.conf should not be necessary. Do not do it until you know exactly what you are doing!

9.5 Changing the value of MemoryLow #

To change the value of MemoryLow run:

# systemctl set-property SAP.slice MemoryLow=...

The changes will take effect immediately.

The underlying cgroup memory controller will round up the value to a multiple of the page size. To avoid confusion, set the value of MemoryLow to a multiple of the page size.

Important: Value of MemoryLow

Never set MemoryLow to a value lower than the memory already accounted in SAP.slice. To check, run:

# systemctl show -p MemoryCurrent SAP.slice

9.6 Monitoring memory usage #

Logging the memory usage can be necessary not only to determine the value for memory.low, but also for monitoring the correct operation of WMP.

To enable monitoring, activate the shipped timer unit:

# systemctl enable --now wmp-sample-memory.timer

Now the timer should be listed by systemctl list-timers:

# systemctl list-timers
NEXT    LEFT       LAST    PASSED  UNIT                     ACTIVATES
...
Tue...  9min left  Tue...  4s ago  wmp-sample-memory.timer  wmp-sample-memory.service
...

If you check the current configuration, you can see that memory data is collected every 10 minutes with a randomized delay of three minutes:

# systemctl cat wmp-sample-memory.timer
# /usr/lib/systemd/system/wmp-sample-memory.timer
[Unit]
Description=WMP periodic log of memory consumption

[Timer]
OnCalendar=*:0/10
RandomizedDelaySec=180
AccuracySec=60

[Install]
WantedBy=timers.target

To change this, create a drop-in file and reload systemd (for example, by increasing the interval to 30 minutes):

# mkdir /etc/systemd/system/wmp-sample-memory.timer.d

# cat <<EOF >/etc/systemd/system/wmp-sample-memory.timer.d/override.conf
[Timer]
OnCalendar=
OnCalendar=*:0/30
EOF

# systemctl daemon-reload

(The first OnCalendar= line is important for deleting previously defined OnCalendar= settings.)

To see the memory consumption, check the system log for lines written by wmp_memory_current:

# grep wmp_memory_current /var/log/messages
...


2020-09-14T12:02:40.337266+02:00 server-03 wmp_memory_current: SAP.slice : memory.low=21474836480 memory.current=2294059008 memory.swap.current=0 , user.slice : memory.low=0 memory.current=5499219968 memory.swap.current=0 , init.scope : memory.low=0 memory.current=8364032 memory.swap.current=0 , system.slice : memory.low=0 memory.current=1863335936 memory.swap.current=0
2020-09-14T12:03:00.767838+02:00 server-03 wmp_memory_current: SAP.slice : memory.low=21474836480 memory.current=2294022144 memory.swap.current=0 , user.slice : memory.low=0 memory.current=5499473920 memory.swap.current=0 , init.scope : memory.low=0 memory.current=8364032 memory.swap.current=0 , system.slice : memory.low=0 memory.current=1862586368 memory.swap.current=0
2020-09-14T12:04:00.337315+02:00 server-03 wmp_memory_current: SAP.slice : memory.low=21474836480 memory.current=2294022144 memory.swap.current=0 , user.slice : memory.low=0 memory.current=5499207680 memory.swap.current=0 , init.scope : memory.low=0 memory.current=8355840 memory.swap.current=0 , system.slice : memory.low=0 memory.current=1862746112 memory.swap.current=0
...

Here is a reformatted log line to get a better impression:

2020-09-14T12:02:40.337266+02:00 server-03 wmp_memory_current:
SAP.slice    : memory.low=21474836480 memory.current=2294059008 memory.swap.current=0 ,
user.slice   : memory.low=0           memory.current=5499219968 memory.swap.current=0 ,
init.scope   : memory.low=0           memory.current=8364032    memory.swap.current=0 ,
system.slice : memory.low=0           memory.current=1863335936 memory.swap.current=0

For each cgroup directly below /sys/fs/cgroup/ one comma-separated block exists. On a normal system, you should find at least user.slice, system.slice, and init.scope. WMP adds SAP.slice.

Each block contains the information about the current value of memory.low and memory.current, and the currently allocated amount of physical memory of processes in this cgroup.

If you enabled swap accounting (swapaccount=1) during setup, you also have memory.swap.current, the amount of swapped-out memory of the cgroup.

All values are in bytes. See Step 6 in Section 9.3.1, “Preparing for Workload Memory Protection”.

Tip: Script for printing

You can find a script to print the information as table or CSV here: https://github.com/scmschmidt/wmp_log_extract

9.7 Verifying correct operation #

Besides monitoring memory consumption and swapping (see Section 9.6, “Monitoring memory usage”), you should also regularly check that all SAP instance processes are in their scopes below SAP.slice.

To do so, run systemd-cgls and check each instance process.

Example:

# systemd-cgls -a /sys/fs/cgroup/SAP.slice
Directory /sys/fs/cgroup/SAP.slice:
|-wmp-rd91fd6b3ca0d4c1183659ef4f9a092fa.scope
| |-3349 sapstart pf=/usr/sap/HA0/ERS10/profile/HA0_ERS10_sapha0er
| `-3375 er.sapHA0_ERS10 pf=/usr/sap/HA0/ERS10/profile/HA0_ERS10_sapha0er N...
|-wmp-r360ebfe09bcd4df4873ef69898576199.scope
| |-3572 sapstart pf=/usr/sap/HA0/SYS/profile/HA0_D01_sapha0ci
| |-3624 dw.sapHA0_D01 pf=/usr/sap/HA0/SYS/profile/HA0_D01_sapha0ci
...

A simpler test would be to list all processes, including cgroups, for all SIDs used on the system.

Example:

> ps -eo user,pid,cgroup:60,args | grep -e [h]a0adm
ha0adm    2062 0::/user.slice/user-1001.slice/user@1001.service/init.scope  /usr/lib/systemd/systemd --user
ha0adm    2065 0::/user.slice/user-1001.slice/user@1001.service/init.scope  (sd-pam)
ha0adm    3081 0::/SAP.slice/wmp-r73c594e050904c9c922a312dd9a28fd4.scope    sapstart pf=/usr/sap/HA0/SYS/profile/HA0_ASCS00_sapha0as
ha0adm    3133 0::/SAP.slice/wmp-r73c594e050904c9c922a312dd9a28fd4.scope    ms.sapHA0_ASCS00 pf=/usr/sap/HA0/SYS/profile/HA0_ASCS00_sapha0as
ha0adm    3134 0::/SAP.slice/wmp-r73c594e050904c9c922a312dd9a28fd4.scope    en.sapHA0_ASCS00 pf=/usr/sap/HA0/SYS/profile/HA0_ASCS00_sapha0as
ha0adm    3327 0::/SAP.slice/wmp-ra42489517eb846c282c57681e627a496.scope    sapstart pf=/usr/sap/HA0/ERS10/profile/HA0_ERS10_sapha0er
...

All instance processes except sapstartsrv need to be in a scope below 0::/SAP.slice/.

To verify the correct setup, use the wmp-check tool. See Section 9.3.2, “Reboot and verification” for more details.

9.8 Uninstalling Workload Memory Protection #

Stop the SAP system completely. The sapinit.service has to be stopped, but can stay enabled. All SAP processes have to be terminated.
Remove any changes made to SAP.slice, such as setting MemoryLow:
```
# systemctl revert SAP.slice
```
(Optional) Remove the package sapwmp:
```
# zypper remove sapwmp
```
This step is optional. The package can stay on the system without having an influence.
(Optional) Remove systemd.unified_cgroup_hierarchy=true from GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub.
This step is optional. You can keep cgroup2 without using WMP.
Rewrite the GRUB2 configuration:
```
# grub2-mkconfig -o /boot/grub2/grub.cfg
```
After the next boot, the system is switched back to the hybrid cgroup hierarchy.
Remove the line to call sapwmp-capture from each SAP instance profile (usually located in /usr/sap/SID/SYS/profile/):
```
Execute_20 = local /usr/lib/sapwmp/sapwmp-capture -a
```
Important: Backup is necessary
Before editing an instance profile, create a backup! Errors in a profile can prevent an SAP system from starting!
Important: About editing profiles directly
Edit the instance profiles directly only if you have not imported the profiles into the database to manage them via the SAP GUI (transaction RZ11). If you have imported them, use the SAP GUI to add the lines. Profile files located in the file system are overwritten, and any manual changes will be lost!
Reboot the system and verify that your SAP system has been started successfully.

9.9 Systems running both `systemd`-based and regular SAP instances #

With SAP kernel version 788 and certain specified patch levels of older kernel releases (722, 753, 777, 781, 785), the SAP Start Service supports systemd. Such a systemd-based instance comes with its own systemd service that the SAP Start Service places into its own cgroup (SAPSID_NR.service) under SAP.slice. This affects the Workload Memory Protection setup. For systemd-based SAP instances, the following instructions apply:

Skip the step that adds the capture program to the instance profile. If you are migrating an instance to support systemd, remove this entry from the profile. Keep in mind that systemd-based instances are always put into a cgroup under SAP.slice and therefore become part of Workload Memory Protection protection.
Set MemoryLow=infinity for the SAPSID_NR.service to make the protection work correctly. Example for SAPNW1_01.service:
```
sudo systemctl set-property SAPNW1_01.service MemoryLow=infinity
```
Set MemoryLow=infinity for the saphostagent.service to make the protection work correctly
```
sudo systemctl set-property saphostagent.service MemoryLow=infinity
```

The following example demonstrates a mixed environment. The SAP Host Agent and instance 01 are systemd-based, instance 00 is not. Both instances are under SAP.slice either in the cgroup managed by the SAP Start Service (SAPNW1_01.service) or Workload Memory Protection (wmp-rece5b7fa372e4619a9623e120aa23a23.scope). MemoryLow= for SAP.slice has been set as well as MemoryLow=infinity for all the cgroups below.

>  systemd-cgls
Control group /:
-.slice
├─SAP.slice 
│ ├─SAPNW1_01.service 
│ │ ├─ 887 /usr/sap/NW1/ASCS01/exe/sapstartsrv pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ │ ├─2838 sapstart pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ │ ├─2895 ms.sapNW1_ASCS01 pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ │ └─2896 en.sapNW1_ASCS01 pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ ├─saphostagent.service 
│ │ ├─ 900 /usr/sap/hostctrl/exe/saphostexec pf=/usr/sap/hostctrl/exe/host_profile -nodaemon -trace
│ │ ├─ 984 /usr/sap/hostctrl/exe/sapstartsrv pf=/usr/sap/hostctrl/exe/host_profile
│ │ └─2428 /usr/sap/hostctrl/exe/saposcol -l -w60 pf=/usr/sap/hostctrl/exe/host_profile
│ └─wmp-rece5b7fa372e4619a9623e120aa23a23.scope 
│   ├─5522 sapstart pf=/usr/sap/NW1/SYS/profile/NW1_D00_systemdproj
│   ├─7824 dw.sapNW1_D00 pf=/usr/sap/NW1/SYS/profile/NW1_D00_systemdproj
│   ├─7825 ig.sapNW1_D00 -mode=profile pf=/usr/sap/NW1/SYS/profile/NW1_D00_systemdproj
...

>  systemctl show -p MemoryLow SAP.slice
MemoryLow=24584065024

>  systemctl show -p MemoryLow wmp-rece5b7fa372e4619a9623e120aa23a23.scope 
MemoryLow=infinity

>  systemctl show -p MemoryLow SAPNW1_01.service 
MemoryLow=infinity

>  systemctl show -p MemoryLow saphostagent.service 
MemoryLow=infinity

Further information about the systemd integration is available at SAP Note 139184 - Linux: systemd integration for sapstartsrv and SAP Host Agent.

9.10 Systems running only `systemd`-based instances #

Note

SUSE's HA cluster solutions are supported for fully systemd-based SAP systems.

An SAP system consisting of systemd-based instances can only rely on cgroup2 for memory protection.

The instances are placed into a dedicated cgroup (default is SAP.slice) by the SAP Start Service. Therefore it is only necessary to switch to the unified cgroup2 hierarchy and set MemoryLow= correctly.

If Workload Memory Protection is already configured, follow the instructions in Section 9.9, “Systems running both systemd-based and regular SAP instances”. Afterwards, you can remove the package sapwmp from the system, unless you want to use the wmp-sample-memory.timer and wmp-sample-memory.service to monitor memory usage.

To configure the memory protection for a system consisting of only systemd-based SAP instances, perform the following steps:

Stop all SAP instances and the SAP Host Agent.
Switch to the unified cgroup hierarchy, rewrite the boot loader (step 4 and 5 in Section 9.3.1, “Preparing for Workload Memory Protection”).
Set MemoryLow= for SAP.slice (step 6 in Section 9.3.1, “Preparing for Workload Memory Protection”).
Set MemoryLow=infinity for saphostagent.service and for the services of all SAP instances.
Reboot the system.

The following example demonstrates a systemd-based environment. The SAP Host Agent and all instances are in their cgroups under SAP.slice. MemoryLow= for SAP.slice has been set as well as MemoryLow=infinity for all the cgroups below.

>  systemd-cgls
Control group /:
-.slice
├─SAP.slice 
│ ├─SAPNW1_00.service 
│ │ ├─5522 sapstart pf=/usr/sap/NW1/SYS/profile/NW1_D00_systemdproj
│ │ ├─7824 dw.sapNW1_D00 pf=/usr/sap/NW1/SYS/profile/NW1_D00_systemdproj
│ │ ├─7825 ig.sapNW1_D00 -mode=profile pf=/usr/sap/NW1/SYS/profile/NW1_D00_systemdproj
...
│ ├─SAPNW1_01.service 
│ │ ├─ 887 /usr/sap/NW1/ASCS01/exe/sapstartsrv pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ │ ├─2838 sapstart pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ │ ├─2895 ms.sapNW1_ASCS01 pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ │ └─2896 en.sapNW1_ASCS01 pf=/usr/sap/NW1/SYS/profile/NW1_ASCS01_systemdproj
│ └─saphostagent.service 
│   ├─ 900 /usr/sap/hostctrl/exe/saphostexec pf=/usr/sap/hostctrl/exe/host_profile -nodaemon -trace
│   ├─ 984 /usr/sap/hostctrl/exe/sapstartsrv pf=/usr/sap/hostctrl/exe/host_profile
│   └─2428 /usr/sap/hostctrl/exe/saposcol -l -w60 pf=/usr/sap/hostctrl/exe/host_profile
...

>  systemctl show -p MemoryLow SAP.slice
MemoryLow=24584065024

>  systemctl show -p MemoryLow wmp-rece5b7fa372e4619a9623e120aa23a23.scope 
MemoryLow=infinity

>  systemctl show -p MemoryLow SAPNW1_01.service 
MemoryLow=infinity

>  systemctl show -p MemoryLow saphostagent.service 
MemoryLow=infinity