Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to SUSE Linux Enterprise Real Time 15 SP1

2 The Basic Shielding Model

Although any setup of cpusets can really be described as shielding, there is one prevalent shielding model in use that is so common that cset has a subcommand that is dedicated to its use. This subcommand is called shield.

The concept behind this model is the use of three cpusets:

  • Root cpusetis always present in all configurations and contains all CPUs.

  • System cpusetcontains CPUs which are used for system tasks. These are the normal tasks that are not important, but which need to run on the system.

  • User cpusetthe shield, contains CPUs which are used for important tasks. Only those tasks that are somehow important, usually tasks whose performance determines the overall rating for the machine, are run in the user cpuset.

The shield subcommand manages all of these cpusets and lets you define the CPUs and memory nodes that are in the shielded and unshielded sets. The subcommand automatically moves all movable tasks on the system into the unshielded cpuset on shield activation, and back into the root cpuset on shield tear down. The subcommand lets you move tasks into and out of the shield. Additionally, you can move special tasks (kernel threads) which normally run in the root cpuset into the unshielded set. This makes your shield have even less disturbance.

The shield subcommand abstracts the management of these cpusets away from you. It provides options that drive how the shield is set up, which tasks are to be shielded or not, and the status of the shield. In fact, you need not be bothered with the naming of the required cpusets or even where the cpuset file system is mounted. cset and the shield subcommand takes care of all that.

If you need to define more cpusets for your application, it is likely that this simple shielding is not rich enough for you. In this case, you should transition to using the set and proc subcommands described in Chapter 3, Full Featured Cpuset Manipulation Commands.

2.1 A Simple Shielding Example

Assume that we have a 4-core machine that has uniform memory access. This means there are 4 CPUs at your disposal and there is only one memory node available. On such machines, we do not need to specify any memory node parameters to cset, it sets up the only available memory node by default.

Usually, one wants to dedicate as many CPUs to the shield as possible and leave a minimal set of CPUs for normal system processing. The reasoning for this is, the performance of the important tasks will rule the performance of the installation as a whole. These important tasks need as many resources available to them as possible, exclusive of other, unimportant tasks that are running on the system.

Note
Note: Definition of Task

In this document task is used to represent either a process or a thread that is running on the system.

2.2 Setup and Teardown of the Shield

To set up a shield of 3 CPUs with 1 CPU left for low priority system processing, issue the following command.

tux > cset shield -c 1-3
cset: --> activating shielding:
cset: moving 176 tasks from root into system cpuset...
[==================================================]%
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running

This command does several things. First, it creates a user cpuset with what is called a CPUSPEC (CPU specification) from the -c/--cpu option. This CPUSPEC specifies to use CPUs 1 through 3 inclusively. Next, the command creates a system cpuset with a CPUSPEC that is the inverse of the -c option for the current machine. On this machine that cpuset will only contain the first CPU, CPU0. Next, all user space processes running in the root cpuset are transferred to the system cpuset. This makes all those processes run only on CPU0. The effect of this is that the shield consists of CPUs 1 through 3 and they are now idling.

Note that the command did not move the kernel threads that are running in the root cpuset to the system cpuset. This is because you may want these kernel threads to use all available CPUs. If you do not, then you can use the -k/--kthread option as described below.

The shield setup command above outputs the information of which cpusets were created and how many tasks are running on each. To see the current status of the shield again, issue this command:

tux > cset shield
cset: --> shielding system active with
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running

Which shows us that the shield is set up and that 176 tasks are running in the system cpuset—the unshielded cpuset.

It is important to move all possible tasks from the root cpuset to the unshielded system cpuset because a task’s cpuset property is inherited by its children. Since we’ve moved all running tasks (including init) to the unshielded system cpuset, that means that any new tasks that are spawned will also run in the unshielded system cpuset.

Some kernel threads can be moved into the unshielded system cpuset as well. These are the threads that are not bound to specific CPUs. If a kernel thread is bound to a specific CPU, then it is generally not a good idea to move that thread to the system set because at worst it may hang the system and at best it will slow the system down significantly. These threads are usually the IRQ threads on a real time Linux kernel, for example, and you should not move these kernel threads into system. If you leave them in the root cpuset, then they will have access to all CPUs.

However, if your application demands an even quieter shield, then you can move all movable kernel threads into the unshielded system set with the following command.

tux > cset shield -k on
cset: --> activating kthread shielding
cset: kthread shield activated, moving 70 tasks into system cpuset...
[==================================================]%
cset: done

You can see that this moved an additional 70 tasks to the unshielded system cpuset. Note that the -k/--kthread on parameter can be given at the shield's creation time. You do not need to perform these two steps separately if you know you will want kernel thread shielding as well. Executing cset shield again shows us the current state of the shield.

tux > cset shield
cset: --> shielding system active with
cset: "system" cpuset of CPUSPEC(0) with 246 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running

You can get a detailed listing of what is running in the shield by adding either -s/--shield or -u/--unshield to the shield subcommand and using the verbose flag. You will get output similar to the following.

tux > cset shield --unshield -v
cset: "system" cpuset of CPUSPEC(0) with 251 tasks running
USER     PID   PPID  SPPr TASK NAME
-------- ----- ----- ---- ---------
root     1     0     Soth init [5]
root     2     0     Soth [kthreadd]
root     84    2     Sf50 [IRQ-9
]...
alext    31796 31789 Soth less
root     32653 25222 Roth python ./cset shield --unshield -v

Note that the listing is abbreviated; we do have 251 tasks running in the system set. However, the SPPr field may need a little explanation. SPPr stands for State, Policy and Priority. You can see that the initial two tasks are Stopped and running in timeshare priority, marked as oth (for other). The [IRQ-9] task is also stopped, but marked at real time FIFO policy with a priority of 50. The last task in the listing is the cset command itself and is marked as running. Also note that adding a second -v/--verbose option will not restrict the output to fit into an 80 character screen.

Tear down of the shield, stopping the shield in other words, is done with the -r/--reset option to the shield subcommand. When this command is issued, both the system and user cpusets are deleted and any tasks that are running in both of those cpusets are moved to the root cpuset. Once so moved, all tasks will have access to all resources on the system. For example:

tux > cset shield --reset
cset: --> deactivating/reseting shielding
cset: moving 0 tasks from "/user" user set to root set...
cset: moving 250 tasks from "/system" system set to root set...
[==================================================]%
cset: deleting "/user" and "/system" sets
cset: done

2.3 Moving Interesting Tasks Into and Out of the Shield

Now that we have a shield running, the objective is to run our important processes in that shield. These processes can be anything, but usually they are directly related to the purpose of the machine. There are two ways to run tasks in the shield:

  • Exec a process into the shield

  • Move an already running task into the shield

2.3.1 Execing a Process Into the Shield

Running a new process in the shield can be done with the -e/--exec option to the shield subcommand. This is the simplest way to get a task to run in the shield. For this example, let’s exec a new Bash shell into the shield with the following commands.

tux > cset shield -s
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: done


tux > cset shield -e bash
cset: --> last message, executed args into cpuset "/user", new pid is: 13300


tux > cset shield -s -v
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
USER     PID   PPID  SPPr TASK NAME
-------- ----- ----- ---- ---------
root     13300 8583  Soth bash
root     13329 13300 Roth python ./cset shield -s -v


tux > exit


tux > cset shield -s
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: done

The first command above lists the status of the shield. You see that the shield is defined as CPUs 1 through 3 inclusive and currently there are no tasks running in it.

The second command execs the Bash shell into the shield with the -e option. The last message of cset lists the PID of the new process.

Note
Note: Separating the Tool Options From the cset Command

cset follows the tradition of separating the tool options from the command to be execed options with a double dash (--). This is not shown in this simple example, but if the command you want to exec also takes options, separate them with the double dash as follows:

tux > cset shield -e mycommand -- -v

The -v will be passed to mycommand, and not to cset.

The next command lists the status of the shield again. There are two tasks running shielded: our new shell and the cset status command itself. Remember that the cpuset property of a task is inherited by its children. Since we ran the new shell in the shield, its child, which is the status command, also ran in the shield.

Tip
Tip: Execing a Shell Into the Shield Is Useful

Execing a shell into the shield is a useful way to experiment with running tasks in the shield since all children of the shell will also run in the shield.

The last command exits the shell after which we request a shield status again and see that, once again, it does not contain any tasks.

You may have noticed in the output above that both the new shell and the status command are running as the root user. This is because cset needs to run as root and so all it is children will also run as root. If you need to run a process under a different user and or group, you may use the --user and --group options for execution as follows.

tux > cset shield --user=alext --group=users -e bash
cset: --> last message, executed args into cpuset "/user", new pid is: 14212


tux > cset shield -s -v
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
USER     PID   PPID  SPPr TASK NAME
-------- ----- ----- ---- ---------
alext    14212 8583  Soth bash
alext    14241 14212 Roth python ./cset shield -s -v

2.3.2 Moving a Running Task Into and Out of the Shield

While execing a process into the shield is undoubtedly useful, most of the time, you will want to move already running tasks into and out of the shield. The cset shield subcommand includes two options for doing this: -s/--shield and -u/--unshield. These options require a PIDSPEC (process specification) to also be specified with the -p/--pid option. The PIDSPEC defines which tasks get operated on. The PIDSPEC can be a single process ID, a list of process IDs separated by commas, and a list of process ID ranges separated by dashes, groups of which are separated by commas. For example:

--shield --pid 1234

This PIDSPEC argument specifies that PID 1234 be shielded.

--shield --pid 1234,42,1934,15000,15001,15002

This PIDSPEC argument specifies that this list of PIDs only be moved into the shield.

--unshield -p 5000,5100,6010-7000,9232

This PIDSPEC argument specifies that PIDs 5000, 5100 and 9232 be unshielded (moved out of the shield) along with any existing PID that is in the range 6010 through 7000 inclusive.

Note
Note: Information About the Range In a PIDSPEC

A range in a PIDSPEC does not need to have tasks running for every number in that range. In fact, it is not even an error if there are no tasks running in that range: none will be moved in that case. The range only specifies to act on any tasks that have a PID or TID that is within that range.

Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of tasks into and out of the shield. Additionally, there is one more option that can help with multi-threaded processes, and that is the --threads flag. If this flag is used together with a shield or unshield command with a PIDSPEC and if any of the task IDs in the PIDSPEC belong to a thread in a process container, then all the sibling threads in that process container will get shielded or unshielded as well. This flag provides an easy mechanism to shield/unshield all threads of a process by simply specifying one thread in that process.

In the following example, we move the current shell into the shield with a range PIDSPEC and back out with the Bash variable for the current PID.

tux > echo $$
22018


tux > cset shield -s -p 22010-22020
cset: --> shielding following pidspec: 22010-22020
cset: done


tux > cset shield -s -v
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
USER     PID   PPID  SPPr TASK NAME
-------- ----- ----- ---- ---------
root     3770  22018 Roth python ./cset shield -s -v
root     22018 5034  Soth bash
cset: done


tux > cset shield -u -p $$
cset: --> unshielding following pidspec: 22018
cset: done


tux > cset shield -s
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: done
Print this page