Applies to SUSE Linux Enterprise Server 15 SP6

44 Understanding Linux audit #

The Linux audit framework as shipped with this version of SUSE Linux Enterprise Server provides a CAPP-compliant (Controlled Access Protection Profiles) auditing system that reliably collects information about any security-relevant event. The audit records can be examined to determine whether any violation of the security policies has been committed, and by whom.

Providing an audit framework is an important requirement for a CC-CAPP/EAL (Common Criteria-Controlled Access Protection Profiles/Evaluation Assurance Level) certification. Common Criteria (CC) for Information Technology Security Information is an international standard for independent security evaluations. Common Criteria helps customers judge the security level of any IT product they intend to deploy in mission-critical setups.

Common Criteria security evaluations have two sets of evaluation requirements, functional and assurance requirements. Functional requirements describe the security attributes of the product under evaluation and are summarized under the Controlled Access Protection Profiles (CAPP). Assurance requirements are summarized under the Evaluation Assurance Level (EAL). EAL describes any activities that must take place for the evaluators to be confident that security attributes are present, effective and implemented. Examples for activities of this kind include documenting the developers' search for security vulnerabilities, the patch process, and testing.

This guide provides a basic understanding of how audit works and how it can be set up. For more information about Common Criteria itself, refer to the Common Criteria Web site.

Linux audit helps make your system more secure by providing you with a means to analyze what is happening on your system in great detail. It does not, however, provide additional security itself—it does not protect your system from code malfunctions or any kind of exploits. Instead, audit is useful for tracking these issues and helps you take additional security measures, like AppArmor, to prevent them.

Audit consists of several components, each contributing crucial functionality to the overall framework. The audit kernel module intercepts the system calls and records the relevant events. The auditd daemon writes the audit reports to disk. Various command-line utilities take care of displaying, querying and archiving the audit trail.

Audit enables you to do the following:

Associate users with processes

Audit maps processes to the user ID that started them. This makes it possible for the administrator or security officer to exactly trace which user owns which process and is potentially doing malicious operations on the system.

Important: Renaming user IDs

Audit does not handle the renaming of UIDs. Therefore avoid renaming UIDs (for example, changing tux from uid=1001 to uid=2000) and obsolete UIDs rather than renaming them. Otherwise you would need to change auditctl data (audit rules) and would have problems retrieving old data correctly.

Review the audit trail

Linux audit provides tools that write the audit reports to disk and translate them into human readable format.

Review particular audit events

Audit provides a utility that allows you to filter the audit reports for certain events of interest. You can filter for:

User
Group
Audit ID
Remote Host Name
Remote Host Address
System Call
System Call Arguments
File
File Operations
Success or Failure

Apply a selective audit

Audit provides the means to filter the audit reports for events of interest and to tune audit to record selected events. You can create your own set of rules and have the audit daemon record those of interest to you.

Guarantee the availability of the report data

Audit reports are owned by root and removable by root. Unauthorized users cannot remove the audit logs.

Prevent audit data loss

If the kernel runs out of memory, the audit daemon's backlog is exceeded, or its rate limit is exceeded, audit can trigger a shutdown of the system to keep events from escaping audit's control. This shutdown would be an immediate halt of the system triggered by the audit kernel component without synchronizing the latest logs to disk. The default configuration is to log a warning to syslog rather than to halt the system.

If the system runs out of disk space when logging, the audit system can be configured to perform clean shutdown. The default configuration tells the audit daemon to stop logging when it runs out of disk space.

44.1 Introducing the components of Linux audit #

Important: audispd merged into auditd

In SUSE Linux Enterprise Server 15 SP4 the code for audispd has been merged into auditd. All audispd configurations are now in /etc/audit/auditd.conf and /etc/audit/plugins.d.

The following figure illustrates how the various components of audit interact with each other:

Figure 44.1: Introducing the components of Linux audit #

Straight arrows represent the data flow between components while dashed arrows represent lines of control between components.

auditd: The audit daemon writes the audit messages that were generated through the audit kernel interface and triggered by application and system activity to disk. The way the audit daemon is started is controlled by systemd. The audit system functions (when started) are controlled by /etc/audit/auditd.conf. For more information about auditd and its configuration, refer to Section 44.2, “Configuring the audit daemon”.
auditctl: The auditctl utility controls the audit system. It controls the log generation parameters and kernel settings of the audit interface and the rule sets that determine which events are tracked. For more information about auditctl, refer to Section 44.3, “Controlling the audit system using auditctl”.
audit rules: The file /etc/audit/audit.rules contains a sequence of auditctl commands that are loaded at system boot time immediately after the audit daemon is started. For more information about audit rules, refer to Section 44.4, “Passing parameters to the audit system”.
aureport: The aureport utility allows you to create custom reports from the audit event log. This report generation can easily be scripted, and the output can be used by various other applications, for example, to plot these results. For more information about aureport, refer to Section 44.5, “Understanding the audit logs and generating reports”.
ausearch: The ausearch utility can search the audit log file for certain events using various keys or other characteristics of the logged format. For more information about ausearch, refer to Section 44.6, “Querying the audit daemon logs with ausearch”.
autrace: The autrace utility traces individual processes in a fashion similar to strace. The output of autrace is logged to the audit log. For more information about autrace, refer to Section 44.7, “Analyzing processes with autrace”.
aulast: Prints a list of the last logged-in users, similarly to last. aulast searches back through the audit logs (or the given audit log file) and displays a list of all users logged in and out based on the range of time in the audit logs.
aulastlog: Prints the last login for all users of a machine similar to the way lastlog does. The login name, port and last login time are printed.

44.2 Configuring the audit daemon #

Before you can start generating audit logs and processing them, configure the audit daemon itself. The /etc/audit/auditd.conf configuration file determines how the audit system functions when the daemon has been started. For most use cases, the default settings shipped with SUSE Linux Enterprise Server should suffice. For CAPP environments, most of these parameters need tweaking. The following example is the default configuration:

Example 44.1: Default /etc/audit/auditd.conf #

local_events = yes
write_logs = yes
log_file = /var/log/audit/audit.log
log_group = audit
log_format = RAW
flush = INCREMENTAL_ASYNC
freq = 50
max_log_file = 8
num_logs = 5
priority_boost = 4
name_format = NONE
##name = mydomain
max_log_file_action = ROTATE
space_left = 75
space_left_action = SYSLOG
verify_email = yes
action_mail_acct = root
admin_space_left = 50
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND
disk_error_action = SUSPEND
use_libwrap = yes
##tcp_listen_port = 60
tcp_listen_queue = 5
tcp_max_per_addr = 1
##tcp_client_ports = 1024-65535
tcp_client_max_idle = 0
transport = TCP
distribute_network = no
q_depth = 1200
overflow_action = SYSLOG
max_restarts = 10
plugin_dir = /etc/audit/plugins.d
end_of_event_timeout = 2

Refer to man 5 auditd.conf for an explanation of these options.

Depending on whether you want your environment to satisfy the requirements of CAPP, you need to be extra restrictive when configuring the audit daemon. Where you need to use particular settings to meet the CAPP requirements, a “CAPP Environment” note tells you how to adjust the configuration.

When the daemon configuration in /etc/audit/auditd.conf is complete, the next step is to focus on controlling the amount of auditing the daemon does, and to assign sufficient resources and limits to the daemon so it can operate smoothly.

44.3 Controlling the audit system using `auditctl` #

auditctl controls the status and basic system parameters of the audit daemon. It controls the amount of auditing performed on the system. Using audit rules, auditctl controls which components of your system are subjected to the audit and to what extent they are audited. Audit rules can be passed to the audit daemon on the auditctl command line or by composing a rule set and instructing the audit daemon to process this file. By default, the auditd daemon is configured to check for audit rules under /etc/audit/audit.rules. For more details on audit rules, refer to Section 44.4, “Passing parameters to the audit system”.

The main auditctl commands to control basic audit system parameters are:

auditctl -e to enable or disable audit
auditctl -f to control the failure flag
auditctl -r to control the rate limit for audit messages
auditctl -b to control the backlog limit
auditctl -s to query the current status of the audit daemon
auditctl -S specifies which system call to audit. Before running auditctl -S on your system, add -F arch=b64 to prevent the architecture mismatch warning.

The -e, -f, -r, and -b options can also be specified in the audit.rules file to avoid having to enter them each time the audit daemon is started.

Any time you query the status of the audit daemon with auditctl -s or change the status flag with auditctl -eFLAG, a status message (including information on each of the above-mentioned parameters) is printed. The following example highlights the typical audit status message.

Example 44.2: Example output of auditctl -s #

enabled 1
failure 1
pid 790
rate_limit 0
backlog_limit 64
lost 0
backlog 0
backlog_wait_time 15000
loginuid_immutable 0 unlocked

Table 44.1: Audit status flags #

Flag	Meaning [Possible Values]	Command
`enabled`	Set the enable flag. [0..2] 0=disable, 1=enable, 2=enable and lock down the configuration. Note that this disables logging syscalls, and other events may still be logged. (See `man 3 audit_set_enabled` in the audit-devel.)	`auditctl` `-e [0\|1\|2]`
`flag`	Set the failure flag. [0..2] 0=silent, 1=printk, 2=panic (immediate halt without synchronizing pending data to disk)	`auditctl` `-f [0\|1\|2]`
`pid`	Process ID under which `auditd` is running.	—
`rate_limit`	Set a limit in messages per second. If the rate is not zero and is exceeded, the action specified in the failure flag is triggered.	`auditctl` `-r RATE`
`backlog_limit`	Specify the maximum number of outstanding audit buffers allowed. If all buffers are full, the action specified in the failure flag is triggered.	`auditctl` `-b BACKLOG`
`lost`	Count the current number of lost audit messages.	—
`backlog`	Count the current number of outstanding audit buffers.	—

44.4 Passing parameters to the audit system #

Commands to control the audit system can be invoked individually from the shell using auditctl or batch read from a file using auditctl - R. This latter method is used by the init scripts to load rules from the file /etc/audit/audit.rules after the audit daemon has been started. The rules are executed in order from top to bottom. Each of these rules would expand to a separate auditctl command. The syntax used in the rules file is the same as that used for the auditctl command.

Changes made to the running audit system by executing auditctl on the command line are not persistent across system restarts. For changes to persist, add them to the /etc/audit/audit.rules file and, if they are not currently loaded into audit, restart the audit system to load the modified rule set by using the systemctl restart auditd command.

Example 44.3: Example audit rules—audit system parameters #

-b 10001
-f 12
-r 103
-e 14

1	Specify the maximum number of outstanding audit buffers. Depending on the level of logging activity, you might need to adjust the number of buffers to avoid causing too heavy an audit load on your system.
2	Specify the failure flag to use. See Table 44.1, “Audit status flags” for possible values.
3	Specify the maximum number of messages per second that may be issued by the kernel. See Table 44.1, “Audit status flags” for details.
4	Enable or disable the audit subsystem.

Using audit, you can track any kind of file system access to important files, configurations or resources. You can add watches on these and assign keys to each kind of watch for better identification in the logs.

Example 44.4: Example audit rules—file system auditing #

-w /etc/shadow1
-w /etc -p rx2
-w /etc/passwd -k fk_passwd -p rwxa3

1	The `-w` option tells audit to add a watch to the file specified, in this case `/etc/shadow`. All system calls requesting access permissions to this file are analyzed.
2	This rule adds a watch to the `/etc` directory and applies permission filtering for read and execute access to this directory (`-p rx`). Any system call requesting any of these two permissions is analyzed. The creation of new files and the deletion of existing ones are logged as directory-related events. To get more specific events for files located under this particular directory, you should add a separate rule for each file. A file must exist before you add a rule containing a watch on it. Auditing files as they are created is not supported.
3	This rule adds a file watch to `/etc/passwd`. Permission filtering is applied for read, write, execute, and attribute change permissions. The `-k` option allows you to specify a key to use to filter the audit logs for this particular event later (for example with `ausearch`). You may use the same key on different rules to be able to group rules when searching for them. It is also possible to apply multiple keys to a rule.

System call auditing lets you track your system's behavior on a level even below the application level. When designing these rules, consider that auditing a great many system calls may increase your system load and cause you to run out of disk space. Consider carefully which events need tracking and how they can be filtered to be even more specific.

Example 44.5: Example audit rules—system call auditing #

-a exit,always -S mkdir1
-a exit,always -S access -F a1=42
-a exit,always -S ipc -F a0=23
-a exit,always -S open -F success!=04
-a task,always -F auid=05
-a task,always -F uid=0 -F auid=501 -F gid=wheel6

1	This rule activates auditing for the `mkdir` system call. The `-a` option adds system call rules. This rule triggers an event whenever the `mkdir` system call is entered (`exit`, `always`). The `-S` option specifies the system call to which this rule should be applied.
2	This rule adds auditing to the access system call, but only if the second argument of the system call (`mode`) is `4` (`R_OK`). `exit,always` tells audit to add an audit context to this system call when entering it, and to write out a report when it gets audited.
3	This rule adds an audit context to the IPC multiplexed system call. The specific `ipc` system call is passed as the first syscall argument and can be selected using `-F a0=IPC_CALL_NUMBER`.
4	This rule audits failed attempts to call open.
5	This rule is an example of a task rule (keyword: `task`). It is different from the other rules above in that it applies to processes that are forked or cloned. To filter these kinds of events, you can use fields that are known at fork time, such as UID, GID, and AUID. This example rule filters for all tasks carrying an audit ID of `0`.
6	This last rule makes heavy use of filters. All filter options are combined with a logical AND operator, meaning that this rule applies to all tasks that carry the audit ID of `501`, run as `root`, and have `wheel` as the group. A process is given an audit ID on user login. This ID is then handed down to any child process started by the initial process of the user. Even if the user changes their identity, the audit ID stays the same and allows tracing actions to the original user.

Tip: Filtering system call arguments

For more details on filtering system call arguments, refer to Section 46.6, “Filtering system call arguments”.

You cannot only add rules to the audit system, but also remove them. There are different methods for deleting the entire rule set at once or for deleting system call rules or file and directory watches:

Example 44.6: Deleting audit rules and events #

-D1
-d exit,always -S mkdir2
-W /etc3

1	Clear the queue of audit rules and delete any preexisting rules. This rule is used as the first rule in `/etc/audit/audit.rules` files to make sure that the rules that are about to be added do not clash with any preexisting ones. The `auditctl` `-D` command is also used before doing an `autrace` to avoid having the trace rules clash with any rules present in the `audit.rules` file.
2	This rule deletes a system call rule. The `-d` option must precede any system call rule that needs to be deleted from the rule queue, and must match exactly.
3	This rule tells audit to discard the rule with the directory watch on `/etc` from the rules queue. This rule deletes any rule containing a directory watch on `/etc`, regardless of any permission filtering or key options.

To get an overview of which rules are currently in use in your audit setup, run auditctl -l. This command displays all rules with one rule per line.

Example 44.7: Listing rules with auditctl -l #

exit,always watch=/etc perm=rx
exit,always watch=/etc/passwd perm=rwxa key=fk_passwd
exit,always watch=/etc/shadow perm=rwxa
exit,always syscall=mkdir
exit,always a1=4 (0x4) syscall=access
exit,always a0=2 (0x2) syscall=ipc
exit,always success!=0 syscall=open

Note: Creating filter rules

You can build sophisticated audit rules by using the various filter options. Refer to the auditctl(8) man page for more information about the options available for building audit filter rules and audit rules.

44.5 Understanding the audit logs and generating reports #

To understand what the aureport utility does, it is vital to know how the logs generated by the audit daemon are structured, and what exactly is recorded for an event. Then you can decide which report types are most appropriate for your needs.

44.5.1 Understanding the audit logs #

The following examples highlight two typical events that are logged by audit and how their trails in the audit log are read. The audit log or logs (if log rotation is enabled) are stored in the /var/log/audit directory.

The logs record two types of information: record types and event fields. The record types are identified by type= in each log entry. Event fields are all other items on the left side of the equals signs. In the following examples, type=SYSCALL and type=CWDare record types, and arch=c000003e and syscall=2 are event fields, followed by their values.

Refer to the /usr/include/libaudit.h file (from the audit-devel package) to see a complete list of record types and their definitions.

Run the ausyscall --dump command to see a table of syscall numbers, and what they represent:

> ausyscall --dump
Using x86_64 syscall table:
0       read
1       write
2       open
3       close
4       stat
5       fstat
[...]

The first example is a simple less command. The second example covers a great deal of PAM activity in the logs when a user tries to remotely log in to a machine running audit.

Example 44.8: A simple audit event—viewing the audit log #

type=SYSCALL msg=audit(1234874638.599:5207): arch=c000003e syscall=2
success=yes exit=4 a0=62fb60 a1=0 a2=31 a3=0 items=1 ppid=25400 pid
=25616 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
tty=pts1 ses=1164 comm="less" exe="/usr/bin/less" key="doc_log"
type=CWD msg=audit(1234874638.599:5207):  cwd="/root"
type=PATH msg=audit(1234874638.599:5207): item=0 name="/var/log/audit/
audit.log" inode=1219041 dev=08:06 mode=0100644 ouid=0 ogid=0 rdev=00:00

The above event, a simple less /var/log/audit/audit.log, wrote three messages to the log. All of them are closely linked together and you would not be able to make sense of one of them without the others. The first message reveals the following information:

type

The type of event recorded. In this case, it assigns the SYSCALL type to an event triggered by a system call. The CWD event was recorded to record the current working directory at the time of the syscall. A PATH event is generated for each path passed to the system call. The open system call takes one path argument and generates one PATH event. It is important to understand that the PATH event reports the path name string argument without any further interpretation, so a relative path requires manual combination with the path reported by the CWD event to determine the object accessed.

msg

A message ID enclosed in brackets. The ID splits into two parts. All characters before the : represent a Unix epoch time stamp. The number after the colon represents the actual event ID. All events that are logged from one application's system call have the same event ID. If the application makes a second system call, it gets another event ID.

arch

References the CPU architecture of the system call. Decode this information using the -i option on any of your ausearch commands when searching the logs.

syscall

The type of system call as it would have been printed by an strace on this particular system call. This data is taken from the list of system calls under /usr/include/asm/unistd.h and may vary depending on the architecture. In this case, syscall=2 refers to the open system call (see man open(2)) invoked by the less application.

success

Whether the system call succeeded or failed.

exit

The exit value returned by the system call. For the open system call used in this example, this is the file descriptor number. This varies by system call.

a0 to a3

The first four arguments to the system call in numeric form. The values of these are system call dependent. In this example (an open system call), the following are used:

a0=62fb60 a1=8000 a2=31 a3=0

a0 is the start address of the passed path. a1 is the flags. 8000 in hex notation translates to 100000 in octal notation, which in turn translates to O_LARGEFILE. a2 is the mode, which, because O_CREAT was not specified, is unused. a3 is not passed by the open system call. Check the manual page of the relevant system call to find out which arguments are used with it.

items

The number of strings passed to the application.

ppid

The process ID of the parent of the process analyzed.

pid

The process ID of the process analyzed.

auid

The audit ID. A process is given an audit ID on user login. This ID is then handed down to any child process started by the initial process of the user. Even if the user changes their identity (for example, becomes root), the audit ID stays the same. Thus you can always trace actions to the original user who logged in.

uid

The user ID of the user who started the process. In this case, 0 for root.

gid

The group ID of the user who started the process. In this case, 0 for root.

euid, suid, fsuid

Effective user ID, set user ID, and file system user ID of the user that started the process.

egid, sgid, fsgid

Effective group ID, set group ID, and file system group ID of the user that started the process.

tty

The terminal from which the application was started. In this case, a pseudo-terminal used in an SSH session.

ses

The login session ID. This process attribute is set when a user logs in and can tie any process to a particular user login.

comm

The application name under which it appears in the task list.

exe

The resolved path to the binary program.

subj

auditd records whether the process is subject to any security context, such as AppArmor. unconstrained, as in this case, means that the process is not confined with AppArmor. If the process had been confined, the binary path plus the AppArmor profile mode would have been logged.

key

If you are auditing many directories or files, assign key strings to each of these watches. You can use these keys with ausearch to search the logs for events of this type.

The second message triggered by the example less call does not reveal anything apart from the current working directory when the less command was executed.

The third message reveals the following (the type and message flags have already been introduced):

item: In this example, item references the a0 argument—a path—that is associated with the original SYSCALL message. Had the original call had more than one path argument (such as a cp or mv command), an additional PATH event would have been logged for the second path argument.
name: Refers to the path passed as an argument to the open system call.
inode: Refers to the inode number corresponding to name.
dev: Specifies the device on which the file is stored. In this case, 08:06, which stands for /dev/sda1 or “first partition on the first IDE device.”
mode: Numerical representation of the file's access permissions. In this case, root has read and write permissions and their group (root) has read access while the entire rest of the world cannot access the file.
ouid and ogid: Refer to the UID and GID of the inode itself.
rdev: Not applicable for this example. The rdev entry applies to block or character devices, not to files.

Example 44.9, “An advanced audit event—login via SSH” highlights the audit events triggered by an incoming SSH connection. Most of the messages are related to the PAM stack and reflect the different stages of the SSH PAM process. Several of the audit messages carry nested PAM messages in them that signify that a particular stage of the PAM process has been reached. Although the PAM messages are logged by audit, audit assigns its own message type to each event:

Example 44.9: An advanced audit event—login via SSH #

type=USER_AUTH msg=audit(1234877011.791:7731): user pid=26127 uid=0 1
auid=4294967295 ses=4294967295 msg='op=PAM:authentication acct="root" exe="/usr/sbin/sshd"
(hostname=jupiter.example.com, addr=192.168.2.100, terminal=ssh res=success)'
type=USER_ACCT msg=audit(1234877011.795:7732): user pid=26127 uid=0 2
auid=4294967295 ses=4294967295 msg='op=PAM:accounting acct="root" exe="/usr/sbin/sshd"
(hostname=jupiter.example.com, addr=192.168.2.100, terminal=ssh res=success)'
type=CRED_ACQ msg=audit(1234877011.799:7733): user pid=26125 uid=0 3
auid=4294967295 ses=4294967295 msg='op=PAM:setcred acct="root" exe="/usr/sbin/sshd"
(hostname=jupiter.example.com, addr=192.168.2.100, terminal=/dev/pts/0 res=success)'
type=LOGIN msg=audit(1234877011.799:7734): login pid=26125 uid=0
old auid=4294967295 new auid=0 old ses=4294967295 new ses=1172
type=USER_START msg=audit(1234877011.799:7735): user pid=26125 uid=0 4
auid=0 ses=1172 msg='op=PAM:session_open acct="root" exe="/usr/sbin/sshd"
(hostname=jupiter.example.com, addr=192.168.2.100, terminal=/dev/pts/0 res=success)'
type=USER_LOGIN msg=audit(1234877011.823:7736): user pid=26128 uid=0 5
auid=0 ses=1172 msg='uid=0: exe="/usr/sbin/sshd"
(hostname=jupiter.example.com, addr=192.168.2.100, terminal=/dev/pts/0 res=success)'
type=CRED_REFR msg=audit(1234877011.828:7737): user pid=26128 uid=0 6
auid=0 ses=1172 msg='op=PAM:setcred acct="root" exe="/usr/sbin/sshd"
(hostname=jupiter.example.com, addr=192.168.2.100, terminal=/dev/pts/0 res=success)'

1	PAM reports that is has successfully requested user authentication for `root` from a remote host (jupiter.example.com, 192.168.2.100). The terminal where this is happening is `ssh`.
2	PAM reports that it has successfully determined whether the user is authorized to log in.
3	PAM reports that the appropriate credentials to log in have been acquired and that the terminal changed to a normal terminal (`/dev/pts0`).
4	PAM reports that it has successfully opened a session for `root`.
5	The user has successfully logged in. This event is the one used by `aureport` `-l` to report about user logins.
6	PAM reports that the credentials have been successfully reacquired.

44.5.2 Generating custom audit reports #

The raw audit reports stored in the /var/log/audit directory tend to become bulky and hard to understand. To more easily find relevant messages, use the aureport utility and create custom reports.

The following use cases highlight a few of the possible report types that you can generate with aureport:

Read audit logs from another file

When the audit logs have moved to another machine or when you want to analyze the logs of several machines on your local machine without wanting to connect to each of these individually, move the logs to a local file and have aureport analyze them locally:

> sudo aureport -if myfile

Summary Report
======================
Range of time in logs: 03/02/09 14:13:38.225 - 17/02/09 14:52:27.971
Selected time for report: 03/02/09 14:13:38 - 17/02/09 14:52:27.971
Number of changes in configuration: 13
Number of changes to accounts, groups, or roles: 0
Number of logins: 6
Number of failed logins: 13
Number of authentications: 7
Number of failed authentications: 573
Number of users: 1
Number of terminals: 9
Number of host names: 4
Number of executables: 17
Number of files: 279
Number of AVC's: 0
Number of MAC events: 0
Number of failed syscalls: 994
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of keys: 2
Number of process IDs: 1211
Number of events: 5320

The above command, aureport without any arguments, provides the standard general summary report generated from the logs contained in myfile. To create more detailed reports, combine the -if option with any of the options below. For example, generate a login report that is limited to a certain time frame:

> sudo aureport -l -ts 14:00 -te 15:00 -if myfile

Login Report
============================================
# date time auid host term exe success event
============================================
1. 17/02/09 14:21:09 root: 192.168.2.100 sshd /usr/sbin/sshd no 7718
2. 17/02/09 14:21:15 0 jupiter /dev/pts/3 /usr/sbin/sshd yes 7724

Convert numeric entities to text

Some information, such as user IDs, are printed in numeric form. To convert these into a human-readable text format, add the -i option to your aureport command.

Create a rough summary report

If you are interested in the current audit statistics (events, logins, processes, etc.), run aureport without any other option.

Create a summary report of failed events

To break down the overall statistics of plain aureport to the statistics of failed events, use aureport --failed:

> sudo aureport --failed

Failed Summary Report
======================
Range of time in logs: 03/02/09 14:13:38.225 - 17/02/09 14:57:35.183
Selected time for report: 03/02/09 14:13:38 - 17/02/09 14:57:35.183
Number of changes in configuration: 0
Number of changes to accounts, groups, or roles: 0
Number of logins: 0
Number of failed logins: 13
Number of authentications: 0
Number of failed authentications: 574
Number of users: 1
Number of terminals: 5
Number of host names: 4
Number of executables: 11
Number of files: 77
Number of AVC's: 0
Number of MAC events: 0
Number of failed syscalls: 994
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of keys: 2
Number of process IDs: 708
Number of events: 1583

Create a summary report of successful events

If you want to break down the overall statistics of a plain aureport to the statistics of successful events, use aureport --success:

> sudo aureport --success

Success Summary Report
======================
Range of time in logs: 03/02/09 14:13:38.225 - 17/02/09 15:00:01.535
Selected time for report: 03/02/09 14:13:38 - 17/02/09 15:00:01.535
Number of changes in configuration: 13
Number of changes to accounts, groups, or roles: 0
Number of logins: 6
Number of failed logins: 0
Number of authentications: 7
Number of failed authentications: 0
Number of users: 1
Number of terminals: 7
Number of host names: 3
Number of executables: 16
Number of files: 215
Number of AVC's: 0
Number of MAC events: 0
Number of failed syscalls: 0
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of keys: 2
Number of process IDs: 558
Number of events: 3739

Create summary reports

In addition to the dedicated summary reports (main summary and failed and success summary), use the --summary option with most of the other options to create summary reports for a particular area of interest. Not all reports support this option, however. This example creates a summary report for user login events:

> sudo aureport -u -i --summary

User Summary Report
===========================
total  auid
===========================
5640  root
13  tux
3  wilber

Create a report of events

To get an overview of the events logged by audit, use the aureport -e command. This command generates a numbered list of all events including date, time, event number, event type, and audit ID.

> sudo aureport -e -ts 14:00 -te 14:21

Event Report
===================================
# date time event type auid success
===================================
1. 17/02/09 14:20:27 7462 DAEMON_START 0 yes
2. 17/02/09 14:20:27 7715 CONFIG_CHANGE 0 yes
3. 17/02/09 14:20:57 7716 USER_END 0 yes
4. 17/02/09 14:20:57 7717 CRED_DISP 0 yes
5. 17/02/09 14:21:09 7718 USER_LOGIN -1 no
6. 17/02/09 14:21:15 7719 USER_AUTH -1 yes
7. 17/02/09 14:21:15 7720 USER_ACCT -1 yes
8. 17/02/09 14:21:15 7721 CRED_ACQ -1 yes
9. 17/02/09 14:21:15 7722 LOGIN 0 yes
10. 17/02/09 14:21:15 7723 USER_START 0 yes
11. 17/02/09 14:21:15 7724 USER_LOGIN 0 yes
12. 17/02/09 14:21:15 7725 CRED_REFR 0 yes

Create a report from all process events

To analyze the log from a process's point of view, use the aureport -p command. This command generates a numbered list of all process events including date, time, process ID, name of the executable, system call, audit ID, and event number.

aureport -p

Process ID Report
======================================
# date time pid exe syscall auid event
======================================
1. 13/02/09 15:30:01 32742 /usr/sbin/cron 0 0 35
2. 13/02/09 15:30:01 32742 /usr/sbin/cron 0 0 36
3. 13/02/09 15:38:34 32734 /usr/lib/gdm/gdm-session-worker 0 -1 37

Create a report from all system call events

To analyze the audit log from a system call's point of view, use the aureport -s command. This command generates a numbered list of all system call events including date, time, number of the system call, process ID, name of the command that used this call, audit ID, and event number.

> sudo aureport -s

Syscall Report
=======================================
# date time syscall pid comm auid event
=======================================
1. 16/02/09 17:45:01 2 20343 cron -1 2279
2. 16/02/09 17:45:02 83 20350 mktemp 0 2284
3. 16/02/09 17:45:02 83 20351 mkdir 0 2285

Create a report from all executable events

To analyze the audit log from an executable's point of view, use the aureport -x command. This command generates a numbered list of all executable events including date, time, name of the executable, the terminal it is run in, the host executing it, the audit ID, and event number.

aureport -x

Executable Report
====================================
# date time exe term host auid event
====================================
1. 13/02/09 15:08:26 /usr/sbin/sshd sshd 192.168.2.100 -1 12
2. 13/02/09 15:08:28 /usr/lib/gdm/gdm-session-worker :0 ? -1 13
3. 13/02/09 15:08:28 /usr/sbin/sshd ssh 192.168.2.100 -1 14

Create a report about files

To generate a report from the audit log that focuses on file access, use the aureport -f command. This command generates a numbered list of all file-related events including date, time, name of the accessed file, number of the system call accessing it, success or failure of the command, the executable accessing the file, audit ID, and event number.

> sudo aureport -f

File Report
===============================================
# date time file syscall success exe auid event
===============================================
1. 16/02/09 17:45:01 /etc/shadow 2 yes /usr/sbin/cron -1 2279
2. 16/02/09 17:45:02 /tmp/ 83 yes /bin/mktemp 0 2284
3. 16/02/09 17:45:02 /var 83 no /bin/mkdir 0 2285

Create a report about users

To generate a report from the audit log that illustrates which users are running what executables on your system, use the aureport -u command. This command generates a numbered list of all user-related events including date, time, audit ID, terminal used, host, name of the executable, and an event ID.

aureport -u

User ID Report
====================================
# date time auid term host exe event
====================================
1. 13/02/09 15:08:26 -1 sshd 192.168.2.100 /usr/sbin/sshd 12
2. 13/02/09 15:08:28 -1 :0 ? /usr/lib/gdm/gdm-session-worker 13
3. 14/02/09 08:25:39 -1 ssh 192.168.2.101 /usr/sbin/sshd 14

Create a report about logins

To create a report that focuses on login attempts to your machine, run the aureport -l command. This command generates a numbered list of all login-related events including date, time, audit ID, host and terminal used, name of the executable, success or failure of the attempt, and an event ID.

> sudo aureport -l -i

Login Report
============================================
# date time auid host term exe success event
============================================
1. 13/02/09 15:08:31 tux: 192.168.2.100 sshd /usr/sbin/sshd no 19
2. 16/02/09 12:39:05 root: 192.168.2.101 sshd /usr/sbin/sshd no 2108
3. 17/02/09 15:29:07 geeko: ? tty3 /bin/login yes 7809

Limit a report to a certain time frame

To analyze the logs for a particular time frame, such as the working hours of Feb 16, 2009, first find out whether this data is contained in the current audit.log or whether the logs have been rotated in by running aureport -t:

aureport -t

Log Time Range Report
=====================
/var/log/audit/audit.log: 03/02/09 14:13:38.225 - 17/02/09 15:30:01.636

The current audit.log contains all the desired data. Otherwise, use the -if option to point the aureport commands to the log file that contains the needed data.

Then, specify the start date and time and the end date and time of the desired time frame and combine it with the report option needed. This example focuses on login attempts:

> sudo aureport -ts 02/16/09 8:00 -te 02/16/09 18:00 -l

Login Report
============================================
# date time auid host term exe success event
============================================
1. 16/02/09 12:39:05 root: 192.168.2.100 sshd /usr/sbin/sshd no 2108
2. 16/02/09 12:39:12 0 192.168.2.100 /dev/pts/1 /usr/sbin/sshd yes 2114
3. 16/02/09 13:09:28 root: 192.168.2.100 sshd /usr/sbin/sshd no 2131
4. 16/02/09 13:09:32 root: 192.168.2.100 sshd /usr/sbin/sshd no 2133
5. 16/02/09 13:09:37 0 192.168.2.100 /dev/pts/2 /usr/sbin/sshd yes 2139

The start date and time are specified with the -ts option. Any event that has a time stamp equal to or after your given start time appears in the report. If you omit the date, aureport assumes that you meant today. If you omit the time, it assumes that the start time should be midnight of the date specified.

Specify the end date and time with the -te option. Any event that has a time stamp equal to or before your given event time appears in the report. If you omit the date, aureport assumes that you meant today. If you omit the time, it assumes that the end time should be now. Use the same format for the date and time as for -ts.

All reports except the summary ones are printed in column format and sent to STDOUT, which means that this data can be written to other commands easily. The visualization scripts introduced in Section 44.8, “Visualizing audit data” are examples of how to further process the data generated by audit.

44.6 Querying the audit daemon logs with `ausearch` #

The aureport tool helps you to create overall summaries of what is happening on the system, but if you are interested in the details of a particular event, ausearch is the tool to use.

ausearch allows you to search the audit logs using special keys and search phrases that relate to most of the flags that appear in event messages in /var/log/audit/audit.log. Not all record types contain the same search phrases. There are no hostname or uid entries in a PATH record, for example.

When searching, make sure that you choose appropriate search criteria to catch all records you need. Otherwise, you could be searching for a specific type of record and still get various other related records along with it. This is caused by different parts of the kernel contributing additional records for events that are related to the one to find. For example, you would always get a PATH record along with the SYSCALL record for an open system call.

Tip: Using multiple search options

Any of the command line options can be combined with logical AND operators to narrow down your search.

Read audit logs from another file

> sudo ausearch - option -if myfile

Convert numeric results into text

Some information, such as user IDs are printed in numeric form. To convert these into human readable text format, add the -i option to your ausearch command.

Search by audit event ID

If you have previously run an audit report or done an autrace, you should analyze the trail of a particular event in the log. Most of the report types described in Section 44.5, “Understanding the audit logs and generating reports” include audit event IDs in their output. An audit event ID is the second part of an audit message ID, which consists of a Unix epoch time stamp and the audit event ID separated by a colon. All events that are logged from one application's system call have the same event ID. Use this event ID with ausearch to retrieve this event's trail from the log.

Use a command similar to the following:

> sudo ausearch -a 5207
----
time->Tue Feb 17 13:43:58 2009
type=PATH msg=audit(1234874638.599:5207): item=0 name="/var/log/audit/audit.log" inode=1219041 dev=08:06 mode=0100644 ouid=0 ogid=0 rdev=00:00
type=CWD msg=audit(1234874638.599:5207):  cwd="/root"
type=SYSCALL msg=audit(1234874638.599:5207): arch=c000003e syscall=2 success=yes exit=4 a0=62fb60 a1=0 a2=31 a3=0 items=1 ppid=25400 pid=25616 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=1164 comm="less" exe="/usr/bin/less" key="doc_log"

The ausearch -a command grabs all records in the logs that are related to the audit event ID provided and displays them. This option can be combined with any other option.

Search by message type

To search for audit records of a particular message type, use the ausearch -m MESSAGE_TYPE command. Examples of valid message types include PATH, SYSCALL, and USER_LOGIN. Running ausearch -m without a message type displays a list of all message types.

Search by login ID

To view records associated with a particular login user ID, use the ausearch -ul command. It displays any records related to the user login ID specified provided that user had been able to log in successfully.

Search by user ID

View records related to any of the user IDs (both user ID and effective user ID) with ausearch -ua. View reports related to a particular user ID with ausearch -ui UID. Search for records related to a particular effective user ID, use the ausearch -ue EUID. Searching for a user ID means the user ID of the user creating a process. Searching for an effective user ID means the user ID and privileges that are required to run this process.

Search by group ID

View records related to any of the group IDs (both group ID and effective group ID) with the ausearch -ga command. View reports related to a particular user ID with ausearch -gi GID. Search for records related to a particular effective group ID, use ausearch -ge EGID.

Search by command line name

View records related to a certain command, using the ausearch -c COMM_NAME command, for example, ausearch -c less for all records related to the less command.

Search by executable name

View records related to a certain executable with the ausearch -x EXE command, for example ausearch -x /usr/bin/less for all records related to the /usr/bin/less executable.

Search by system call name

View records related to a certain system call with the ausearch -sc SYSCALL command, for example, ausearch -sc open for all records related to the open system call.

Search by process ID

View records related to a certain process ID with the ausearch -p PID command, for example ausearch -p 13368 for all records related to this process ID.

Search by event or system call success value

View records containing a certain system call success value with ausearch -sv SUCCESS_VALUE, for example, ausearch -sv yes for all successful system calls.

Search by file name

View records containing a certain file name with ausearch -f FILE_NAME, for example, ausearch -f /foo/bar for all records related to the /foo/bar file. Using the file name alone would work as well, but using relative paths does not work.

Search by terminal

View records of events related to a certain terminal with ausearch -tm TERM, for example, ausearch -tm ssh to view all records related to events on the SSH terminal and ausearch -tm tty to view all events related to the console.

Search by host name

View records related to a certain remote host name with ausearch -hn HOSTNAME, for example, ausearch -hn jupiter.example.com. You can use a host name, fully qualified domain name, or numeric network address.

Search by key field

View records that contain a certain key assigned in the audit rule set to identify events of a particular type. Use the ausearch -k KEY_FIELD, for example, ausearch -k CFG_etc to display any records containing the CFG_etc key.

Search by word

View records that contain a certain string assigned in the audit rule set to identify events of a particular type. The whole string is matched on file name, host name, and terminal. Use the ausearch -w WORD.

Limit a search to a certain time frame

Use -ts and -te to limit the scope of your searches to a certain time frame. The -ts option is used to specify the start date and time and the -te option is used to specify the end date and time. These options can be combined with any of the above. The use of these options is similar to use with aureport.

44.7 Analyzing processes with `autrace` #

Other than monitoring your system using the rules you set up, you can also perform dedicated audits of individual processes using the autrace command. autrace works similarly to the strace command, but gathers slightly different information. The output of autrace is written to /var/log/audit/audit.log and does not look any different from the standard audit log entries.

When performing an autrace on a process, make sure that any audit rules are purged from the queue to avoid these rules clashing with the ones autrace adds itself. Delete the audit rules with the auditctl -D command. This stops all normal auditing.

> sudo auditctl -D

No rules

autrace /usr/bin/less

Waiting to execute: /usr/bin/less
Cleaning up...
No rules
Trace complete. You can locate the records with 'ausearch -i -p 7642'

Always use the full path to the executable to track with autrace. After the trace is complete, autrace provides the event ID of the trace, so you can analyze the entire data trail with ausearch. To restore the audit system to use the audit rule set again, restart the audit daemon with systemctl restart auditd.

44.8 Visualizing audit data #

Neither the data trail in /var/log/audit/audit.log nor the different report types generated by aureport, described in Section 44.5.2, “Generating custom audit reports”, provide an intuitive reading experience to the user. The aureport output is formatted in columns and thus easily available to any sed, Perl or awk scripts that users might connect to the audit framework to visualize the audit data.

The visualization scripts (see Section 45.6, “Configuring log visualization”) are one example of how to use standard Linux tools available with SUSE Linux Enterprise Server or any other Linux distribution to create easy-to-read audit output. The following examples help you understand how the plain audit reports can be transformed into human readable graphics.

The first example illustrates the relationship of programs and system calls. To get to this kind of data, you need to determine the appropriate aureport command that delivers the source data from which to generate the final graphic:

> sudo aureport -s -i

Syscall Report
=======================================
# date time syscall pid comm auid event
=======================================
1. 16/02/09 17:45:01 open 20343 cron unset 2279
2. 16/02/09 17:45:02 mkdir 20350 mktemp root 2284
3. 16/02/09 17:45:02 mkdir 20351 mkdir root 2285
...

The first thing that the visualization script needs to do on this report is to extract those columns that are of interest. In this example, they are the syscall and the comm columns. The output is sorted and duplicates removed; then the final output is written into the visualization program itself:

LC_ALL=C aureport -s -i | awk '/^[0-9]/ { print $6" "$4 }' | sort | uniq | mkgraph

Figure 44.2: Flow graph—program versus system call relationship #

The second example illustrates the different types of events and how many of each type have been logged. The appropriate aureport command to extract this kind of information is aureport -e:

> sudo aureport -e -i --summary

Event Summary Report
======================
total  type
======================
2434  SYSCALL
816  USER_START
816  USER_ACCT
814  CRED_ACQ
810  LOGIN
806  CRED_DISP
779  USER_END
99  CONFIG_CHANGE
52  USER_LOGIN

Because this type of report already contains a two-column output, it is fed into the visualization script and transformed into a bar chart.

> sudo aureport -e -i --summary  | mkbar events

Figure 44.3: Bar chart—common event types #

For background information about the visualization of audit data, refer to the Web site of the audit project at https://people.redhat.com/sgrubb/audit/visualize/index.html.

44.9 Relaying audit event notifications #

The auditing system also allows external applications to access and use the auditd daemon in real time. This feature is provided by so called audit dispatcher which allows, for example, intrusion detection systems to use auditd to receive enhanced detection information.

The configuration of auditd is stored in /etc/audit/auditd.conf. The file includes the following options:

q_depth: Specifies the size of the event dispatcher internal queue. If syslog complains about audit events getting dropped, increase this value. Default is 250.
overflow_action: Specifies the way the audit daemon reacts to the internal queue overflow. Possible values are ignore (nothing happens), syslog (issues a warning to syslog), suspend (stop processing events), single (the computer system is put in single user mode), or halt (shuts the system down).
priority_boost: Specifies the priority for the audit event dispatcher (in addition to the audit daemon priority itself). Default is 4 which means no change in priority.
name_format: Specifies the way the computer node name is inserted into the audit event. Possible values are none (no computer name is inserted), hostname (name returned by the gethostname system call), fqd (fully qualified domain name of the machine), numeric (IP address of the machine), or user (user defined string from the name option). Default is none.
name: Specifies a user-defined string which identifies the machine. The name_format option must be set to user, otherwise this option is ignored.
max_restarts: A non-negative number that tells the audit event dispatcher how many times it can try to restart a crashed plug-in. The default is 10.

Example 44.10: Example /etc/audit/auditd.conf #

  q_depth = 250
  overflow_action = SYSLOG
  priority_boost = 4
  name_format = HOSTNAME
  #name = mydomain

The plug-in programs install their configuration files in a special directory, /etc/audit/plugins.d. The plug-in configuration files have the following options:

active: Specifies if the program uses auditd. Possible values are yes or no.
direction: Specifies the way the plug-in was designed to communicate with audit. It informs the event dispatcher in which directions the events flow. Possible values are in or out.
path: Specifies the absolute path to the plug-in executable. In case of internal plug-ins, this option specifies the plug-in name.
type: Specifies the way the plug-in is to be run. Possible values are builtin or always. Use builtin for internal plug-ins (af_unix and syslog) and always for most (if not all) other plug-ins. Default is always.
args: Specifies the argument that is passed to the plug-in program. Normally, plug-in programs read their arguments from their configuration file and do not need to receive any arguments. There is a limit of two arguments.
format: Specifies the format of data that the audit dispatcher passes to the plug-in program. Valid options are binary or string. binary passes the data exactly as the event dispatcher receives them from the audit daemon. string instructs the dispatcher to change the event into a string that is parsable by the audit parsing library. Default is string.

Example 44.11: Example /etc/audit/plugins.d/syslog.conf #

  active = no
  direction = out
  path = /sbin/audisp-syslog
  type = builtin
  args = LOG_INFO
  format = string