Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / SUSE Linux Enterprise Micro Documentation / Quick Start Guides / Administration Guide
SUSE Linux Enterprise Micro 5.1

Administration Guide

  
Publication Date: November 14, 2024

This guide describes the administration of SUSE Linux Enterprise Micro.

1 Snapshots

  
Warning
Warning: Snapshots are mandatory

As snapshots are crucial for the correct functioning of SLE Micro, do not disable the feature, and ensure that the root partition is big enough to store the snapshots.

When a snapshot is created, both the snapshot and the original point to the same blocks in the file system. So, initially a snapshot does not occupy additional disk space. If data in the original file system is modified, changed data blocks are copied while the old data blocks are kept for the snapshot.

Snapshots always reside on the same partition or subvolume on which the snapshot has been taken. It is not possible to store snapshots on a different partition or subvolume. As a result, partitions containing snapshots need to be larger than partitions which do not contain snapshots. The exact amount depends strongly on the number of snapshots you keep and the amount of data modifications. As a rule of thumb, give partitions twice as much space as you normally would. To prevent disks from running out of space, old snapshots are automatically cleaned up.

Snapshot that are known to be working properly are marked as important.

1.1 Directories excluded from snapshots

  

As some directories store user specific or volatile data, these directories are excluded from snapshots:

/home

Contains users' data. Excluded so that the data will not be included in snapshots and thus potentially overwritten by a rollback operation.

/root

Contains root's data. Excluded so that the data will not be included in snapshots and thus potentially overwritten by a rollback operation.

/opt

Third-party products usually get installed to /opt. Excluded so that these applications are not uninstalled during rollbacks.

/srv

Contains data for Web and FTP servers. Excluded in order to avoid data loss on rollbacks.

/usr/local

This directory is used when manually installing software. It is excluded to avoid uninstalling these installations on rollbacks.

/var

This directory contains many variable files, including logs, temporary caches, third-party products in /var/opt, and is the default location for virtual machine images and databases. Therefore, a separate subvolume is created with Copy-On-Write disabled, so as to exclude all of this variable data from snapshots.

/tmp

The directory contains temporary data.

the architecture-specific /boot/grub2 directory

Rollback of the boot loader binaries is not supported.

1.2 Showing exclusive disk space used by snapshots

  

Snapshots share data, for efficient use of storage space, so using ordinary commands like du and df won't measure used disk space accurately. When you want to free up disk space on Btrfs with quotas enabled, you need to know how much exclusive disk space is used by each snapshot, rather than shared space. The btrfs command provides a view of space used by snapshots: 

# btrfs qgroup show -p /
qgroupid         rfer         excl parent  
--------         ----         ---- ------  
0/5          16.00KiB     16.00KiB ---     
[...]    
0/272         3.09GiB     14.23MiB 1/0     
0/273         3.11GiB    144.00KiB 1/0     
0/274         3.11GiB    112.00KiB 1/0     
0/275         3.11GiB    128.00KiB 1/0     
0/276         3.11GiB     80.00KiB 1/0     
0/277         3.11GiB    256.00KiB 1/0     
0/278         3.11GiB    112.00KiB 1/0     
0/279         3.12GiB     64.00KiB 1/0     
0/280         3.12GiB     16.00KiB 1/0     
1/0           3.33GiB    222.95MiB ---

The qgroupid column displays the identification number for each subvolume, assigning a qgroup level/ID combination.

The rfer column displays the total amount of data referred to in the subvolume.

The excl column displays the exclusive data in each subvolume.

The parent column shows the parent qgroup of the subvolumes.

The final item, 1/0, shows the totals for the parent qgroup. In the above example, 222.95 MiB will be freed if all subvolumes are removed. Run the following command to see which snapshots are associated with each subvolume: 

# btrfs subvolume list -st /

2 Administration using transactional updates

  

SLE Micro was designed to use a read-only root file system. This means that after the deployment is complete, you are not able to perform direct modifications to the root file system, e.g. by using zypper. Instead, SUSE Linux Enterprise Micro introduces the concept of transactional updates which enables you to modify your system and keep it up to date.

The key features of transactional updates are the following:

  • They are atomic - the update is applied only if it completes successfully.

  • Changes are applied in a separate snapshot and so do not influence the running system.

  • Changes can easily be rolled back.

Each time you call the transactional-update command to change your system—either to install a package, perform an update or apply a patch—the following actions take place:

Procedure 1: Modifying the root file system
  

  1. A new read-write snapshot is created from your current root file system, or from a snapshot that you specified.

  2. All changes are applied (updates, patches or package installation).

  3. The snapshot is switched back to read-only mode.

  4. The new root file system snapshot is prepared, so that it will be active after you reboot.

  5. After rebooting, the new root file system is set as the default snapshot.

    Note
    Note

    Bear in mind that without rebooting your system, the changes will not be applied.

Warning
Warning

In case you do not reboot your machine before performing further changes, the transactional-update command will create a new snapshot from the current root file system. This means that you will end up with several parallel snapshots, each including that particular change but not changes from the other invocations of the command. After reboot, the most recently created snapshot will be used as your new root file system, and it will not include changes done in the previous snapshots.

2.1 transactional-update usage

  

The transactional-update command enables atomic installation or removal of updates; updates are applied only if all of them can be successfully installed. transactional-update creates a snapshot of your system and use it to update the system. Later you can restore this snapshot. All changes become active only after reboot.

The transactional-update command syntax is as follows: 

transactional-update [option] [general_command] [package_command] standalone_command
Note
Note: Running transactional-update without arguments.

If you do not specify any command or option while running the transactional-update command, the system updates itself.

Possible command parameters are described further.

transactional-update options
  
--interactive, -i

Can be used along with a package command to turn on interactive mode.

--non-interactive, -n

Can be used along with a package command to turn on non-interactive mode.

--continue [number], -c

The --continue option is for making multiple changes to an existing snapshot without rebooting.

The default transactional-update behavior is to create a new snapshot from the current root file system. If you forget something, such as installing a new package, you have to reboot to apply your previous changes, run transactional-update again to install the forgotten package, and reboot again. You cannot run the transactional-update command multiple times without rebooting to add more changes to the snapshot, because this will create separate independent snapshots that do not include changes from the previous snapshots.

Use the --continue option to make as many changes as you want without rebooting. A separate snapshot is made each time, and each snapshot contains all the changes you made in the previous snapshots, plus your new changes. Repeat this process as many times as you want, and when the final snapshot includes everything you want, reboot the system, and your final snapshot becomes the new root file system.

Another useful feature of the --continue option is you may select any existing snapshot as the base for your new snapshot. The following example demonstrates running transactional-update to install a new package in a snapshot based on snapshot 13, and then running it again to install another package: 

# transactional-update pkg install package_1
# transactional-update --continue 13 pkg install package_2
--no-selfupdate

Disables self updating of transactional-update.

--drop-if-no-change, -d

Discards the snapshot created by transactional-update if there were no changes to the root file system. If there are some changes to the /etc directory, those changes merged back to the current file system.

--quiet

The transactional-update command will not output to stdout.

--help, -h

Prints help for the transactional-update command.

--version

Displays the version of the transactional-update command.

The general commands are the following:

General commands
  
cleanup-snapshots

The command marks all unused snapshots that are intended to be removed.

cleanup-overlays

The command removes all unused overlay layers of /etc.

cleanup

The command combines the cleanup-snapshots and cleanup-overlays commands. For more details refer to Section 2.2, “Snapshots cleanup”.

grub.cfg

Use this command to rebuild the GRUB boot loader configuration file.

bootloader

The command reinstall the boot loader.

initrd

Use the command to rebuild initrd.

kdump

In case you perform changes to your hardware or storage, you may need to rebuild the kdump initrd.

shell

Opens a read-write shell in the new snapshot before exiting. The command is typically used for debugging purposes.

reboot

The system reboots after the transactional-update is complete.

run <command>

Runs the provided command in a new snapshot.

setup-selinux

Installs and enables targeted SELinux policy.

The package commands are the following:

Important
Important: Installing packages outside of the official SLE Micro repositories

The installation of packages from repositories other than the official ones (for example, the SUSE Linux Enterprise Server repositories) is not supported and not recommended. To use the tools available for SUSE Linux Enterprise Server, run the toolbox container and install the tools inside the container. For details about the toolbox container, refer to Section 5, “toolbox for SLE Micro debugging”.

Package commands
  
dup

Performs upgrade of your system. The default option for this command is --non-interactive.

migration

The command migrates your system to a selected target. Typically it is used to upgrade your system if it has been registered via SUSE Customer Center.

patch

Checks for available patches and installs them. The default option for this command is --non-interactive.

pkg install

Installs individual packages from the available channels using the zypper install command. This command can also be used to install Program Temporary Fix (PTF) RPM files. The default option for this command is --interactive. 

# transactional-update pkg install package_name

or 

# transactional-update pkg install rpm1 rpm2
pkg remove

Removes individual packages from the active snapshot using the zypper remove command. This command can also be used to remove PTF RPM files. The default option for this command is --interactive. 

# transactional-update pkg remove package_name
pkg update

Updates individual packages from the active snapshot using the zypper update command. Only packages that are part of the snapshot of the base file system can be updated. The default option for this command is --interactive. 

# transactional-update pkg update package_name
register

The register command enables you to register/deregister your system. For a complete usage description, refer to Section 2.1.1, “The register command”.

up

Updates installed packages to newer versions. The default option for this command is --non-interactive.

The standalone commands are the following:

Standalone commands
  
rollback <snapshot number>

This sets the default subvolume. The current system is set as the new default root file system. If you specify a number, that snapshot is used as the default root file system. On a read-only file system, it does not create any additional snapshots. 

# transactional-update rollback snapshot_number
rollback last

This command sets the last known to be working snapshot as the default.

status

This prints a list of available snapshots. The currently booted one is marked with an asterisk, the default snapshot is marked with a plus sign.

2.1.1 The register command

  

The register command enables you to handle all tasks regarding registration and subscription management. You can supply the following options:

--list-extensions

With this option, the command will list available extensions for your system. You can use the output to find a product identifier for product activation.

-p, --product

Use this option to specify a product for activation. The product identifier has the following format: <name>/<version>/<architecture>, for example sle-module-live-patching/15.3/x86_64. The appropriate command will then be the following: 

# transactional-update register -p sle-module-live-patching/15.3/x86_64
-r, --regcode

Register your system with the provided registration code. The command will register the subscription and enable software repositories.

-d, --de-register

The option deregisters the system, or when used along with the -p option, deregisters an extension.

-e, --email

Specify an email address that will be used in SUSE Customer Center for registration.

--url

Specify the URL of your registration server. The URL is stored in the configuration and will be used in subsequent command invocations. For example: 

# transactional-update register --url https://scc.suse.com
-s, --status

Displays the current registration status in JSON format.

--write-config

Writes the provided options value to the /etc/SUSEConnect configuration file.

--cleanup

Removes old system credentials.

--version

Prints the version.

--help

Displays usage of the command.

2.2 Snapshots cleanup

  

If you run the command transactional-update cleanup, all old snapshots without a cleanup algorithm will have one set. All important snapshots are also marked. The command also removes all unreferenced (and thus unused) /etc overlay directories in /var/lib/overlay.

The snapshots with the set number cleanup algorithm will be deleted according to the rules configured in /etc/snapper/configs/root by the following parameters:

NUMBER_MIN_AGE

Defines the minimum age of a snapshot (in seconds) that can be automatically removed.

NUMBER_LIMIT/NUMBER_LIMIT_IMPORTANT

Defines the maximum count of stored snapshots. The cleaning algorithms delete snapshots above the specified maximum value, without taking the snapshot and file system space into account. The algorithms also delete snapshots above the minimum value until the limits for the snapshot and file system are reached.

The snapshot cleanup is also preformed regularly by systemd.

2.3 System rollback

  

GRUB 2 enables booting from btrfs snapshots and thus allows you to use any older functional snapshot in case that the new snapshot does not work correctly.

When booting a snapshot, the parts of the file system included in the snapshot are mounted read-only; all other file systems and parts that are excluded from snapshots are mounted read-write and can be modified.

Tip
Tip: Rolling back to a specific installation state

An initial bootable snapshot is created at the end of the initial system installation. You can go back to that state at any time by booting this snapshot. The snapshot can be identified by the description after installation.

There are two methods how you can perform a system rollback.

In case your current snapshot is functional, you can use the following procedure for system rollback.

Procedure 2: Rollback from a running system
  

  1. Choose the snapshot that should be set as default, run: 

    # transactional-update status

    to get a list of available snapshots. Note the number of the snapshot to be set as default.

  2. Set the snapshot as the default by running: 

    # transactional-update rollback snapshot_number

    If you omit the snapshot number, the current snapshot will be set as default.

  3. Reboot your system to boot in to the new default snapshot.

The following procedure is used in case the current snapshot is broken and you are not able to boot into it.

Procedure 3: Rollback to a working snapshot
  

  1. Reboot your system and select Start bootloader from a read-only snapshot

  2. Choose a snapshot to boot. The snapshots are sorted according to the date of creation, with the latest one at the top.

  3. Log in to your system and check whether everything works as expected. Data written to directories excluded from the snapshots will stay untouched.

  4. If the snapshot you booted into is not suitable for rollback, reboot your system and choose another one.

    If the snapshot works as expected, you can perform rollback by running the following command: 

    # transactional-update rollback

    And reboot afterwards.

2.4 Managing automatic transactional updates

  

Automatic updates are controlled by a systemd.timer that runs once per day. This applies all updates, and informs rebootmgrd that the machine should be rebooted. You may adjust the time when the update runs, see systemd.timer(5) documentation.

You can disable automatic transactional updates with this command: 

# systemctl --now disable transactional-update.timer

3 Health checker

  

Health checker is a program delivered with SLE Micro that checks whether services are running properly during booting of your system.

During the boot process, systemd calls Health checker, which in turn calls its plugins. Each plugin checks a particular service or condition. If each check passes, a status file (/var/lib/misc/health-checker.state) is created. The status file marks the current root file system as correct.

If any of the health checker plugins reports an error, the action taken depends on a particular condition, as described below:

The snapshot is booted for the first time.

If the current snapshot is different from the last one that worked properly, an automatic rollback to the last working snapshot is performed. This means that the last change performed to the file system broke the snapshot.

The snapshot has already booted correctly in the past.

There could be just a temporary problem, and the system is rebooted automatically.

The reboot of a previously correctly booted snapshot has failed.

If there was already a problem during boot and automatic reboot has been triggered, but the problem still persists, then the system is kept running to enable to the administrator to fix the problem. The services that are tested by the health checker plugins are stopped if possible.

3.1 Adding custom plugins

  

Health checker supports the addition of your own plugins to check services during the boot process. Each plugin is a bash script that must fulfill the following requirements:

  • Plugins are located within a specific directory—/usr/libexec/health-checker

  • The service that will be checked by the particular plugin must be defined in the Unit section of the /usr/lib/systemd/system/health-checker.service file. For example, the etcd service is defined as follows: 

    [Unit]
...
After=etcd.service
...

  • Each plugin must have functions called run.checks and stop_services defined. The run.checks function checks whether a particular service has started properly. Bear in mind that service that has not been enabled by systemd, should be ignored. The function stop_services is called to stop the particular service in case the service has not been started properly. You can use the plugin template for your reference.

4 SLE Micro administration using Cockpit

  

Cockpit is a web-based graphical interface that enables you to manage your SLE Micro deployments from one place. Cockpit is included in the delivered raw images, or can be installed if you are installing your own instances manually. For the Cockpit installation process refer to Section 10.9.2, “Software”.

In case Cockpit is not installed in your system. You can install it additionally by running: 

# transactional-update pkg install -t pattern  microos-cockpit

After reboot, Cockpit will be available in your system.

Before running Cockpit on you machine, you need to enable the cockpit socket in systemd by running: 

# systemctl enable --now cockpit.socket

In case you have enabled the firewall, you also must open the firewall for Cockpit as follows: 

# firewall-cmd --permanent --zone=public --add-service=cockpit

And then reload the firewall configuration by running: 

# firewall-cmd --reload

Now you can access the Cockpit web interface by opening the following address in your web browser: 

https://IP_ADDRESS_OF_MACHINE:9090

A login screen opens. To login, use the same credentials as you use to login to your machine via console or SSH.

Cockpit login screen
Figure 1: Cockpit login screen
  

After successful login, an overview of your machine opens. Here you can view and administer your system's performance, network interfaces, Podman containers, services, and logs. You can also access your machine using shell in a terminal emulator.

Cockpit dashboard
Figure 2: Cockpit dashboard
  

5 toolbox for SLE Micro debugging

  

SLE Micro uses the transactional-update command to apply changes to the system, but the changes are applied only after reboot. That solution has several benefits, but it also has some disadvantages. If you need to debug your system and install a new tool, the tool will be available only after reboot. Therefore you are not able to debug the currently running system. For this reason a utility called toolbox has been developed.

toolbox is a small script that pulls a container image and runs a privileged container based on that image. In the toolbox container you can install any tool you want with zypper and then use the tool without rebooting your system.

To start the toolbox container, run the following: 

# /usr/bin/toolbox

If the script completes successfully, you will see the toolbox container prompt.

Note
Note: Obtaining the toolbox image

You can also use Podman or Cockpit to pull the toolbox image and start a container based on that image.