7 Concept and Workflow #

KIWI NG installs packages into an appliance from the repositories defined in the image description. This means that at least one repository must be defined. Otherwise, KIWI NG cannot fetch any packages.

A repository is added to the description via the repository element which is a child of the top-level image element:

<image schemaversion="8.0" name="{exc_image_base_name}">
    <!-- snip -->
    <repository type="rpm-md" alias="kiwi" priority="1">
        <source path="obs://Virtualization:Appliances:Builder/openSUSE_Leap_15.5"/>
    </repository>
    <repository type="rpm-md" alias="OS" imageinclude="true">
        <source path="{exc_repo}"/>
    </repository>
</image>

The above example specifies two repositories:

The repository element accepts one source child element that contains the URL of the repository in an correct format along with the following optional attributes:

The package element represents a single package to be installed (or removed), whose name is specified via the mandatory name attribute:

<image schemaversion="8.0" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="bootstrap">
        <package name="udev"/>
    </packages>
</image>

This adds the package udev to the list of packages to be added to the initial filesystem. Note, that the value that you pass via the name attribute is passed directly to the used package manager. Thus, if the package manager supports other means how packages can be specified, you may pass them in this context too. For example, RPM based package managers (like dnf or zypper) can install packages via their Provides:. This can be used to add a package that provides a certain capability (for example, Provides: /usr/bin/my-binary) via:

<image schemaversion="8.0" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="bootstrap">
        <package name="/usr/bin/my-binary"/>
    </packages>
</image>

Whether this works depends on the package manager and on the environment that is being used. In the Open Build Service, certain Provides either are not visible or cannot be properly extracted from the KIWI NG description. Therefore, relying on Provides is not recommended.

Packages can also be included only on specific host architectures via the arch attribute. KIWI NG compares the arch attributes value with the host architecture that builds the image according to the output of uname -m.

<image schemaversion="8.0" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="image">
        <package name="grub2"/>
        <package name="grub2-x86_64-efi" arch="x86_64"/>
        <package name="shim" arch="x86_64"/>
    </packages>
</image>

This results in grub2-x86_64-efi and shim being only installed if the build host is a 64bit x86 machine, but grub2 will be installed independent of the architecture.

In certain situations, it is necessary to include additional packages into the image that are not available in the package manager’s native format. KIWI NG supports the inclusion of regular tar archives via the archive element, whose name attribute specifies the filename of the archive (KIWI NG looks for the archive in the image description folder).

<packages type="image">
    <archive name="custom-program1.tgz"/>
    <archive name="custom-program2.tar"/>
</packages>

KIWI NG extracts the archive into the root directory of the image using GNU tar. This means that only archives supported by it can be included. When multiple archive elements are specified then they are applied in a top to bottom order. If a file is already present in the image, then the file from the archive overwrites it (same as with the image overlay).

KIWI NG supports two different methods for removing packages from the appliance:

Both types of removals take place after config.sh is run in the Section 7.10.1, “The Prepare Step” (see also Section 7.6, “User-Defined Scripts”).

Warning

An uninstall packages request deletes:

• the listed packages

• the packages dependent on the listed ones

• any orphaned dependency of the listed packages

Use this feature with caution as it can cause removal of required tools, leading to failures in later build stages.

Removing packages via type="uninstall" can be used to completely remove a build time tool (for example, a compiler), without having to specify all dependencies of that tool (as opposed to when using type="delete"). Consider the following example, where we want to compile a custom program in config.sh. We ship its source code via an archive element and add the build tools (ninja, meson and clang) to <packages type="image"> and <packages type="uninstall">:

<image schemaversion="8.0" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="image">
        <package name="ca-certificates"/>
        <package name="coreutils"/>
        <package name="ninja"/>
        <package name="clang"/>
        <package name="meson"/>
        <archive name="foo_app_sources.tar.gz"/>
    </packages>
    <!-- These packages will be uninstalled after running config.sh -->
    <packages type="uninstall">
        <package name="ninja"/>
        <package name="meson"/>
        <package name="clang"/>
    </packages>
</image>

The tools meson, clang and ninja are then available during the Section 7.10.1, “The Prepare Step”, and they can be used in config.sh (for further details, see Section 7.6, “User-Defined Scripts”), for example to build foo_app:

pushd /opt/src/foo_app
mkdir build
export CC=clang
meson build
cd build && ninja && ninja install
popd

The <packages type="uninstall"> element ensures that the final appliance no longer contains the tools required to build foo_app, thus making the image smaller.

There are also other use cases for type="uninstall", especially for specialized appliances. For containers, you can remove the package shadow (it is required to setup new user accounts) or any remaining partitioning tools (parted or fdisk). All networking tools can be safely uninstalled in images for embedded devices without a network connection.

KIWI NG supports the inclusion of openSUSE products or of namedCollections (patterns in SUSE based distributions or groups for RedHat based distributions). These can be added via the product and namedCollection child elements, which both take the mandatory name attribute and the optional arch attribute.

product and namedCollection can be used to shorten the list of packages that need to be added to the image description. A named pattern, specified with the namedCollection element is a representation of a predefined list of packages. Specifying a pattern installs all packages listed in the named pattern. Support for patterns is distribution-specific and available in SLES, openSUSE, CentOS, RHEL and Fedora. The optional patternType attribute on the packages element allows you to control the installation of dependent packages. You may assign one of the following values to the patternType attribute:

Packages can be explicitly marked to be ignored for installation inside a packages collection. This can be used to exclude certain packages from being installed when using patterns with patternType="plusRecommended" as shown in the following example:

<image schemaversion="8.0" name="{exc_image_base_name}">
    <packages type="image" patternType="plusRecommended">
        <namedCollection name="network-server"/>
        <package name="grub2"/>
        <package name="kernel"/>
        <ignore name="ejabberd"/>
        <ignore name="puppet-server"/>
    </packages>
</image>

Packages can be marked as ignored during the installation by adding a ignore child element with the mandatory name attribute set to the name of the package. Optionally, you can specify the architecture via arch, similarly to The package element.

Warning

Adding ignore elements as children of a <packages type="delete"> or a <packages type="uninstall"> element has no effect! The packages will still be deleted.

Creating a custom script may require some experimenting and testing. To help developers with this task, KIWI NG calls scripts associated with a screen session. The connection to screen is only done if KIWI NG is called with the --debug option.

In this mode, a script can be started using the following template:

# The magic bits are still not set

echo "break"
/bin/bash

Calling the script executes a screen session executes, which gives you access to the break in shell. You can then implement the desired script code in this environment.. Once the shell is closed the KIWI NG process continues.

In addition to providing a fully featured terminal throughout the execution of the script code, you also have have control of the session during the process of the image creation. Listing the active sessions for script execution can be done as follows:

$ sudo screen -list

There is a screen on:
     19699.pts-4.asterix     (Attached)
1 Socket in /run/screens/S-root.

Note

As shown above the screen session for executing the script code provides extended control, which can be considered a security risk. Because of that, KIWI NG only runs scripts through screen when explicitly enabled via the --debug switch. In production, all scripts must run natively and must not require a terminal to operate correctly.

7.6.1.1 Script Template for config.sh / images.sh #

 

KIWI NG provides a collection of methods and variables that offer custom actions. For details, see Functions and Variables Provided by KIWI NG. The following template shows how to import this information into a script:

#======================================
# Include functions & variables
#--------------------------------------
test -f /.kconfig && . /.kconfig
test -f /.profile && . /.profile

...

Warning

Modifications of the unpacked root tree

Keep in mind that there is only one unpacked root tree the script operates in. This means that all changes are permanent and are not automatically restored.

7.6.1.2 Functions and Variables Provided by KIWI NG #

 

KIWI NG creates the .kconfig and .profile files to be sourced by the shell scripts config.sh and images.sh. .kconfig contains several helper functions that can be used to simplify image configuration, while .profile contains environment variables populated from the settings provided in the image description.

7.6.1.2.1 Functions #

 

The .kconfig file provides a common set of functions. Functions specific to SUSE Linux Enterprise and openSUSE start with suse, functions applicable to all Linux distributions start with base.

The following list describes all functions provided by .kconfig:

baseSetRunlevel {value}

Set the default run level.

baseStripAndKeep {list of info-files to keep}

Helper function for the baseStrip* functions that reads a list of files to check from stdin for removing params: files which should be kept.

baseStripLocales {list of locales}

Removes all locales, except for the ones given as the parameter.

baseStripTranslations {list of translations}

Removes all translations, except those given as the parameter.

baseStripUnusedLibs

Removes libraries that are not directly linked against applications in the bin directories.

baseUpdateSysConfig {filename} {variable} {value}

Updates the contents of a sysconfig variable.

baseSystemdServiceInstalled {service}

Prints the path of the first found systemd unit or mount with name passed as the first parameter.

baseSysVServiceInstalled {service}

Prints the name ${service} if a SysV init service with the same name is found; otherwise it prints nothing.

baseSystemdCall {service_name} {args}

Calls systemctl ${args} ${service_name} if a systemd unit, a systemd mount, or a SysV init service with the ${service_name} exists.

baseInsertService {servicename}

Activates the specified service via systemctl.

baseRemoveService {servicename}

Deactivates the specified service via systemctl.

baseService {servicename} {on|off}

Activates or deactivates a service via systemctl. The function requires the service name and the value on or off as parameters.

The following example enables the sshd service on boot:

baseService sshd on

suseInsertService {servicename}

Calls baseInsertService. It exists only for compatibility reasons.

suseRemoveService {servicename}

Calls baseRemoveService. It exists only for compatibility reasons.

suseService {servicename} {on|off}

Calls baseService. It exists only for compatibility reasons.

suseSetupProduct

Creates the /etc/products.d/baseproduct link pointing to the product referenced either by /etc/SuSE-brand or /etc/os-release or the latest prod file available in /etc/products.d

baseVagrantSetup

Configures the image to work as a vagrant box by performing the following changes:

• add the vagrant user to /etc/sudoers or /etc/sudoers.d/vagrant

• insert the insecure vagrant ssh key, apply recommended ssh settings and start the ssh daemon

• create the default shared folder /vagrant

Debug {message}

Helper function to print the supplied message if the variable DEBUG is set to 1 (disabled by default).

Echo {echo commandline}

Helper function to print a message to the controlling terminal.

Rm {list of files}

Helper function to delete files and log the deletion.

7.6.1.2.2 Profile Environment Variables #

 

The .profile environment file is created by KIWI NG and contains a specific set of variables listed below.

$kiwi_compressed

A value of the compressed attribute set in the type element in config.xml.

$kiwi_delete

A list of all packages which are children of the packages element with type="delete" in config.xml.

$kiwi_drivers

A comma-separated list of driver entries as listed in the drivers section of config.xml.

$kiwi_iname

The name of the image as listed in config.xml.

$kiwi_iversion

The image version as a string.

$kiwi_keytable

The contents of the keytable setup as specified in config.xml.

$kiwi_language

The contents of the locale setup as specified in config.xml.

$kiwi_profiles

A comma-separated list of profiles used to build this image.

$kiwi_timezone

The contents of the timezone setup as specified in config.xml.

$kiwi_type

The image type as extracted from the type element in config.xml.

Note

.profile.extra

If there is the file /.profile.extra available in the initrd, KIWI NG imports the file importing /.profile.

The dracut initrd system uses systemd to implement a predefined workflow of services documented in the bootup man page:

http://man7.org/linux/man-pages/man7/dracut.bootup.7.html

To hook in a custom boot script to this workflow, it is necessary to provide a dracut module that dracut picks when KIWI NG calls it. The module files can be provided either as a package or as part of the overlay directory in the image description.

The following example shows how to include a custom hook script before the system rootfs is mounted.

Note

Declaring Extra Tools for Hook Scripts

The install() function called by dracut can define extra tools required by the specified hook script. The inst_multiple command and its parameters instruct dracut to include these extra tools and items into the initrd.

The specified tools and items can be files. Normally, they are executables and libraries required by the hook script.

• Each file must be included in the KIWI NG description either in a package, archive, or in the root tree of the image description directory.

• The parameters of the inst_multiple command are space separated.

• Each parameter can be a single executable name if it exists in /bin, /sbin, /usr/bin, or /usr/sbin` directories.

• Otherwise, a full path to the file is required. This normally applies for libraries and other special files.

When KIWI NG calls dracut, the 90my-module is installed into the generated initrd. At boot time, systemd calls the scripts as part of the dracut-pre-mount.service.

The dracut system offers many other possibilities to customize the initrd than shown in the example above. For more information, visit the dracut project page.

A dracut generated initrd in a KIWI NG image build process includes one or more of the KIWI NG provided dracut modules. The following list documents the available kernel boot parameters for these modules:

Note

Non interactive mode activated by rd.kiwi.oem.installdevice

When setting rd.kiwi.oem.installdevice explicitly through the kernel command line, KIWI NG uses the device without prompting for confirmation.

rd.debug rd.kiwi.debug

less /run/initramfs/log/boot.kiwi

As the first step, KIWI NG creates an unpackaged image tree, also called “root tree”. This directory will be the installation target for software packages to be installed during the image creation process.

For the package installation, KIWI NG relies on the package manager specified in the packagemanager element in config.xml. KIWI NG supports the following package managers: dnf, zypper (default) and apt.

The prepare step consists of the following substeps:

KIWI NG creates the final image during the create step: it converts the unpacked root tree into one or multiple output files appropriate for the respective build type.

It is possible to create multiple images from the same unpacked root tree, for example, a self installing OEM image and a virtual machine image from the same image description. The only prerequisite is that both image types are specified in config.xml.

During the create step the following operations are performed by KIWI NG: