Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]

7 Concept and Workflow

Note
Note

Abstract

The following sections describe the concept and general workflow of building appliances with KIWI NG 9.23.55.

7.1 Host Requirements To Build Images

When building OS images, several tools and sub-systems are used and required on the host KIWI NG is called at. For example, to build a virtual disk image, several tools needs to be available on the host that builds the image. This includes tools for partition table setup or tools to create filesystems.

The number of required components depends on the selected image type and the features used with the image. We cannot expect the users of KIWI NG to know about each and every component that is needed to build the image. Therefore a concept to help with the host requirements exists and is named kiwi-systemdeps

The kiwi-systemdeps concept consists out of a collection of sub-packages provided with the python-kiwi main package. Each individual package requires a number of tools and subsystem packages which belongs to the package category. There are the following systemdeps packages:

kiwi-systemdeps-core:
  • Supports building the simple root archive tbz image type.

  • Installs the package managers which are supported by the target distribution as well as the tar archiving tool.

kiwi-systemdeps-containers:
  • Supports building docker and appx image types.

  • Installs the distribution specific tool chain to build OCI compliant and WSL container images.

kiwi-systemdeps-iso-media:
  • Supports building iso image types and oem install media.

  • Installs all tools required to build ISO filesystems.

  • Depends on the -core, -filesystems and -bootloaders kiwi-systemdeps packages.

kiwi-systemdeps-bootloaders:
  • Supports building bootable oem and iso image types.

  • Installs all bootloader tools depending on the host architecture to allow setup and install of the bootloader. The pulled in components are required for any image that is able to boot through some BIOS or firmware.

  • Depends on the -core kiwi-systemdeps packages.

Note
Note

The iso type is an exception which might not require the -bootloaders systemdeps. In case of the firmware attribute to be set to bios, KIWI NG builds bootable ISO images still based on isolinux which is provided with the -iso-media systemdeps. However, by default, any KIWI NG created ISO image is BIOS and EFI capable and based on the grub bootloader which causes a requirement to the -bootloaders systemdeps.

kiwi-systemdeps-filesystems:
  • Supports building fs-type, oem, pxe, kis and live iso image types.

  • Installs all tools to create filesystems supported with KIWI NG. The pulled in components are needed for any image type that needs to create a filesystem. This excludes the archive based image types like docker, appx or tbz. The package also installs tools one level below the actual filesystem creation toolkit. These are components to manage loop devices as well as partition table setup and subsystem support like LVM and LUKS.

  • Depends on the -core kiwi-systemdeps packages.

kiwi-systemdeps-disk-images:
  • Supports building the oem image type.

  • Installs all tools to create virtual disks. In KIWI NG, virtual disks are created using the QEMU toolchain.

  • Depends on the -filesystems and -bootloaders kiwi-systemdeps packages.

kiwi-systemdeps-image-validation:
  • Installs the jing tool to validate the image description. This is useful for detailed error reports from KIWI NG in case of an image description validation error. In addition, the anymarkup Python module is installed if the the option to install recommended packages is set. With anymarkup available, KIWI NG can also handle image descriptions in another format than the XML markup, like YAML.

Depending on the image type the kiwi-systemdeps packages can help to setup the host system quickly for the task to build an image. In case the host should support everything there is also the main kiwi-systemdeps package which has a dependency on all other existing systemdeps packages.

Note
Note

Pulling in all kiwi-systemdeps packages can result in quite some packages to become installed on the host. This is because the required packages itself comes with a number of dependencies like java for jing as one example.

7.2 Setting up Repositories

A crucial part of each appliance is the repository selection. KIWI NG allows the end user to completely customize the selection of repositories and packages via the repository element.

7.2.1 Adding repositories

KIWI NG installs packages into your appliance from the repositories defined in the image description. Therefore at least one repository must be defined, as KIWI NG will otherwise not be able to fetch any packages.

A repository is added to the description via the repository element, which is a child of the top-level image element:

<image schemaversion="7.4" name="{exc_image_base_name}">
    <!-- snip -->
    <repository type="rpm-md" alias="kiwi" priority="1">
        <source path="obs://Virtualization:Appliances:Builder/openSUSE_Leap_15.3"/>
    </repository>
    <repository type="rpm-md" alias="OS" imageinclude="true">
        <source path="{exc_repo}"/>
    </repository>
</image>

In the above snippet we defined two repositories:

  1. The repository belonging to the KIWI NG project: obs://Virtualization:Appliances:Builder/openSUSE_Leap_15.3 at the Open Build Service (OBS)

  2. The RPM repository belonging to the OS project: {exc_repo}, at the Open Build Service (OBS). The translated http URL will also be included in the final appliance.

The repository element accepts one source child element, which contains the URL to the repository in an appropriate format and the following optional attributes:

  • imageinclude: Specify whether this repository should be added to the resulting image, defaults to false.

  • imageonly: A repository with imageonly="true" will not be available during image build, but only in the resulting appliance. Defaults to false.

  • priority: An integer priority for all packages in this repository. If the same package is available in more than one repository, then the one with the highest priority is used.

  • alias: Name to be used for this repository, it will appear as the repository’s name in the image, which is visible via zypper repos or dnf repolist. KIWI NG will construct an alias from the path in the source child element (replacing each / with a _), if no value is given.

  • repository_gpgcheck: Specify whether or not this specific repository is configured to to run repository signature validation. If not set, the package manager’s default is used.

  • package_gpgcheck: Boolean value that specifies whether each package’s GPG signature will be verified. If omitted, the package manager’s default will be used

  • components: Distribution components used for deb repositories, defaults to main.

  • distribution: Distribution name information, used for deb repositories.

  • profiles: List of profiles to which this repository applies.

  • customize: Script to run custom modifications to the repo file(s). repo files allows for several customization options which not all of them are supported to be set by kiwi through the current repository schema. As the options used do not follow any standard and are not compatible between package managers and distributions, the only generic way to handle this is through a script hook which is invoked with the repo file as parameter for each file created by KIWI NG.

    An example for a script call to add the module_hotfixes option for a dnf compatible repository configuration could look like this

    repo_file=$1
    echo 'module_hotfixes = 1' >> ${repo_file}
    Note
    Note

    If the script is provided as relative path it will be searched in the image description directory

7.2.1.1 Supported repository paths

The actual location of a repository is specified in the source child element of repository via its only attribute path. KIWI NG supports the following paths types:

  • http://URL or https://URL or ftp://URL: a URL to the repository available via HTTP(s) or FTP.

  • obs://$PROJECT/$REPOSITORY: evaluates to the repository $REPOSITORY of the project $PROJECT available on the Open Build Service (OBS). By default KIWI NG will look for projects on build.opensuse.org, but this can be overridden using the runtime configuration file (see Section 7.7, “The Runtime Configuration File”). Note that it is not possible to add repositories using the obs:// path from different OBS instances (use direct URLs to the .repo file instead in this case).

  • obsrepositories:/: special path only available for builds using the Open Build Service. The repositories configured for the OBS project in which the KIWI NG image resides will be available inside the appliance. This allows you to configure the repositories of your image from OBS itself and not having to modify the image description.

  • dir:///path/to/directory or file:///path/to/file: an absolute path to a local directory or file available on the host building the appliance.

  • iso:///path/to/image.iso: the specified ISO image will be mounted during the build of the KIWI NG image and a repository will be created pointing to the mounted ISO.

7.3 Adding and Removing Packages

On top of the Section 7.2, “Setting up Repositories” setup the package setup is required. KIWI NG allows the end user to completely customize the selection of packages via the packages element.

<image schemaversion="7.4" name="{exc_image_base_name}">
    <packages type="bootstrap">
        <package name="udev"/>
        <package name="filesystem"/>
        <package name="openSUSE-release"/>
        <!-- additional packages installed before the chroot is created -->
    </packages>
    <packages type="image">
        <package name="patterns-openSUSE-base"/>
        <!-- additional packages to be installed into the chroot -->
    </packages>
</image>

The packages element provides a collection of different child elements that instruct KIWI NG when and how to perform package installation or removal. Each packages element acts as a group, whose behavior can be configured via the following attributes:

  • : either bootstrap, image, delete, uninstall or one of the following build types: docker, iso, oem, kis, oci.

    Packages for type="bootstrap" are pre-installed to populate the images’ root file system before chrooting into it.

    Packages in type="image" are installed immediately after the initial chroot into the new root file system.

    Packages in type="delete" and type="uninstall" are removed from the image, for details see Uninstall System Packages.

    And packages which belong to a build type are only installed when that specific build type is currently processed by KIWI NG.

  • profiles: a list of profiles to which this package selection applies (see Section 7.4, “Image Profiles”).

  • patternType: selection type for patterns, supported values are: onlyRequired, plusRecommended, see: The product and namedCollection element.

The following sections describes the different child elements of a packages group.

7.3.1 The package element

The package element represents a single package to be installed (or removed), whose name is specified via the mandatory name attribute:

<image schemaversion="7.4" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="bootstrap">
        <package name="udev"/>
    </packages>
</image>

which adds the package udev to the list of packages to be added to the initial filesystem. Note, that the value that you pass via the name attribute is passed directly to the used package manager. Thus, if the package manager supports other means how packages can be specified, you may pass them in this context too. For example, RPM based package managers (like dnf or zypper) can install packages via their Provides:. This can be used to add a package that provides a certain capability (e.g. Provides: /usr/bin/my-binary) via:

<image schemaversion="7.4" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="bootstrap">
        <package name="/usr/bin/my-binary"/>
    </packages>
</image>

Whether this works depends on the package manager and on the environment that is being used. In the Open Build Service, certain Provides either are not visible or cannot be properly extracted from the KIWI NG description. Therefore, relying on Provides is not recommended.

Packages can also be included only on specific host architectures via the arch attribute. KIWI NG compares the arch attributes value with the host architecture that builds the image according to the output of uname -m.

<image schemaversion="7.4" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="image">
        <package name="grub2"/>
        <package name="grub2-x86_64-efi" arch="x86_64"/>
        <package name="shim" arch="x86_64"/>
    </packages>
</image>

which results in grub2-x86_64-efi and shim being only installed if the build host is a 64bit x86 machine, but grub2 will be installed independent of the architecture.

7.3.2 The archive element

It is sometimes necessary to include additional packages into the image which are not available in the package manager’s native format. KIWI NG supports the inclusion of ordinary tar archives via the archive element, whose name attribute specifies the filename of the archive (KIWI NG looks for the archive in the image description folder).

<packages type="image">
    <archive name="custom-program1.tgz"/>
    <archive name="custom-program2.tar"/>
</packages>

KIWI NG will extract the archive into the root directory of the image using GNU tar, thus only archives supported by it can be included. When multiple archive elements are specified then they will be applied in a top to bottom order. If a file is already present in the image, then the file from the archive will overwrite it (same as with the image overlay).

7.3.3 Uninstall System Packages

KIWI NG supports two different methods how packages can be removed from the appliance:

  1. Packages present as a child element of <packages type="uninstall"> will be gracefully uninstalled by the package manager alongside with dependent packages and orphaned dependencies.

  2. Packages present as a child element of <packages type="delete"> will be removed by RPM/DPKG without any dependency check, thus potentially breaking dependencies and compromising the underlying package database.

Both types of removals take place after config.sh is run in the Section 7.10.1, “The Prepare Step” (see also Section 7.6, “User Defined Scripts”).

Warning
Warning

An uninstall packages request deletes:

  • the listed packages,

  • the packages dependent on the listed ones, and

  • any orphaned dependency of the listed packages.

Use this feature with caution as it can easily cause the removal of sensitive tools leading to failures in later build stages.

Removing packages via type="uninstall" can be used to completely remove a build time tool (e.g. a compiler) without having to specify a all dependencies of that tool (as one would have when using type="delete"). Consider the following example where we wish to compile a custom program in config.sh. We ship its source code via an archive element and add the build tools (ninja, meson and clang) to <packages type="image"> and <packages type="uninstall">:

<image schemaversion="7.4" name="{exc_image_base_name}">
    <!-- snip -->
    <packages type="image">
        <package name="ca-certificates"/>
        <package name="coreutils"/>
        <package name="ninja"/>
        <package name="clang"/>
        <package name="meson"/>
        <archive name="foo_app_sources.tar.gz"/>
    </packages>
    <!-- These packages will be uninstalled after running config.sh -->
    <packages type="uninstall">
        <package name="ninja"/>
        <package name="meson"/>
        <package name="clang"/>
    </packages>
</image>

The tools meson, clang and ninja are then available during the Section 7.10.1, “The Prepare Step” and can thus be used in config.sh (for further details, see Section 7.6, “User Defined Scripts”), for example to build foo_app:

pushd /opt/src/foo_app
mkdir build
export CC=clang
meson build
cd build && ninja && ninja install
popd

The <packages type="uninstall"> element will make sure that the final appliance will no longer contain our tools required to build foo_app, thus making our image smaller.

There are also other use cases for type="uninstall", especially for specialized appliances. For containers one can often remove the package shadow (it is required to setup new user accounts) or any left over partitioning tools (parted or fdisk). All networking tools can be safely uninstalled in images for embedded devices without a network connection.

7.3.4 The product and namedCollection element

KIWI NG supports the inclusion of openSUSE products or of namedCollections (patterns in SUSE based distributions or groups for RedHat based distributions). These can be added via the product and namedCollection child elements, which both take the mandatory name attribute and the optional arch attribute.

product and namedCollection can be utilized to shorten the list of packages that need to be added to the image description tremendously. A named pattern, specified with the namedCollection element is a representation of a predefined list of packages. Specifying a pattern will install all packages listed in the named pattern. Support for patterns is distribution specific and available in SLES, openSUSE, CentOS, RHEL and Fedora. The optional patternType attribute on the packages element allows you to control the installation of dependent packages. You may assign one of the following values to the patternType attribute:

  • onlyRequired: Incorporates only patterns and packages that the specified patterns and packages require. This is a “hard dependency” only resolution.

  • plusRecommended: Incorporates patterns and packages that are required and recommended by the specified patterns and packages.

7.3.5 The ignore element

Packages can be explicitly marked to be ignored for installation inside a packages collection. This useful to exclude certain packages from being installed when using patterns with patternType="plusRecommended" as shown in the following example:

<image schemaversion="7.4" name="{exc_image_base_name}">
    <packages type="image" patternType="plusRecommended">
        <namedCollection name="network-server"/>
        <package name="grub2"/>
        <package name="kernel"/>
        <ignore name="ejabberd"/>
        <ignore name="puppet-server"/>
    </packages>
</image>

Packages can be marked as ignored during the installation by adding a ignore child element with the mandatory name attribute set to the package’s name. Optionally one can also specify the architecture via the arch similarly to The package element.

Warning
Warning

Adding ignore elements as children of a <packages type="delete"> or a <packages type="uninstall"> element has no effect! The packages will still get deleted.

7.4 Image Profiles

A profile is a namespace for additional settings that can be applied by KIWI NG on top of the default settings (or other profiles), thereby allowing to build multiple appliances with the same build type but with different configurations.

The use of profiles is advisable to distinguish image builds of the same type but with different settings. In the following example, two virtual machine images of the oem type are configured: one for QEMU (using the qcow2 format) and one for VMWare (using the vmdk format).

<image schemaversion="7.4" name="{exc_image_base_name}">
    <profiles>
        <profile name="QEMU" description="virtual machine for QEMU"/>
        <profile name="VMWare" description="virtual machine for VMWare"/>
    </profiles>
    <preferences>
        <version>15.0</version>
        <packagemanager>zypper</packagemanager>
    </preferences>
    <preferences profiles="QEMU">
        <type image="oem" format="qcow2" filesystem="ext4">
    </preferences>
    <preferences profiles="VMWare">
        <type image="oem" format="vmdk" filesystem="ext4">
    </preferences>
</image>

Each profile is declared via the element , which itself must be a child of profiles and must contain the name and description attributes. The description is only present for documentation purposes, name on the other hand is used to instruct KIWI NG which profile to build via the command line. Additionally, one can provide the boolean attribute import, which defines whether this profile should be used by default when KIWI NG is invoked via the command line.

A profile inherits the default settings which do not belong to any profile. It applies only to elements that contain the profile in their profiles attribute. The attribute profiles expects a comma separated list of profiles for which the settings of this element apply.

Profiles can furthermore inherit settings from another profile via the requires sub-element:

<profiles>
    <profile name="VM" description="virtual machine"/>
    <profile name="QEMU" description="virtual machine for QEMU">
        <requires profile="VM"/>
    </profile>
</profiles>

The profile QEMU would inherit the settings from VM in the above example.

For further details on the usage of profiles see Section 11.18, “Building Images with Profiles”

7.5 Adding Users

User accounts can be added or modified via the users element, which supports a list of multiple user child elements:

<image schemaversion="7.4" name="{exc_image_base_name}">
    <users>
        <user
            password="this_is_soo_insecure"
            home="/home/me" name="me"
            groups="users" pwdformat="plain"
        />
        <user
            password="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0"
            home="/root" name="root" groups="root"
        />
    </users>
</image>

Each user element represents a specific user that is added or modified. The following attributes are mandatory:

  • name: the UNIX username

  • password: The password for this user account. It can be provided either in cleartext form (pwdformat="plain") or in ’ed form (pwdformat="encrypted"). Plain passwords are discouraged, as everyone with access to the image description would know the password. It is recommended to generate a hash of your password using openssl as follows:

    $ openssl passwd -1 -salt 'xyz' YOUR_PASSWORD

Additionally, the following optional attributes can be specified:

  • home: the path to the user’s home directory

  • groups: A comma separated list of UNIX groups. The first element of the list is used as the user’s primary group. The remaining elements are appended to the user’s supplementary groups. When no groups are assigned then the system’s default primary group will be used.

  • : The numeric user id of this account.

  • pwdformat: The format in which password is provided, either plain or encrypted (the latter is the default).

7.6 User Defined Scripts

Note
Note

Abstract

This chapter describes the purpose of the user defined scripts config.sh, image.sh and disk.sh, which can be used to further customize an image in ways that are not possible via the image description alone.

KIWI NG supports the following optional scripts that it runs in a root environment (chroot) containing your new appliance:

post_bootstrap.sh

runs at the end of the bootstrap phase as part of the Section 7.10.1, “The Prepare Step”. The script can be used to configure the package manager with additional settings that should apply in the following chroot based installation step which completes the installation. The script is not dedicated to this use and can also be used for other tasks.

config.sh

runs at the end of the Section 7.10.1, “The Prepare Step” and after users have been set and the overlay tree directory has been applied. It is usually used to apply a permanent and final change of data in the root tree, such as modifying a package provided config file.

images.sh

is executed at the beginning of the Section 7.10.2, “The Create Step”. It runs in the same image root tree that has been created by the prepare step but is invoked any time an image should be created from that root tree. It is usually used to apply image type specific changes to the root tree such as a modification to a config file that should be done when building a live iso but not when building a virtual disk image.

disk.sh

is executed for the disk image type oem only and runs after the synchronisation of the root tree into the disk image loop file. At call time of the script the device name of the currently mapped root device is passed as a parameter. The chroot environment for this script call is the virtual disk itself and not the root tree as with config.sh and images.sh. The script disk.sh is usually used to apply changes at parts of the system that are not an element of the file based root tree such as the partition table, the bootloader or filesystem attributes.

KIWI NG executes scripts via the operating system if their executable bit is set (in that case a shebang is mandatory) otherwise they will be invoked via the BASH. If a script exits with a non-zero exit code then KIWI NG will report the failure and abort the image creation.

7.6.1 Script Template for config.sh / images.sh

KIWI NG provides a collection of methods and variables that supports users with custom operations. For details see Functions and Variables Provided by KIWI NG. The following template shows how to import this information in your script:

#======================================
# Include functions & variables
#--------------------------------------
test -f /.kconfig && . /.kconfig
test -f /.profile && . /.profile

...
Warning
Warning

Modifications of the unpacked root tree

Keep in mind that there is only one unpacked root tree the script operates in. This means that all changes are permanent and will not be automatically restored!

7.6.2 Functions and Variables Provided by KIWI NG

KIWI NG creates the .kconfig and .profile files to be sourced by the shell scripts config.sh and images.sh. .kconfig contains various helper functions which can be used to simplify the image configuration and .profile contains environment variables which get populated from the settings provided in the image description.

7.6.2.1 Functions

The .kconfig file provides a common set of functions. Functions specific to SUSE Linux Enterprise and openSUSE begin with the name suse, functions applicable to all Linux distributions start with the name base.

The following list describes all functions provided by .kconfig:

baseSetRunlevel {value}

Set the default run level.

baseStripAndKeep {list of info-files to keep}

Helper function for the baseStrip* functions, reads the list of files to check from stdin for removing params: files which should be kept

baseStripLocales {list of locales}

Remove all locales, except for the ones given as the parameter.

baseStripTranslations {list of translations}

Remove all translations, except for the ones given as the parameter.

baseStripUnusedLibs

Remove libraries which are not directly linked against applications in the bin directories.

baseUpdateSysConfig {filename} {variable} {value}

Update the contents of a sysconfig variable

baseSystemdServiceInstalled {service}

Prints the path of the first found systemd unit or mount with name passed as the first parameter.

baseSysVServiceInstalled {service}

Prints the name ${service} if a SysV init service with that name is found, otherwise it prints nothing.

baseSystemdCall {service_name} {args}

Calls systemctl ${args} ${service_name} if a systemd unit, a systemd mount or a SysV init service with the ${service_name} exist.

baseInsertService {servicename}

Activate the given service via systemctl.

baseRemoveService {servicename}

Deactivate the given service via systemctl.

baseService {servicename} {on|off}

Activate or deactivate a service via systemctl. The function requires the service name and the value on or off as parameters.

Example to enable the sshd service on boot:

baseService sshd on
suseInsertService {servicename}

Calls baseInsertService and exists only for compatibility reasons.

suseRemoveService {servicename}

Calls baseRemoveService and exists only for compatibility reasons.

suseService {servicename} {on|off}

Calls baseService and exists only for compatibility reasons.

suseSetupProduct

Creates the /etc/products.d/baseproduct link pointing to the product referenced by either /etc/SuSE-brand or /etc/os-release or the latest prod file available in /etc/products.d

baseVagrantSetup

Configures the image to work as a vagrant box by performing the following changes:

  • add the vagrant user to /etc/sudoers or /etc/sudoers.d/vagrant

  • insert the insecure vagrant ssh key, apply recommended ssh settings and start the ssh daemon

  • create the default shared folder /vagrant

Debug {message}

Helper function to print the supplied message if the variable DEBUG is set to 1 (it is off by default).

Echo {echo commandline}

Helper function to print a message to the controlling terminal.

Rm {list of files}

Helper function to delete files and log the deletion.

7.6.2.2 Profile Environment Variables

The .profile environment file is created by KIWI NG and contains a specific set of variables which are listed below.

$kiwi_compressed

The value of the compressed attribute set in the element in config.xml.

$kiwi_delete

A list of all packages which are children of the packages element with type="delete" in config.xml.

$kiwi_drivers

A comma separated list of the driver entries as listed in the drivers section of the config.xml.

$kiwi_iname

The name of the image as listed in config.xml.

$kiwi_iversion

The image version as a string.

$kiwi_keytable

The contents of the keytable setup as done in config.xml.

$kiwi_language

The contents of the locale setup as done in config.xml.

$kiwi_profiles

A comma separated list of profiles used to build this image.

$kiwi_timezone

The contents of the timezone setup as done in config.xml.

$kiwi_type

The image type as extracted from the element in config.xml.

7.6.3 Configuration Tips

  • Locale configuration:

KIWI in order to set the locale relies on systemd-firstboot, which in turn writes the locale configuration file /etc/locale.conf. The values for the locale settings are taken from the description XML file in the <locale> element under <preferences>.

KIWI assumes systemd adoption to handle these locale settings, in case the build distribution does not honor /etc/locale.conf this is likely to not produce any effect on the locale settings. As an example, in SLE12 distribution the locale configuration is already possible by using the systemd toolchain, however this approach overlaps with SUSE specific managers such as YaST. In that case using systemd-firstboot is only effective if locales in /etc/sysconfig/language are not set or if the file does not exist at all. In SLE12 /etc/sysconfig/language has precendence over /etc/locale.conf for compatibility reasons and management tools could still relay on files for locale settings.

In any case the configuration is still possible in KIWI by using any distribution specific way to configure the locale setting inside the config.sh script or by adding any additional configuration file as part of the overlay root-tree.

  • Stateless systemd UUIDs:

Machine ID files are created and set (/etc/machine-id, /var/lib/dbus/machine-id) during the image package installation when systemd and/or dbus are installed. Those UUIDs are intended to be unique and set only once in each deployment. KIWI NG follows the systemd recommendations and wipes any /etc/machine-id content, leaving it as an empty file. Note, this only applies to images based on a dracut initrd, it does not apply for container images.

In case this setting is also required for a non dracut based image, the same result can achieved by removing /etc/machine-id in config.sh.

Note
Note

Avoid interactive boot

It is important to remark that the file /etc/machine-id is set to an empty file instead of deleting it. systemd may trigger systemd-firstboot service if this file is not present, which leads to an interactive firstboot where the user is asked to provide some data.

Note
Note

Avoid inconsistent /var/lib/dbus/machine-id

Note that /etc/machine-id and /var/lib/dbus/machine-idmust contain the same unique ID. On modern systems /var/lib/dbus/machine-id is already a symlink to /etc/machine-id. However on older systems those might be two different files. This is the case for SLE-12 based images. If you are targeting these older operating systems, it is recommended to add the symlink creation into config.sh:

#======================================
# Make machine-id consistent with dbus
#--------------------------------------
if [ -e /var/lib/dbus/machine-id ]; then
    rm /var/lib/dbus/machine-id
fi
ln -s /etc/machine-id /var/lib/dbus/machine-id

7.7 The Runtime Configuration File

KIWI NG supports an additional configuration file for runtime specific settings that do not belong into the image description but which are persistent and would be unsuitable for command line parameters.

The runtime configuration file must adhere to the YAML syntax and can be provided via the global --config option at call time of KIWI NG. If no config file is provided at the commandline, KIWI NG searches for the runtime configuration file in the following locations:

  1. ~/.config/kiwi/config.yml

  2. /etc/kiwi.yml

A default runtime config file in /etc/kiwi.yml is provided with the python3-kiwi package. The file contains all settings as comments including a short description of each setting.

7.8 Customizing the Boot Process

Most Linux systems use a special boot image to control the system boot process after the system firmware, BIOS or UEFI, hands control of the hardware to the operating system. This boot image is called the initrd. The Linux kernel loads the initrd, a compressed cpio initial RAM disk, into the RAM and executes init or, if present, linuxrc.

Depending on the image type, KIWI NG creates the boot image automatically during the create step. It uses a tool called dracut to create this initrd. Dracut generated initrd archives can be extended by custom modules to add functionality which is not natively provided by dracut itself. In the scope of KIWI NG the following dracut modules are used:

kiwi-dump

Serves as an image installer. It provides the required implementation to install a KIWI NG image on a selectable target. This module is required if one of the attributes installiso, installstick or installpxe is set to true in the image type definition

kiwi-dump-reboot

Serves to boot the system into the installed image after installation is completed.

kiwi-live

Boots up a KIWI NG live image. This module is required if the iso image type is selected

kiwi-overlay

Allows to boot disk images configured with the attribute overlayroot set to true. Such a disk has its root partition compressed and readonly and boots up using overlayfs for the root filesystem using an extra partition on the same disk for persistent data.

kiwi-repart

Resizes an OEM disk image after installation onto the target disk to meet the size constraints configured in the oemconfig section of the image description. The module takes over the tasks to repartition the disk, resizing of RAID, LVM, LUKS and other layers and resizing of the system filesystems.

kiwi-lib

Provides functions of general use and serves as a library usable by other dracut modules. As the name implies, its main purpose is to function as library for the above mentioned kiwi dracut modules.

Note
Note

Using Custom Boot Image Support

Apart from the standard dracut based creation of the boot image, KIWI NG supports the use of custom boot images for the image types oem and pxe. The use of a custom boot image is activated by setting the following attribute in the image description:

<type ... initrd_system="kiwi"/>

Along with this setting it is now mandatory to provide a reference to a boot image description in the boot attribute like in the following example:

<type ... boot="netboot/suse-tumbleweed"/>

Such boot descriptions for the OEM and PXE types are currently still provided by the KIWI NG packages but will be moved into its own repository and package soon.

The custom boot image descriptions allows a user to completely customize what and how the initrd behaves by its own implementation. This concept is mostly used in PXE environments which are usually highly customized and requires a specific boot and deployment workflow.

7.8.1 Boot Image Hook-Scripts

The dracut initrd system uses systemd to implement a predefined workflow of services which are documented in the bootup man page at:

http://man7.org/linux/man-pages/man7/dracut.bootup.7.html

To hook in a custom boot script into this workflow it’s required to provide a dracut module which is picked up by dracut at the time KIWI NG calls it. The module files can be either provided as a package or as part of the overlay directory in your image description

The following example demonstrates how to include a custom hook script right before the system rootfs gets mounted.

  1. Create a subdirectory for the dracut module:

    $ mkdir -p root/usr/lib/dracut/modules.d/90my-module
  2. Register the dracut module in a configuration file:

    $ vi root/etc/dracut.conf.d/90-my-module.conf
    
    add_dracutmodules+=" my-module "
  3. Create the hook script:

    $ touch root/usr/lib/dracut/modules.d/90my-module/my-script.sh
  4. Create a module setup file in root/usr/lib/dracut/modules.d/90my-module/module-setup.sh with the following content:

    #!/bin/bash
    
    # called by dracut
    check() {
        # check module integrity
    }
    
    # called by dracut
    depends() {
        # return list of modules depending on this one
    }
    
    # called by dracut
    installkernel() {
        # load required kernel modules when needed
        instmods _kernel_module_list_
    }
    
    # called by dracut
    install() {
        declare moddir=${moddir}
        inst_multiple _tools_my_module_script_needs_
    
        inst_hook pre-mount 30 "${moddir}/my-script.sh"
    }

That’s it! At the time KIWI NG calls dracut the 90my-module will be taken into account and is installed into the generated initrd. At boot time systemd calls the scripts as part of the dracut-pre-mount.service.

The dracut system offers a lot more possibilities to customize the initrd than shown in the example above. For more information, visit the dracut project page.

7.8.2 Boot Image Parameters

A dracut generated initrd in a KIWI NG image build process includes one or more of the KIWI NG provided dracut modules. The following list documents the available kernel boot parameters for this modules:

rd.kiwi.debug

Activates the debug log file for the KIWI NG part of the boot process at /run/initramfs/log/boot.kiwi.

rd.kiwi.install.pxe

Tells an OEM installation image to lookup the system image on a remote location specified in rd.kiwi.install.image.

rd.kiwi.install.image=URI

Specifies the remote location of the system image in a PXE based OEM installation

rd.kiwi.install.pass.bootparam

Tells an OEM installation image to pass an additional boot parameters to the kernel used to boot the installed image. This can be used e.g. to pass on first boot configuration for a PXE image. Note, that options starting with rd.kiwi are not passed on to avoid side effects.

rd.kiwi.oem.maxdisk=size[KMGT]

Configures the maximum disk size an unattended OEM installation should consider for image deployment. Unattended OEM deployments default to deploying on /dev/sda (more exactly, the first device not filtered out by oem-device-filter). With RAID controllers, it can happen that your buch of big JBOD disks is for example /dev/sda to /dev/sdi and the 480G RAID1 configured for OS deployment is /dev/sdj. With rd.kiwi.oem.maxdisk=500G the deployment will land on that RAID disk.

rd.live.overlay.size

Tells a live ISO image the size for the tmpfs filesystem that is used for the overlayfs mount process. If the write area of the overlayfs mount uses this tmpfs, any new data written during the runtime of the system will fillup this space. The default value used is set to 50% which means one half of the available RAM space can be used for writing new data.

rd.live.overlay.persistent

Tells a live ISO image to prepare a persistent write partition.

rd.live.overlay.cowfs

Tells a live ISO image which filesystem should be used to store data on the persistent write partition.

rd.live.cowfile.mbsize

Tells a live ISO image the size of the COW file in MB. When using tools like live-grub-stick the live ISO will be copied as a file on the target device and a GRUB loopback setup is created there to boot the live system from file. In such a case the persistent write setup, which usually creates an extra write partition on the target, will fail in almost all cases because the target has no free and unpartitioned space available. Because of that a cow file(live_system.cow) instead of a partition is created. The cow file will be created in the same directory the live iso image file was read from by grub and takes the configured size or the default size of 500MB.

rd.live.dir

Tells a live ISO image the directory which contains the live OS root directory. Defaults to LiveOS.

rd.live.squashimg

Tells a live ISO image the name of the squashfs image file which holds the OS root. Defaults to squashfs.img.

7.8.2.1 Boot Debugging

If the boot process encounters a fatal error, the default behavior is to stop the boot process without any possibility to interact with the system. Prevent this behavior by activating dracut’s builtin debug mode in combination with the kiwi debug mode as follows:

rd.debug rd.kiwi.debug

This should be set at the Kernel command line. With those parameters activated, the system will enter a limited shell environment in case of a fatal error during boot. The shell contains a basic set of commands and allows for a closer look to:

less /run/initramfs/log/boot.kiwi

7.9 Overview

KIWI NG builds so-called system images (a fully installed and optionally configured system in a single file) of a Linux distribution in two steps (for further details, see Image Building Process):

  1. Prepare operation: generate an unpacked image tree of your image. The unpacked tree is a directory containing the future file system of your image, generated from your image description.

  2. Create operation: the unpacked tree generated in step 1 is packaged into the format required for the final usage (e.g. a qcow2 disk image to launch the image with QEMU).

KIWI NG executes these steps using the following components, which it expects to find in the description directory:

Chapter 8, Image Description:

The config.xml file contains the image description, which is a collection of general settings of the final image, like the image layout installed packages, present users, etc.

Note
Note

The filename config.xml is not mandatory, the image description file can also have an arbitrary name plus the *.kiwi extension. KIWI NG first looks for a config.xml file. If it cannot be found, it picks the first *.kiwi file.

Section 7.6, “User Defined Scripts”:

If present, custom configuration shell scripts run at different stages of the build process. They can be used to fine tune the image in ways that are not possible via the settings provided in config.xml.

Overlay tree directory:

The overlay tree is a folder (called root) or a tarball (called root.tar.gz) that contains files and directories that will be copied into the unpacked image tree during the Prepare operation. The copying is executed after all the packages included in config.xml have been installed. Any already present files are overwritten.

CD root user data:

For live ISO images and install ISO images an optional archive is supported. This is a tar archive matching the name config-cdroot.tar[.compression_postfix].

If present, the archive will be unpacked as user data on the ISO image. For example, this is used to add license files or user documentation. The documentation can then be read directly from the CD/DVD without booting from the media.

7.10 Image Building Process

KIWI NG creates images in a two step process: The first step, the prepare operation, generates a so-called unpacked image tree (directory) using the information provided in the config.xml configuration file (see Chapter 8, Image Description)

The second step, the create operation, creates the packed image or image in the specified format based on the unpacked image tree and the information provided in the config.xml configuration file.

Image Creation Architecture
Figure 7.1: Image Creation Architecture

7.10.1 The Prepare Step

As the first step, KIWI NG creates an unpackaged image tree, also called “root tree”. This directory will be the installation target for software packages to be installed during the image creation process.

For the package installation, KIWI NG relies on the package manager specified in the packagemanager element in config.xml. KIWI NG supports the following package managers: dnf, zypper (default) and apt.

The prepare step consists of the following substeps:

  1. Create Target Root Directory

    By default KIWI NG aborts with an error if the target root tree already exists to avoid accidental deletion of an existing unpacked image. The option --allow-existing-root can be used to work based on an existing root tree

  2. Bootstrap Target Root Directory

    First, KIWI NG configures the package manager to use the repositories specified in the configuration file, via the command line, or both. After the repository setup, the packages specified in the bootstrap section of the image description are installed in a temporary directory external to the target root tree. This establishes the initial environment to support the completion of the process in a chroot setting. At the end of the bootstrap phase the script post_bootstrap.sh is executed, if present.

    Note
    Note

    The essential bootstrap packages are usually filesystem and glibc-locale to specify as part of the bootstrap. The dependency chain of these two packages is usually sufficient to populate the bootstrap environment with all required software to support the installation of packages into the new root tree.

  3. Install Packages

    After the bootstrap phase all other <packages> sections are used to complete the installation as chroot operation. KIWI NG uses the package manager as installed in the bootstrap phase and installs all other packages as configured.

    Note
    Note

    The installation of software packages through the selected package manager may install unwanted packages. Removing these packages can be accomplished by marking them for deletion in the image description, see Section 7.3.3, “Uninstall System Packages”.

  4. Apply the Overlay Tree

    Next, KIWI NG applies all files and directories present in the overlay directory named root or in the compressed overlay root.tar.gz to the target root tree. Files already present in the target root directory are overwritten. This allows you to overwrite any file that was installed by one of the packages during the installation phase.

  5. Apply Archives

    All archives specified in the archive element of the config.xml file are applied in the specified order (top to bottom) after the overlay tree copy operation is complete (see Section 7.3.2, “The archive element”). Files and directories are extracted relative to the top level of the new root tree. As with the overlay tree, it is possible to overwrite files already existing in the target root tree.

  6. Execute the user-defined script config.sh

    At the end of the preparation stage the script config.sh is executed (if present). It is run in the top level directory of the target root tree. The script’s primary function is to complete the system configuration. For more details about custom scripts see Section 7.6, “User Defined Scripts”

  7. Modify the Root Tree

    The unpacked image tree is now finished to be converted into the final image in the create step. It is possible to make manual modifications to the unpacked tree before it is converted into the final image.

    Since the unpacked image tree is just a directory, it can be modified using the standard tools. Optionally, it is also possible to “change root (chroot)” into it, for instance to invoke the package manager. Beside the standard file system layout, the unpacked image tree contains an additional directory named /image that is not present in a regular system. It contains information KIWI NG requires during the create step, including a copy of the config.xml file.

    By default, KIWI NG will not stop after the prepare step and will directly proceed with the create step. Therfore to perform manual modifications, proceed as follows:

    $ kiwi-ng system prepare $ARGS
    $ # make your changes
    $ kiwi-ng system create $ARGS
    Warning
    Warning

    Modifications of the unpacked root tree

    Do not make any changes to the system, since they are lost when re-running the prepare step again. Additionally, you may introduce errors that occur during the create step which are difficult to track. The recommended way to apply changes to the unpacked image directory is to change the configuration and re-run the prepare step.

7.10.2 The Create Step

KIWI NG creates the final image during the create step: it converts the unpacked root tree into one or multiple output files appropriate for the respective build type.

It is possible to create multiple images from the same unpacked root tree, for example, a self installing OEM image and a virtual machine image from the same image description. The only prerequisite is that both image types are specified in config.xml.

During the create step the following operations are performed by KIWI NG:

  1. Execute the User-defined Script images.sh

    At the beginning of the image creation process the script named images.sh is executed (if present). For more details about custom scripts see Section 7.6, “User Defined Scripts”

  2. Create the Requested Image Type

    KIWI NG converts the unpacked root into an output format appropriate for the requested build type.

Print this page