Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]

6 KIWI Plugins

Note
Note

This document provides a list of the existing KIWI Next Generation (KIWI NG) plugins which provides extended functionality for version 9.25.12.

6.1 Building in a Self-Contained Environment

Note
Note

Abstract

Users building images with KIWI NG face problems if they want to build an image matching one of the following criteria:

  • build should happen as non root user.

  • build should happen on a host system distribution for which no KIWI NG packages exists.

  • build happens on an incompatible host system distribution compared to the target image distribution. For example building an apt/dpkg based system on an rpm based system.

  • run more than one build process at the same time on the same host.

  • run a build process for a different target architecture compared to the host architecture (Cross Arch Image Build)

This document describes how to perform the build process in a self contained environment using fast booting virtual machines to address the issues listed above.

The changes on the machine to become a build host will be reduced to the requirements of the KIWI NG boxed plugin

6.1.1 Requirements

Add the KIWI NG repo from the Open Build Service. For details see Section 2.1, “Installation from OBS”. The following KIWI NG plugin needs to be installed on the build system:

$ sudo zypper in python3-kiwi_boxed_plugin

6.1.2 Building with the boxbuild command

The installation of the KIWI NG boxed plugin has registered a new kiwi command named boxbuild. The command implementation uses KVM as virtualization technology and runs the KIWI NG build command inside of a KVM controlled virtual machine. For running the build process in a virtual machine it’s required to provide VM images that are suitable to perform this job. We call the VM images boxes and they contain kiwi itself as well as all other components needed to build appliances. Those boxes are hosted in the Open Build Service and are publicly available at the Subprojects tab in the: Virtualization:Appliances:SelfContained project.

As a user you don’t need to work with the boxes because this is all done by the plugin and provided as a service by the KIWI NG team. The boxbuild command knows where to fetch the box and also cares for an update of the box when it has changed.

Building an image with the boxbuild command is similar to building with the build command. The plugin validates the given command call with the capabilities of the build command. Thus one part of the boxbuild command is exactly the same as with the build command. The separation between boxbuild and build options is done using the -- separator like shown in the following example:

$ kiwi-ng --type iso system boxbuild --box leap -- \
      --description kiwi/build-tests/x86/leap/test-image-disk \
      --set-repo obs://openSUSE:Leap:15.3/standard \
      --target-dir /tmp/myimage
Note
Note

The provided --description and --target-dir options are setup as shared folders between the host and the box. No other data will be shared with the host.

6.1.3 Sharing Backends

As mentioned above, the boxbuild call shares the two host directories provided in --description and --target-dir with the box. To do this the following sharing backends are supported:

--9p-sharing

With QEMU’s 9pfs you can create virtual filesystem devices (virtio-9p-device) and expose them to the box. For more information see 9pfs. Using this sharing backend does not require any setup procedure from the user and is also the default for boxbuild

--sshfs-sharing

SSHFS is a FUSE-based filesystem client for mounting remote directories over a Secure Shell connection (SSH). In boxbuild this is used to mount directories from the host into the box. Because this runs through an SSH connection the host must allow connections from the box. If you plan to use sshfs add the following SSH public key to the ~/.ssh/authorized_keys file of the user which is expected to call boxbuild

echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtiqDaYgEMkr7za7qc4iPXftgu/j3sodPOtpoG8PinwRX6/3xZteOJzCH2qCZjEgA5zsP9lxy/119cWXvdxFUvyEINjH77unzRnaHj/yTXPhHuhHgAiEubuHer2gZoOs+UH4cGJLKCrabjTjZdeK9KvL+hoAgJaWxDUvGsXYDQTBHXlKjniOL1MGbltDBHnYhu4k+PjjJ+UEBN+8+F74Y5fYgIovXXY88WQrybuEr1eAYjhvk/ln6TKw1P6uvVMuIbAGUgnZFntDCI91Qw8ps1j+lX3vNc8ZBoOwM6nHZqq4FAqbXuH+NvQFS/xDM6wwZQhAe+14dTQBA5F1mgCVf+fSbteb0/CraSGmgKIM8aPnK8rfF+BY6Jar3AJFKVRPshRzrQj6CWYu3BfmOLupCpqOK2XFyoU2lEpaZDejgPSJq/IBGZdjKplWJFF8ZRQ01a8eX8K2fjrQt/4k9c7Pjlg1aDH8Sf+5+vcehlSNs1d50wnFoaIPrgDdy04omiaJ8= kiwi@boxbuild" >> ~/.ssh/authorized_keys

The public key mentioned here is associated with an SSH key pair we provide in the pre-built box images.

Warning
Warning

If the sshfs backend is used without the host trusting the box, the boxbuild call will become interactive at the time of the sshfs mount. In this case the user might be asked for a passphrase or depending on the host sshd setup the request will be declined and the boxbuild fails.

--virtiofs-sharing

QEMU virtio-fs shared file system daemon. Share a host directory tree with a box through a virtio-fs device. For more information see virtiofs. Using this sharing backend does not require any setup procedure from the user

Warning
Warning

virtiofs support was added but considered experimental and not yet stable across the distributions. Feedback welcome.

6.2 Building based on Containers

Note
Note

Abstract

When building images exposes one of the following requirements the stackbuild plugin provides an opportunity to address it:

  • Preserve the image rootfs for a later rebuild without requiring the original software repositories.

  • Build an image based on an existing container.

  • Build an image based on a container stack.

  • Transform a container into a KIWI NG image type

6.2.1 Installation

Add the KIWI NG repo from the Open Build Service. For details see Section 2.1, “Installation from OBS”. The following KIWI NG plugin needs to be installed on the build system:

$ sudo zypper in python3-kiwi_stackbuild_plugin

6.2.2 Concept

The design of the stackbuild plugin is two fold:

First the plugin comes with a command called stash which allows to store a kiwi built root tree as an OCI container. OCI stands for Open Container Interface and is a defacto standard format in the container world. Once the container got created it can be managed using the preferred container toolchain. The plugin code itself uses podman to work with containers.

As a next step and with the root tree as a container the plugin offers the opportunity to build images based on one ore more containers. That’s also the reason why the plugin is called stackbuild as it allows you to stack different root containers together. Consequently the other command provided is named stackbuild.

The stash and stackbuild commands can be used independently from each other. If there is already a registry with containers that should be used to build images from, stackbuild can directly consume them.

This concept leads to a number of use cases and a few of them were picked and put into the abstract of this article. For the purpose of documenting the functionality of the plugin only a part of the possibilities are taken into account as follows:

6.2.3 Create a stash

The stash command creates an OCI compliant container from a given KIWI Next Generation (KIWI NG) image root tree and registers it in the local container registry. From there a user can push it to any registry of choice.

The following example creates a stash of a Tumbleweed build and illustrates how to register it in a foreign container registry:

# Build some image...
$ git clone https://github.com/OSInside/kiwi.git
$ sudo kiwi-ng system build \
    --description kiwi/build-tests/x86/tumbleweed/test-image-MicroOS/ \
    --set-repo http://download.opensuse.org/tumbleweed/repo/oss \
    --target-dir /tmp/myTWToday

# Stash the image root into a container
$ sudo kiwi-ng system stash \
    --root /tmp/myTWToday/build/image-root \
    --container-name twmos-snapshot

# Register the stash in a registry
$ podman login
$ podman push twmos-20211008 \
    docker://docker.io/.../twmos-snapshot:2021-10-08

If the stash command is called multiple times with the same container-name this leads to a new layer in the container for each call. To inspect the number of layers added to the container the following command can be used:

$ podman inspect twmos-snapshot

To list all stashes created by the stash command the following command can be used

$ kiwi-ng system stash --list

6.2.4 Rebuild from a stash

The stackbuild command takes the given container(s) from the local or remote registry and uses it/them to either rebuild an image from that data or build a new image on top of that data. If multiple containers are given the stackbuild command stacks them together in the order as they were provided.

Note
Note

When using multiple containers the result stack root tree is created from a sequence of rsync commands into the same target directory. The stackbuild plugin does this with any container content given and does not check, validate or guarantee that the selection of containers are actually stackable or leads to an usable root tree. This means it’s in the responsibility of the caller to make sure the provided containers can actually be stacked together in the given order.

To simply rebuild the image from the stash created in Create a stash call stackbuild as follows:

# Delete the image
$ sudo rm -rf /tmp/myTWToday

# Rebuild image from stash
$ sudo kiwi-ng system stackbuild \
    --stash twmos-snapshot:2021-10-08 \
    --target-dir /tmp/myTWToday

This rebuilds the image from the stash and the KIWI NG configuration inside of the stash. As all rootfs data is already in the stash, the command will not need external resources to rebuild the image.

6.2.5 Turn a container into a VM image

Another use case for the stackbuild plugin is the transformation of container images into another image type that is supported by KIWI NG. The following example demonstrates how an existing container image from the openSUSE registry can be turned into a virtual machine image.

When moving a container into a virtual machine image the following aspects has to be taken into account:

  1. A container image usually has no kernel installed.

  2. A container image usually has no bootloader installed.

  3. A container image usually has no user configured.

For a VM image the mentioned aspects are mandatory. Therefore the following KIWI NG image description contains this additional information which the container cannot provide: Create the KIWI NG description as follows:

$ mkdir container_to_VM_layer
$ vi container_to_VM_layer/config.kiwi

And place the following content:

<?xml version="1.0" encoding="utf-8"?>

<image schemaversion="6.8" name="Leap-VM">
    <description type="system">
        <author>The Author</author>
        <contact>user@example.org</contact>
        <specification>
            Leap Container as VM
        </specification>
    </description>
    <preferences>
        <type image="oem" filesystem="xfs" firmware="uefi">
            <oemconfig>
                <oem-resize>false</oem-resize>
            </oemconfig>
        </type>
        <version>1.99.1</version>
        <packagemanager>zypper</packagemanager>
        <locale>en_US</locale>
        <keytable>us</keytable>
        <timezone>UTC</timezone>
    </preferences>
    <repository type="rpm-md" alias="Leap_15_3">
        <source path="obs://openSUSE:Leap:15.3/standard"/>
    </repository>
    <packages type="image">
        <package name="grub2"/>
        <package name="grub2-x86_64-efi" arch="x86_64"/>
        <package name="grub2-i386-pc"/>
        <package name="shim"/>
        <package name="kernel-default"/>
    </packages>
    <users>
        <user password="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root" groups="root"/>
    </users>
</image>

To build the virtual machine image from the current hosted Leap 15.3 container at SUSE, call the following stackbuild command:

$ sudo kiwi-ng system stackbuild \
    --stash leap:15.3 \
    --from-registry registry.opensuse.org/opensuse \
    --target-dir /tmp/myLeap \
    --description container_to_VM_layer

The resulting virtual machine image can be booted as follows:

$ qemu-kvm Leap-VM.x86_64-1.99.1.raw
Print this page