This document provides a list of the existing KIWI Next Generation (KIWI NG) plugins which provides extended functionality for version 9.25.12.
Abstract
Users building images with KIWI NG face problems if they want to build an image matching one of the following criteria:
build should happen as non root user.
build should happen on a host system distribution for which no KIWI NG packages exists.
build happens on an incompatible host system distribution compared to the target image distribution. For example building an apt/dpkg based system on an rpm based system.
run more than one build process at the same time on the same host.
run a build process for a different target architecture compared to the host architecture (Cross Arch Image Build)
This document describes how to perform the build process in a self contained environment using fast booting virtual machines to address the issues listed above.
The changes on the machine to become a build host will
be reduced to the requirements of the KIWI NG boxed plugin
Add the KIWI NG repo from the Open Build Service. For details see Section 2.1, “Installation from OBS”. The following KIWI NG plugin needs to be installed on the build system:
$ sudo zypper in python3-kiwi_boxed_plugin
The installation of the KIWI NG boxed plugin has registered a new kiwi
command named boxbuild
. The command implementation uses KVM as
virtualization technology and runs the KIWI NG build
command inside of
a KVM controlled virtual machine. For running the build process in a
virtual machine it’s required to provide VM images that are suitable
to perform this job. We call the VM images boxes
and they contain
kiwi itself as well as all other components needed to build appliances.
Those boxes are hosted in the Open Build Service and are publicly
available at the Subprojects
tab in the: Virtualization:Appliances:SelfContained
project.
As a user you don’t need to work with the boxes because this is all done
by the plugin and provided as a service by the KIWI NG team. The boxbuild
command knows where to fetch the box and also cares for an update of the
box when it has changed.
Building an image with the boxbuild
command is similar to building with
the build
command. The plugin validates the given command call with the
capabilities of the build
command. Thus one part of the boxbuild
command
is exactly the same as with the build
command. The separation between
boxbuild
and build
options is done using the --
separator like
shown in the following example:
$ kiwi-ng --type iso system boxbuild --box leap -- \
--description kiwi/build-tests/x86/leap/test-image-disk \
--set-repo obs://openSUSE:Leap:15.3/standard \
--target-dir /tmp/myimage
The provided --description
and --target-dir
options are
setup as shared folders between the host and the box. No other
data will be shared with the host.
As mentioned above, the boxbuild
call shares the two host directories
provided in --description
and --target-dir
with the box. To do this
the following sharing backends are supported:
--9p-sharing
With QEMU’s 9pfs
you can create virtual filesystem devices
(virtio-9p-device) and expose them to the box. For more information
see 9pfs. Using
this sharing backend does not require any setup procedure from the
user and is also the default for boxbuild
--sshfs-sharing
SSHFS is a FUSE-based filesystem client for mounting remote
directories over a Secure Shell connection (SSH). In boxbuild
this is used to mount directories from the host into the box.
Because this runs through an SSH connection the host must allow
connections from the box. If you plan to use sshfs
add the
following SSH public key to the ~/.ssh/authorized_keys
file of the user which is expected to call boxbuild
echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtiqDaYgEMkr7za7qc4iPXftgu/j3sodPOtpoG8PinwRX6/3xZteOJzCH2qCZjEgA5zsP9lxy/119cWXvdxFUvyEINjH77unzRnaHj/yTXPhHuhHgAiEubuHer2gZoOs+UH4cGJLKCrabjTjZdeK9KvL+hoAgJaWxDUvGsXYDQTBHXlKjniOL1MGbltDBHnYhu4k+PjjJ+UEBN+8+F74Y5fYgIovXXY88WQrybuEr1eAYjhvk/ln6TKw1P6uvVMuIbAGUgnZFntDCI91Qw8ps1j+lX3vNc8ZBoOwM6nHZqq4FAqbXuH+NvQFS/xDM6wwZQhAe+14dTQBA5F1mgCVf+fSbteb0/CraSGmgKIM8aPnK8rfF+BY6Jar3AJFKVRPshRzrQj6CWYu3BfmOLupCpqOK2XFyoU2lEpaZDejgPSJq/IBGZdjKplWJFF8ZRQ01a8eX8K2fjrQt/4k9c7Pjlg1aDH8Sf+5+vcehlSNs1d50wnFoaIPrgDdy04omiaJ8= kiwi@boxbuild" >> ~/.ssh/authorized_keys
The public key mentioned here is associated with an SSH key pair we provide in the pre-built box images.
If the sshfs
backend is used without the host trusting the box,
the boxbuild
call will become interactive at the time of the sshfs
mount. In this case the user might be asked for a passphrase or
depending on the host sshd
setup the request will be declined and
the boxbuild fails.
--virtiofs-sharing
QEMU virtio-fs shared file system daemon. Share a host directory tree with a box through a virtio-fs device. For more information see virtiofs. Using this sharing backend does not require any setup procedure from the user
virtiofs support was added but considered experimental and not yet stable across the distributions. Feedback welcome.
Abstract
When building images exposes one of the following requirements the stackbuild plugin provides an opportunity to address it:
Preserve the image rootfs for a later rebuild without requiring the original software repositories.
Build an image based on an existing container.
Build an image based on a container stack.
Transform a container into a KIWI NG image type
Add the KIWI NG repo from the Open Build Service. For details see Section 2.1, “Installation from OBS”. The following KIWI NG plugin needs to be installed on the build system:
$ sudo zypper in python3-kiwi_stackbuild_plugin
The design of the stackbuild plugin is two fold:
First the plugin comes with a command called stash
which allows
to store a kiwi built root tree as an OCI container. OCI stands for
Open Container Interface and is a defacto standard format in the
container world. Once the container got created it can be managed
using the preferred container toolchain. The plugin code itself
uses podman
to work with containers.
As a next step and with the root tree as a container the plugin offers
the opportunity to build images based on one ore more containers.
That’s also the reason why the plugin is called stackbuild as it
allows you to stack different root containers together.
Consequently the other command provided is named stackbuild
.
The stash
and stackbuild
commands can be used independently
from each other. If there is already a registry with containers
that should be used to build images from, stackbuild
can
directly consume them.
This concept leads to a number of use cases and a few of them were picked and put into the abstract of this article. For the purpose of documenting the functionality of the plugin only a part of the possibilities are taken into account as follows:
The stash
command creates an OCI compliant container from a given
KIWI Next Generation (KIWI NG) image root tree and registers it in the local
container registry. From there a user can push it to any registry
of choice.
The following example creates a stash of a Tumbleweed build and illustrates how to register it in a foreign container registry:
# Build some image...
$ git clone https://github.com/OSInside/kiwi.git
$ sudo kiwi-ng system build \
--description kiwi/build-tests/x86/tumbleweed/test-image-MicroOS/ \
--set-repo http://download.opensuse.org/tumbleweed/repo/oss \
--target-dir /tmp/myTWToday
# Stash the image root into a container
$ sudo kiwi-ng system stash \
--root /tmp/myTWToday/build/image-root \
--container-name twmos-snapshot
# Register the stash in a registry
$ podman login
$ podman push twmos-20211008 \
docker://docker.io/.../twmos-snapshot:2021-10-08
If the stash
command is called multiple times with the same
container-name this leads to a new layer in the container for
each call. To inspect the number of layers added to the
container the following command can be used:
$ podman inspect twmos-snapshot
To list all stashes created by the stash
command the following
command can be used
$ kiwi-ng system stash --list
The stackbuild
command takes the given container(s) from the local or
remote registry and uses it/them to either rebuild an image from that
data or build a new image on top of that data. If multiple containers
are given the stackbuild
command stacks them together in the order
as they were provided.
When using multiple containers the result stack root tree is created from a sequence of rsync commands into the same target directory. The stackbuild plugin does this with any container content given and does not check, validate or guarantee that the selection of containers are actually stackable or leads to an usable root tree. This means it’s in the responsibility of the caller to make sure the provided containers can actually be stacked together in the given order.
To simply rebuild the image from the stash created in Create a stash
call stackbuild
as follows:
# Delete the image
$ sudo rm -rf /tmp/myTWToday
# Rebuild image from stash
$ sudo kiwi-ng system stackbuild \
--stash twmos-snapshot:2021-10-08 \
--target-dir /tmp/myTWToday
This rebuilds the image from the stash and the KIWI NG configuration inside of the stash. As all rootfs data is already in the stash, the command will not need external resources to rebuild the image.
Another use case for the stackbuild
plugin is the transformation
of container images into another image type that is supported by KIWI NG.
The following example demonstrates how an existing container image
from the openSUSE registry can be turned into a virtual machine image.
When moving a container into a virtual machine image the following aspects has to be taken into account:
A container image usually has no kernel installed.
A container image usually has no bootloader installed.
A container image usually has no user configured.
For a VM image the mentioned aspects are mandatory. Therefore the following KIWI NG image description contains this additional information which the container cannot provide: Create the KIWI NG description as follows:
$ mkdir container_to_VM_layer
$ vi container_to_VM_layer/config.kiwi
And place the following content:
<?xml version="1.0" encoding="utf-8"?>
<image schemaversion="6.8" name="Leap-VM">
<description type="system">
<author>The Author</author>
<contact>user@example.org</contact>
<specification>
Leap Container as VM
</specification>
</description>
<preferences>
<type image="oem" filesystem="xfs" firmware="uefi">
<oemconfig>
<oem-resize>false</oem-resize>
</oemconfig>
</type>
<version>1.99.1</version>
<packagemanager>zypper</packagemanager>
<locale>en_US</locale>
<keytable>us</keytable>
<timezone>UTC</timezone>
</preferences>
<repository type="rpm-md" alias="Leap_15_3">
<source path="obs://openSUSE:Leap:15.3/standard"/>
</repository>
<packages type="image">
<package name="grub2"/>
<package name="grub2-x86_64-efi" arch="x86_64"/>
<package name="grub2-i386-pc"/>
<package name="shim"/>
<package name="kernel-default"/>
</packages>
<users>
<user password="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root" groups="root"/>
</users>
</image>
To build the virtual machine image from the current hosted Leap 15.3
container at SUSE, call the following stackbuild
command:
$ sudo kiwi-ng system stackbuild \
--stash leap:15.3 \
--from-registry registry.opensuse.org/opensuse \
--target-dir /tmp/myLeap \
--description container_to_VM_layer
The resulting virtual machine image can be booted as follows:
$ qemu-kvm Leap-VM.x86_64-1.99.1.raw