11 Working with Images #

Abstract. This page provides further information for handling ISO images built with KIWI NG and references the following articles:

In KIWI NG all generated ISO images are created to be hybrid. This means, the image can be used as a CD/DVD or as a disk. This works because the ISO image also has a partition table embedded. With more and more computers delivered without a CD/DVD drive this becomes important.

The very same ISO image can be copied onto a USB stick and used as a bootable disk. The following procedure shows how to do this:

Abstract. This page provides further information for handling ISO images built with KIWI NG and references the following articles:

In KIWI NG, all generated ISO images are created to be hybrid. This means, the image can be used as a CD/DVD or as a disk. The deployment of such an image onto a disk like an USB stick normally destroys all existing data on this device. Most USB sticks are pre-formatted with a FAT32 Windows File System and to keep the existing data on the stick untouched a different deployment needs to be used.

The following deployment process copies the ISO image as an additional file to the USB stick and makes the USB stick bootable. The ability to boot from the stick is configured through a SYSLINUX feature which allows to loopback mount an ISO file and boot the kernel and initrd directly from the ISO file.

The initrd loaded in this process must also be able to loopback mount the ISO file to access the root filesystem and boot the live system. The dracut initrd system used by KIWI NG provides this feature upstream called as “iso-scan”. Therefore all KIWI NG generated live ISO images supports this deployment mode.

For copying the ISO file on the USB stick and the setup of the SYSLINUX bootloader to make use of the “iso-scan” feature an extra tool named live-grub-stick exists. The following procedure shows how to setup the USB stick with live-grub-stick:

Abstract. This page provides further information for handling ISO images built with KIWI NG and references the following articles:

In KIWI NG, all generated ISO images are created to be hybrid. This means, the image can be used as a CD/DVD or as a disk. This works because the ISO image also has a partition table embedded. With more and more computers delivered without a CD/DVD drive this becomes important. The deployment of such an image onto a disk like an USB stick normally destroys all existing data on this device. It is also not possible to use USB stick as a data storage device. Most USB sticks are pre-formatted with a FAT32 or exFAT Windows File System and to keep the existing data on the stick untouched a different deployment needs to be used.

Fortunately Grub2 supports booting directly from ISO files. It does not matter whether it is installed on your computer’s hard drive or on a USB stick. The following deployment process copies the ISO image as an additional file to the USB stick or hard drive. The ability to boot from the disk is configured through a Grub2 feature which allows to loopback mount an ISO file and boot the kernel and initrd directly from the ISO file.

The initrd loaded in this process must also be able to loopback mount the ISO file to access the root filesystem and boot the live system. Almost every Linux distribution supports fat32, and more and more of them also support exFAT. For hard drives, Linux filesystems are also supported.

The dracut initrd system used by KIWI NG provides this feature upstream called as “iso-scan/filename”. Therefore all KIWI NG generated live ISO images supports this deployment mode.

The following procedure shows how to setup Grub2 on your hard drive:

For USB sticks, the procedure is identical. You would then install Grub2 on the USB drive and follow the steps above. The use of scripts such as “MultiOS-USB” is strongly recommended.

Abstract. This page provides further information for handling Amazon EC2 images built with KIWI NG and references the following articles:

A virtual disk image which is able to boot in the Amazon EC2 cloud framework has to comply the following constraints:

To meet this requirements add or update the KIWI NG image description as follows:

An image built with the above setup can be uploaded into the Amazon EC2 cloud and registered as image. For further information on how to upload to EC2 see: ec2uploadimg

Abstract. This page provides further information for handling Azure disk images built with KIWI NG and references the following articles:

A virtual disk image which is able to boot in the Microsoft Azure cloud framework has to comply the following constraints:

To meet this requirements update the KIWI NG image description as follows:

An image built with the above setup can be uploaded into the Microsoft Azure cloud and registered as image. For further information on how to upload to Azure see: azurectl

Abstract. This page provides further information for handling GCE images built with KIWI NG and references the following articles:

A virtual disk image which is able to boot in the Google Compute Engine cloud framework has to comply the following constraints:

To meet this requirements update the KIWI NG image description as follows:

An image built with the above setup can be uploaded into the Google Compute Engine cloud and registered as image. For further information on how to upload to Google see: google-cloud-sdk on software.opensuse.org

Abstract. This page provides further information for handling Vagrant controlled disk images built with KIWI NG and references the following article:

Vagrant is a framework to implement consistent processing/testing work environments based on Virtualization technologies. To run a system, Vagrant needs so-called boxes. A box is a TAR archive containing a virtual disk image and some metadata.

To build Vagrant boxes, you can use Packer which is provided by Hashicorp itself. Packer is based on the official installation media (DVDs) as shipped by the distribution vendor.

The KIWI NG way of building images might be helpful, if such a media does not exist or does not suit your needs. For example, if the distribution is still under development or you want to use a collection of your own repositories. Note, that in contrast to Packer KIWI NG only supports the libvirt and VirtualBox providers. Other providers require a different box layout that is currently not supported by KIWI NG.

In addition, you can use the KIWI NG image description as source for the Open Build Service which allows building and maintaining boxes.

Vagrant expects boxes to be setup in a specific way (for details refer to the Vagrant box documentation.), applied to the referenced KIWI NG image description from Section 10.2, “Build a Virtual Disk Image”, the following steps are required:

An image built with the above setup creates a Vagrant box file with the extension .vagrant.libvirt.box or .vagrant.virtualbox.box. Add the box file to Vagrant with the command:

vagrant box add my-box image-file.vagrant.libvirt.box

Note

Using the box with the libvirt provider requires alongside a correct Vagrant installation:

• the plugin vagrant-libvirt to be installed

• a running libvirtd daemon

Once added to Vagrant, boot the box and log in with the following sequence of vagrant commands:

vagrant init my-box
vagrant up --provider libvirt
vagrant ssh

11.7.1 Customizing the embedded Vagrantfile #

 

Warning

This is an advanced topic and not required for most users

Vagrant ship with an embedded Vagrantfile that carries settings specific to this box, for instance the synchronization mechanism for the shared folder. KIWI NG generates such a file automatically for you and it should be sufficient for most use cases.

If a box requires different settings in the embedded Vagrantfile, then the user can provide KIWI NG with a path to an alternative via the attribute embebbed_vagrantfile of the vagrantconfig element: it specifies a relative path to the Vagrantfile that will be included in the finished box.

In the following example snippet from config.xml we add a custom MyVagrantfile into the box (the file should be in the image description directory next to config.sh):

<type image="oem" filesystem="ext4" format="vagrant">
    <bootloader name="grub2" timeout="0"/>
    <vagrantconfig
      provider="libvirt"
      virtualsize="42"
      embedded_vagrantfile="MyVagrantfile"
    />
    <size unit="G">42</size>
    <oemconfig>
        <oem-resize>false</oem-resize>
    </oemconfig>
</type>

The option to provide a custom Vagrantfile can be combined with the usage of profiles (see Section 7.4, “Image Profiles”), so that certain builds can use the automatically generated Vagrantfile (in the following example that is the Virtualbox build) and others get a customized one (the libvirt profile in the following example):

<?xml version="1.0" encoding="utf-8"?>

<image schemaversion="8.0" name="{exc_image_base_name}">
  <!-- description goes here -->
  <profiles>
    <profile name="libvirt" description="Vagrant Box for Libvirt"/>
    <profile name="virtualbox" description="Vagrant Box for VirtualBox"/>
  </profiles>

  <!-- general preferences go here -->

  <preferences profiles="libvirt">
    <type
      image="oem"
      filesystem="ext4"
      format="vagrant">
        <bootloader name="grub2" timeout="0"/>
        <vagrantconfig
          provider="libvirt"
          virtualsize="42"
          embedded_vagrantfile="LibvirtVagrantfile"
        />
        <size unit="G">42</size>
        <oemconfig>
            <oem-resize>false</oem-resize>
        </oemconfig>
   </type>
   </preferences>
   <preferences profiles="virtualbox">
     <type
       image="oem"
       filesystem="ext4"
       format="vagrant">
         <bootloader name="grub2" timeout="0"/>
         <vagrantconfig
           provider="virtualbox"
           virtualbox_guest_additions_present="true"
           virtualsize="42"
         />
         <size unit="G">42</size>
         <oemconfig>
             <oem-resize>false</oem-resize>
         </oemconfig>
     </type>
   </preferences>

   <!-- remaining box description -->
 </image>

Abstract. This page provides further information for handling disk images with an encrypted root filesystem setup. The information here is based on top of the following article:

A virtual disk image can be partially or fully encrypted using the LUKS extension supported by KIWI NG. A fully encrypted image also includes the data in /boot to be encrypted. Such an image requests the passphrase for the master key to be entered at the bootloader stage. A partialy encrypted image keeps /boot unencrypted and on an extra boot partition. Such an image requests the passphrase for the master key later in the boot process when the root partition gets accessed by the systemd mount service. In any case the master passphrase is requested only once.

Update the KIWI NG image description as follows:

Abstract. This page provides further information for handling oem images built with KIWI NG and references the following articles:

If a machine should run the OS completely in memory without the need for any persistent storage, the approach to deploy the image into a ramdisk serves this purpose. KIWI NG allows to create a bootable ISO image which deploys the image into a ramdisk and activates that image with the following oem type definition:

<type image="oem" filesystem="ext4" installiso="true" initrd_system="dracut" installboot="install" kernelcmdline="rd.kiwi.ramdisk ramdisk_size=2048000">
    <bootloader name="grub2" timeout="1"/>
    <oemconfig>
        <oem-skip-verify>true</oem-skip-verify>
        <oem-unattended>true</oem-unattended>
        <oem-unattended-id>/dev/ram1</oem-unattended-id>
        <oem-swap>false</oem-swap>
        <oem-multipath-scan>false</oem-multipath-scan>
     </oemconfig>
 </type>

The type specification above builds an installation ISO image which deploys the System Image into the specified ramdisk device (/dev/ram1). The setup of the ISO image boots with a short boot timeout of 1sec and just runs through the process without asking any questions. In a ramdisk deployment the optional target verification, swap space and multipath targets are out of scope and therefore disabled.

The configured size of the ramdisk specifies the size of the OS disk and must be at least of the size of the System Image. The disk size can be configured with the following value in the kernelcmdline attribute:

An image built with the above setup can be tested in QEMU as follows:

$ sudo qemu -cdrom \
      {exc_image_base_name}.x86_64-1.15.3.install.iso \
      -serial stdio

Note

Enough Main Memory

The machine, no matter if it’s a virtual machine like QEMU or a real machine, must provide enough RAM to hold the image in the ramdisk as well as have enough RAM available to operate the OS and its applications. The KIWI NG build image with the extension .raw provides the System Image which gets deployed into the RAM space. Substract the size of the System Image from the RAM space the machine offers and make sure the result is still big enough for the use case of the appliance. In case of a virtual machine, attach enough main memory to fit this calculation. In case of QEMU this can be done with the -m option

Like all other oem KIWI NG images, also the ramdisk setup supports all the deployments methods as explained in Section 10.3.1, “Deployment Methods” This means it’s also possible to dump the ISO image on a USB stick let the system boot from it and unplug the stick from the machine because the system was deployed into RAM

Note

Limitations Of RamDisk Deployments

Only standard images which can be booted by a simple root mount and root switch can be used. Usually KIWI NG calls kexec after deployment such that the correct, for the image created dracut initrd, will boot the image. In case of a RAM only system kexec does not work because it would loose the ramdisk contents. Thus the dracut initrd driving the deployment is also the environment to boot the system. There are cases where this environment is not suitable to boot the system.

Abstract. This page provides details about the opportunities and limitations to customize the partition table in addition to the volume management settings from Section 11.11, “Custom Disk Volumes”.

KIWI NG has its own partitioning schema which is defined according to several different user configurations: boot firmware, boot partition, expandable layouts, etc. Those supported features have an impact on the partitioning schema.

MBR or GUID partition tables are not flexible, carry limitations and are tied to some specific disk geometry. Because of that the preferred alternative to disk layouts based on traditional partition tables is using flexible approaches like logic volumes.

However, on certain conditions additional entries to the low level partition table are needed. For this purpose the <partitions> section exists and allows to add custom entries like shown in the following example:

<partitions>
    <partition name="var" size="100" mountpoint="/var" filesystem="ext3"/>
</partitions>

Each <partition> entry puts a partition of the configured size in the low level partition table, creates a filesystem on it and includes it to the system’s fstab file. If parts of the root filesystem are moved into its own partition like it’s the case in the above example, this partition will also contain the data that gets installed during the image creation process to that area.

The following attributes must/can be set to configured a partition entry:

Despite the customization options of the partition table shown above there are the following limitations:

Abstract. This chapter provides high level explanations on how to handle volumes or subvolumes definitions for disk images using KIWI NG.

KIWI NG supports defining custom volumes by using the logical volume manager (LVM) for the Linux kernel or by setting volumes at filesystem level when filesystem supports it (e.g. btrfs).

Volumes are defined in the KIWI NG description file config.xml, using systemdisk. This element is a child of the type. Volumes themselves are added via (multiple) volume child elements of the systemdisk element:

<image schemaversion="8.0" name="openSUSE-Leap-15.1">
  <type image="oem" filesystem="btrfs">
    <systemdisk name="vgroup" preferlvm="true">
      <volume name="usr/lib" size="1G" label="library"/>
      <volume name="@root" freespace="500M"/>
      <volume name="etc_volume" mountpoint="etc" copy_on_write="false"/>
      <volume name="bin_volume" size="all" mountpoint="/usr/bin" quota="2G"/>
    </systemdisk>
  </type>
</image>

Additional non-root volumes are created for each volume element. Volume details can be defined by setting the following volume attributes:

Warning

The size attributes for filesystem volumes, as for btrfs, are ignored and have no effect.

The systemdisk element additionally supports the following optional attributes:

Abstract. This page provides details about the partition clone feature and its use cases

KIWI NG allows to create block level clones of certain partitions used in the image. Clones can be created from the root, boot and any other partition listed in the <partitions> element.

A partition clone is a simple byte dump from one block storage device to another. However, this would cause conflicts during boot of the system because all unique identifiers like the UUID of a filesystem will no longer be unique. The clone feature of KIWI NG takes care of this part and re-creates the relevant unique identifiers per cloned partition. KIWI NG allows this also for complex partitions like LVM, LUKS or RAID.

The partition clone(s) will always appear first in the partition table, followed by the origin partition. The origin partition is the one whose identifier will be referenced and used by the system. By default no cloned partition will be mounted or used by the system at boot time.

Let’s take a look at the following example:

<type image="oem" root_clone="1" boot_clone="1" firmware="uefi" filesystem="xfs" bootpartition="true" bootfilesystem="ext4">
    <partitions>
        <partition name="home" size="10" mountpoint="/home" filesystem="ext3" clone="2"/>
    </partitions>
    <oemconfig>
        <oem-systemsize>2048</oem-systemsize>
        <oem-resize>false</oem-resize>
    </oemconfig>
    <size unit="G">100</size>
</type>

With the above setup KIWI NG will create a disk image that contains the following partition table:

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048            6143   2.0 MiB     EF02  p.legacy
   2            6144           47103   20.0 MiB    EF00  p.UEFI
   3           47104          661503   300.0 MiB   8300  p.lxbootclone1
   4          661504         1275903   300.0 MiB   8300  p.lxboot
   5         1275904         1296383   10.0 MiB    8300  p.lxhomeclone1
   6         1296384         1316863   10.0 MiB    8300  p.lxhomeclone2
   7         1316864         1337343   10.0 MiB    8300  p.lxhome
   8         1337344         3864575   2 GiB       8300  p.lxrootclone1
   9         3864576         6287326   2 GiB       8300  p.lxroot

When booting the system only the origin partitions p.lxboot, p.lxroot and p.lxhome will be mounted and visible in e.g. /etc/fstab, the bootloader or the initrd. Thus partition clones are present as a data source but are not relevant for the operating system from a functional perspective.

As shown in the above example there is one clone request for root and boot and a two clone requests for the home partition. KIWI NG does not sanity- check the provided number of clones (e.g. whether your partition table can hold that many partitions).

Warning

There is a limit how many partitions a partition table can hold. This also limits how many clones can be created.

It’s important when using the root_clone attribute to specify a size for the part of the system that represents the root partition. As of today KIWI NG does not automatically divide the root partition into two identical pieces. In order to create a clone of the partition a size specification is required. In the above example the size for root is provided via the oem-systemsize element. Using a root clone and fixed size values has the following consequences:

11.12.1 Use Case #

 

Potential use cases for which a clone of one or more partitions is useful include among others:

Factory Resets:

Creating an image with the option to rollback to the state of the system at deployment time can be very helpful for disaster recovery

System Updates with Rollbacks e.g A/B:

Creating an image which holds extra space allowing to rollback modified data can make a system more robust. For example in a simple A/B update concept, partition A would get updated but would flip to B if A is considered broken after applying the update.

Note

Most probably any use case based on partition clones requires additional software to manage them. KIWI NG provides the option to create the clone layout but it does not provide the software to implement the actual use case for which the partition clones are needed.

Developers writing applications based on a clone layout created with KIWI NG can leverage the metadata file /config.partids. This file is created at build time and contains the mapping between the partition name and the actual partition number in the partition table. For partition clones, the following naming convention applies:

kiwi_(name)PartClone(id)="(partition_number)"

The (name) is either taken from the name attribute of the <partition> element or it is a fixed name assigned by KIWI NG. There are the following reserved partition names for which cloning is supported:

• root

• readonly

• boot

For the mentioned example this will result in the following /config.partids:

kiwi_BiosGrub="1"
kiwi_EfiPart="2"
kiwi_bootPartClone1="3"
kiwi_BootPart="4"
kiwi_homePartClone1="5"
kiwi_homePartClone2="6"
kiwi_HomePart="7"
kiwi_rootPartClone1="8"
kiwi_RootPart="9"

Abstract. This page provides general information how to setup a network boot server that provides all services needed for the PXE boot protocol

To be able to deploy a system through the PXE boot protocol, you need to set up a network boot server providing the services DHCP and tftp. With dnsmasq an utility exists which allows to setup all needed services at once:

11.13.1 Installing and Configuring DHCP and TFTP with dnsmasq #

 

The following instructions can only serve as an example. Depending on your network structure, the IP addresses, ranges and domain settings needs to be adapted to allow the DHCP server to work within your network. If you already have a DHCP server running in your network, make sure that the filename and next-server directives are correctly set on this server.

The following steps describe how to set up dnsmasq to work as DHCP and TFTP server.

1. Install the dnsmasq package.

2. Create the file /etc/dnsmasq.conf and insert the following content

   # Don't function as a DNS server.
   port=0
   
   # Log information about DHCP transactions.
   log-dhcp
   
   # Set the root directory for files available via FTP,
   # usually "/srv/tftpboot":
   tftp-root=TFTP_ROOT_DIR
   
   enable-tftp
   
   dhcp-range=BOOT_SERVER_IP,proxy

   In the next step it’s required to decide for the boot method. There is the PXE loader provided via pxelinux.0 from the syslinux package and there is the GRUB loader provided via the grub package.

   

   Note

   Placeholders

   Replace all placeholders (written in uppercase) with data fitting your network setup.

2.1. insert the following content to use pxelinux.0:

# The boot filename, Server name, Server Ip Address
dhcp-boot=pxelinux.0,,BOOT_SERVER_IP

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken
# DHCP clients.
dhcp-no-override

# PXE menu.  The first part is the text displayed to the user.
# The second is the timeout, in seconds.
pxe-prompt="Booting FOG Client", 1

# The known types are x86PC, PC98, IA64_EFI, Alpha, Arc_x86,
# Intel_Lean_Client, IA32_EFI, BC_EFI, Xscale_EFI and X86-64_EFI
# This option is first and will be the default if there is no input
# from the user.
pxe-service=X86PC, "Boot to FOG", pxelinux.0
pxe-service=X86-64_EFI, "Boot to FOG UEFI", ipxe
pxe-service=BC_EFI, "Boot to FOG UEFI PXE-BC", ipxe

Note

On boot of a network client with that configuration the default pxelinux.0 config file is expected at TFTP_ROOT_DIR/pxelinux.cfg/default

2.2. insert the following content to use grub:

# The boot filename, Server name, Server Ip Address
dhcp-boot=boot/grub2/i386-pc/core.0,,BOOT_SERVER_IP

When using grub the referenced dhcp-boot grub module must be genereated. To do this change the directory to TFTP_ROOT_DIR and create the setvars.conf with the following content:

set root=(tftp)
set net_default_server=BOOT_SERVER_IP
set prefix=boot/grub2

Now call the following commands to create the grub module

$ grub2-mknetdir --net-directory=TFTP_ROOT_DIR --subdir=boot/grub2
$ grub2-mkimage -O i386-pc-pxe \
    --output boot/grub2/i386-pc/core.0 \
    --prefix=/boot/grub2 \
    -c setvars.conf \
  pxe tftp

Note

On boot of a network client with that configuration the grub config file is expected at TFTP_ROOT_DIR/boot/grub2/grub.cfg

• Run the dnsmasq server by calling:

  systemctl start dnsmasq

Abstract. This page explains how to build a file system image for use with KIWI NG’s PXE boot infrastructure. It contains:

PXE is a network boot protocol that is shipped with most BIOS implementations. The protocol sends a DHCP request to get an IP address. When an IP address is assigned, it uses the TFTP protocol to download a Kernel and boot instructions. Contrary to other images built with KIWI NG, a PXE image consists of separate boot, kernel and root filesystem images, since those images need to be made available in different locations on the PXE boot server.

A root filesystem image which can be deployed via KIWI NG’s PXE netboot infrastructure represents the system rootfs in a linux filesystem. A user could loop mount the image and access the contents of the root filesystem. The image does not contain any information about the system disk its partitions or the bootloader setup. All of these information is provided by a client configuration file on the PXE server which controlls how the root filesystem image should be deployed.

Many different deployment strategies are possible, e.g root over NBD (network block device), AoE (ATA over Ethernet), or NFS for diskless and diskfull clients. This particular example shows how to build an overlayfs-based union system based on openSUSE Leap for a diskless client which receives the squashfs compressed root file system image in a ramdisk overlayed via overlayfs and writes new data into another ramdisk on the same system. As diskless client, a QEMU virtual machine is used.

11.14.1 PXE Client Setup Configuration #

 

All PXE boot based deployment methods are controlled by configuration files located in /srv/tftpboot/KIWI on the PXE server. Such a configuration file can either be client-specific (config.MAC_ADDRESS, for example config.00.AB.F3.11.73.C8), or generic (config.default).

In an environment with heterogeneous clients, this allows to have a default configuration suitable for the majority of clients, to have configurations suitable for a group of clients (for example machines with similar or identical hardware), and individual configurations for selected machines.

The configuration file contains data about the image and about configuration, synchronization, and partition parameters. The configuration file has got the following general format:

IMAGE="device;name;version;srvip;bsize;compressed,...,"

DISK="device"
PART="size;id;Mount,...,size;id;Mount"
RAID="raid-level;device1;device2;..."

AOEROOT=ro-device[,rw-device]
NBDROOT="ip-address;export-name;device;swap-export-name;swap-device;write-export-name;write-device"
NFSROOT="ip-address;path"

UNIONFS_CONFIGURATION="rw-partition,compressed-partition,overlayfs"

CONF="src;dest;srvip;bsize;[hash],...,src;dest;srvip;bsize;[hash]"

KIWI_BOOT_TIMEOUT="seconds"
KIWI_KERNEL_OPTIONS="opt1 opt2 ..."

REBOOT_IMAGE=1
RELOAD_CONFIG=1
RELOAD_IMAGE=1

Note

Quoting the Values

The configuration file is sourced by Bash, so the same quoting rules as for Bash apply.

Not all configuration options needs to be specified. It depends on the setup of the client which configuration values are required. The following is a collection of client setup examples which covers all supported PXE client configurations.

11.14.1.1 Setup Client with Remote Root #

 

To serve the image from a remote location and redirect all write operations on a tmpfs, the following setup is required:

# When using AoE, see vblade toolchain for image export

AOEROOT=/dev/etherd/e0.1
UNIONFS_CONFIG=tmpfs,aoe,overlay

# When using NFS, see exports manual page for image export

NFSROOT="192.168.100.2;/srv/tftpboot/image/root"
UNIONFS_CONFIG=tmpfs,nfs,overlay

# When using NBD, see nbd-server manual page for image export

NBDROOT=192.168.100.2;root_export;/dev/nbd0
UNIONFS_CONFIG=tmpfs,nbd,overlay

The above setup shows the most common use case where the image built with KIWI NG is populated over the network using either AoE, NBD or NFS in combination with overlayfs which redirects all write operations to be local to the client. In any case a setup of either AoE, NBD or NFS on the image server is required beforehand.

11.14.1.2 Setup Client with System on Local Disk #

 

To deploy the image on a local disk the following setup is required:

Note

In the referenced x86/tumbleweed/test-image-pxe XML description the pxe type must be changed as follows and the image needs to be rebuild:

<type image="pxe" filesystem="ext3" boot="netboot/suse-tumbleweed"/>

IMAGE="/dev/sda2;kiwi-test-image-pxe.x86_64;1.15.3;192.168.100.2;4096"
DISK="/dev/sda"
PART="5;S;X,X;L;/"

The setup above will create a partition table on sda with a 5MB swap partition (no mountpoint) and the rest of the disk will be a Linux(L) partition with / as mountpoint. The (X) in the PART setup specifies a place holder to indicate the default behaviour.

11.14.1.3 Setup Client with System on Local MD RAID Disk #

 

To deploy the image on a local disk with prior software RAID configuration, the following setup is required:

Note

In the referenced x86/tumbleweed/test-image-pxe XML description the pxe type must be changed as follows and the image needs to be rebuild:

<type image="pxe" filesystem="ext3" boot="netboot/suse-tumbleweed"/>

RAID="1;/dev/sda;/dev/sdb"
IMAGE="/dev/md1;kiwi-test-image-pxe.x86_64;1.15.3;192.168.100.2;4096"
PART="5;S;x,x;L;/"

The first parameter of the RAID line is the RAID level. So far only raid1 (mirroring) is supported. The second and third parameter specifies the raid disk devices which make up the array. If a RAID line is present all partitions in PART will be created as RAID partitions. The first RAID is named md0, the second one md1 and so on. It is required to specify the correct RAID partition in the IMAGE line according to the PART setup. In this case md0 is reserved for the SWAP space and md1 is reserved for the system.

11.14.1.4 Setup Loading of Custom Configuration File(s) #

 

In order to load for example a custom /etc/hosts file on the client, the following setup is required:

CONF="hosts;/etc/hosts;192.168.1.2;4096;ffffffff"

On boot of the client KIWI NG’s boot code will fetch the hosts file from the root of the server (192.168.1.2) with 4k blocksize and deploy it as /etc/hosts on the client. The protocol is by default tftp but can be changed via the kiwiservertype kernel commandline option. For details, see Setup a Different Download Protocol and Server

11.14.1.5 Setup Client to Force Reload Image #

 

To force the reload of the system image even if the image on the disk is up-to-date, the following setup is required:

RELOAD_IMAGE=1

The option only applies to configurations with a DISK/PART setup

11.14.1.6 Setup Client to Force Reload Configuration Files #

 

To force the reload of all configuration files specified in CONF, the following setup is required:

RELOAD_CONFIG=1

By default only configuration files which has changed according to their md5sum value will be reloaded. With the above setup all files will be reloaded from the PXE server. The option only applies to configurations with a DISK/PART setup

11.14.1.7 Setup Client for Reboot After Deployment #

 

To reboot the system after the initial deployment process is done the following setup is required:

REBOOT_IMAGE=1

11.14.1.8 Setup custom kernel boot options #

 

To deactivate the kernel mode setting on local boot of the client the following setup is required:

KIWI_KERNEL_OPTIONS="nomodeset"

Note

This does not influence the kernel options passed to the client if it boots from the network. In order to setup those the PXE configuration on the PXE server needs to be changed

11.14.1.9 Setup a Custom Boot Timeout #

 

To setup a 10sec custom timeout for the local boot of the client the following setup is required.

KIWI_BOOT_TIMEOUT="10"

Note

This does not influence the boot timeout if the client boots off from the network.

11.14.1.10 Setup a Different Download Protocol and Server #

 

By default all downloads controlled by the KIWI NG linuxrc code are performed by an atftp call using the TFTP protocol. With PXE the download protocol is fixed and thus you cannot change the way how the kernel and the boot image (initrd) is downloaded. As soon as Linux takes over, the download protocols http, https and ftp are supported too. KIWI NG uses the curl program to support the additional protocols.

To select one of the additional download protocols the following kernel parameters need to be specified

kiwiserver=192.168.1.1 kiwiservertype=ftp

To set up this parameters edit the file /srv/tftpboot/pxelinux.cfg/default on your PXE boot server and change the append line accordingly.

Note

Once configured all downloads except for kernel and initrd are now controlled by the given server and protocol. You need to make sure that this server provides the same directory and file structure as initially provided by the kiwi-pxeboot package

Abstract. This page provides further information for handling KIS images built with KIWI NG and references the following article:

In KIWI NG, the kiwi-overlay dracut module can be used to boot from a remote exported root filesystem. The exported device is visible as block device on the network client. KIWI NG supports the two export backends NBD (Network Block Device) and AoE (ATA over Ethernet) for this purpose. A system that is booted in this mode will read the contents of the root filesystem from a remote location and targets any write action into RAM by default. The kernel cmdline option rd.root.overlay.write can be used to specify an alternative device to use for writing. The two layers (read and write) are combined using the overlayfs filesystem.

For remote boot of a network client, the PXE boot protocol is used. This functionality requires a network boot server setup on the system. Details how to setup such a server can be found in Section 11.13, “Setting Up a Network Boot Server”.

Before the KIS image can be build, the following configuration step is required:

Now the KIS image can be build as shown in Section 10.6, “Build KIS Image (Kernel, Initrd, System)”. After the build, the following configuration steps are required to boot from the network:

Abstract. This page provides further information for handling ISO images built with KIWI NG and references the following articles:

In KIWI NG, live ISO images can be configured to boot via the PXE boot protocol. This functionality requires a network boot server setup on the system. Details how to setup such a server can be found in Section 11.13, “Setting Up a Network Boot Server”.

After the live ISO was built as shown in Section 10.1, “Build an ISO Hybrid Live Image”, the following configuration steps are required to boot from the network:

Abstract. This page provides information how to setup the KIWI NG XML description to start the SUSE YaST system setup utility at first boot of the image

To be able to use YaST in a non interactive way, create a YaST profile which tells the autoyast module what to do. To create the profile, run:

yast autoyast

Once the YaST profile exists, update the KIWI NG XML description as follows:

Abstract. This page provides further information for customizing the /etc/fstab file which controls the mounting of filesystems at boot time.

In KIWI NG, all major filesystems that were created at image build time are handled by KIWI NG itself and setup in /etc/fstab. Thus there is usually no need to add entries or change the ones added by KIWI NG. However depending on where the image is deployed later it might be required to pre-populate fstab entries that are unknown at the time the image is build.

Possible use cases are for example:

Note

Modifications to the fstab file are a critical change. If done wrongly the risk exists that the system will not boot. In addition this type of modification makes the image specific to its target and creates a dependency to the target hardware, network, etc… Thus this feature should be used with care.

The optimal way to provide custom fstab information is through a package. If this can’t be done the files can also be provided via the overlay file tree of the image description.

KIWI NG supports three ways to modify the contents of the /etc/fstab file:

Note

All three variants to handle the fstab file can be used together. Appending happens first, patching afterwards and the script call is last. When using the script call, there is no validation that checks if the script actually handles fstab or any other file in the image rootfs.

KIWI NG supports so-called profiles inside the XML image description. Profiles act as namespaces for additional settings to be applied on top of the defaults. For further details, see Section 7.4, “Image Profiles”.

$ sudo kiwi-ng --type oem --profile libvirt system build \
      --description kiwi/build-tests/x86/leap/test-image-vagrant \
      --set-repo obs://openSUSE:Leap:15.5/standard \
      --target-dir /tmp/myimage

$ osc build -M $PROFILE

Note

Abstract

This document gives a brief overview how to build images with KIWI NG in version 10.2.2 inside of the Open Build Service. A tutorial on the Open Buildservice itself can be found here: https://en.opensuse.org/openSUSE:Build_Service_Tutorial

The next generation KIWI NG is fully integrated with the Open Build Service. In order to start it’s best to checkout one of the integration test image build projects from the base Testing project Virtualization:Appliances:Images:Testing_$ARCH:$DISTRO at:

https://build.opensuse.org

For example the test images for SUSE on x86 can be found here.

11.20.3 Recommendations #

 

11.20.3.1 Working with OBS #

 

Although OBS is an online service, it is not necessary to test every change by uploading it. OBS will use the same process as osc build does, so if your image builds locally via osc build it should also build online on OBS.

11.20.3.2 Repository Configuration #

 

When setting up the project, enable the images repository: the images repository’s checkbox can be found at the bottom of the selection screen that appears when clicking Add from a Distribution in the Repositories tab. Or specify it manually in the project configuration (it can be accessed via osc meta -e prj):

<repository name="images">
  <arch>x86_64</arch>
</repository>

Furthermore, OBS requires additional repositories from which it obtains your dependent packages. These repositories can be provided in two ways:

1. Add the repositories to the project configuration on OBS and omit them from config.xml. Provide only the following repository inside the image description:

   <repository type="rpm-md">
     <source path="obsrepositories:/"/>
   </repository>

   This instructs OBS to inject the repositories from your project into your appliance.

   Additional repositories can be added by invoking osc meta -e prj and adding a line of the following form as a child of <repository name="images">:

   <path project="$OBS_PROJECT" repository="$REPOSITORY_NAME"/>

   The order in which you add repositories matters: if a package is present in multiple repositories, then it is taken from the first repository. The last repository is subject to path expansion: its repository paths are included as well.

   Don’t forget to add the repository from the Virtualization:Appliances:Builder project, providing the latest stable version of KIWI NG (which you are very likely using for your local builds).

   The following example repository configuration adds the repositories from the Virtualization:Appliances:Builder project and those from the latest snapshot of openSUSE Tumbleweed:

   <project name="Virtualization:Appliances:Images:openSUSE-Tumbleweed">
     <title>Tumbleweed JeOS images</title>
     <description>Host JeOS images for Tumbleweed</description>
     <repository name="images">
       <path project="Virtualization:Appliances:Builder" repository="Factory"/>
       <path project="openSUSE:Factory" repository="snapshot"/>
       <arch>x86_64</arch>
     </repository>
   </project>

   The above can be simplified further using the path expansion of the last repository to:

   <project name="Virtualization:Appliances:Images:openSUSE-Tumbleweed">
     <title>Tumbleweed JeOS images</title>
     <description>Host JeOS images for Tumbleweed</description>
     <repository name="images">
       <path project="Virtualization:Appliances:Builder" repository="Factory"/>
       <arch>x86_64</arch>
     </repository>
   </project>

   Now Virtualization:Appliances:Builder is the last repository, which’ repositories are included into the search path. As openSUSE:Factory/snapshot is among these, it can be omitted from the repository list.

2. Keep the repositories in your config.xml configuration file. If you have installed the latest stable KIWI NG as described in Chapter 2, Installation then you should add the following repository to your projects configuration (accessible via osc meta -e prjconf), so that OBS will pick the latest stable KIWI NG version too:

   <repository name="images">
     <path project="Virtualization:Appliances:Builder" repository="$DISTRO"/>
     <arch>x86_64</arch>
   </repository>

   Replace $DISTRO with the appropriate name for the distribution that you are currently building and optionally adjust the architecture.

We recommend to use the first method, as it integrates better into OBS. Note that your image description will then no longer build outside of OBS though. If building locally is required, use the second method.

Warning

Adding the repositories to project’s configuration makes it impossible to build images for different distributions from the same project.

Since the repositories are added for every package in your project, all your image builds will share the same repositories, thereby resulting in conflicts for different distributions.

We recommend to create a separate project for each distribution. If that is impossible, you can keep all your repositories (including Virtualization:Appliances:Builder) in config.xml. That however usually requires a large number of workarounds via Prefer: settings in the project configuration and is thus not recommended.

11.20.3.3 Project Configuration #

 

The Open Build Service will by default create the same output file as KIWI NG when run locally, but with a custom filename ending (that is unfortunately unpredictable). This has the consequence that the download URL of your image will change with every rebuild (and thus break automated scripts). OBS can create symbolic links with static names to the latest build by adding the following line to the project configuration:

Repotype: staticlinks

If build Vagrant images (see Section 11.7, “Image Description for Vagrant”) add the repository-type vagrant. OBS creates a boxes/ subdirectory in your download repositories, which contains JSON files for Vagrant .

If you have added your repositories to config.xml, you probably see errors of the following type:

unresolvable: have choice for SOMEPACKAGE: SOMEPAKAGE_1 SOMEPACKAGE_2

Instead of starting from scratch and manually adding Prefer: statements to the project configuration, we recommend to copy the current project configuration of the testing project Virtualization:Appliances:Images:Testing_$ARCH:$DISTRO into your own project. It provides a good starting point and can be adapted to your OBS project.

Abstract. This page provides information how to use the SUSE media ISO with KIWI NG

When building an image with KIWI NG, the image description usually points to a number of public/private package source repositories from which the new image root tree will be created. Alternatively the vendor provided product ISO image(s) can be used. The contents of the ISO (DVD) media also provides package source repositories but organized in a vendor specific structure. As a user it’s important to know about this structure such that the KIWI NG image description can consume it.

To use a SUSE product media the following steps are required:

$ sudo mount Product_ISO_file.iso|DVD_drive /media/suse

Once all the individual product and module repos has been created in the KIWI NG image description, the build process can be started as usual.

Note

Because of the manual mount process the /media/suse location stays busy after KIWI NG has created the image. The cleanup of this resource is a responsibility of the user and not done by KIWI NG

Abstract. This page provides information how to build Debian based images without an extra bootstrap process.

When building Debian based images KIWI NG uses apt in the bootstrap and the system phase to create the image root tree. However, apt does not support a native way to bootstrap an empty root tree. Therefore the bootstrap phase uses apt only to resolve the given bootstrap packages and to download these packages from the given repositories. The list of packages is then manually extracted into the new root tree which is not exactly the same as if apt would have installed them natively. For the purpose of creating an initial tree to begin with, this procedure is acceptable though.

If, for some reasons, this bootstrap procedure is not applicable, KIWI NG allows for an alternative process which is based on a prebuilt bootstrap-root archive provided as a package.

To make use of a bootstrap_package, the name of that package needs to be referenced in the KIWI NG description as follows:

<packages type="bootstrap" bootstrap_package="bootstrap-root">
    <package name="a"/>
    <package name="b"/>
</packages>

The boostrap process now changes in a way that the provided bootstrap_package bootstrap-root will be installed on the build host machine. Next KIWI NG searches for a tar archive file /var/lib/bootstrap/bootstrap-root.ARCH.tar.xz, where ARCH is the name of the host architecture e.g x86_64. If found the archive gets unpacked and serves as the bootstrap root tree to begin with. The optionally provided additional bootstrap packages, a and b in this example will be installed like system packages via chroot and apt. Usually no additional bootstrap packages are needed as they could all be handled as system packages.