Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / Building Linux System Appliances / Troubleshooting

5 Troubleshooting

 
Note
Note

Abstract

This document describes situations which leads to issues during build or boot time of the image build with KIWI NG. The suggested solutions are considered best practice but are just one out of other possible solution candidates.

5.1 Build Host Constraints

 

For building images a host system is required that runs the build process. Tools to create the image are used from that host and this creates an indirect dependency to the target image. For example; Building an Ubuntu image requires the apt and dpkg tools and metadata to be available and functional on the host to build an Ubuntu image. There are many more of those host vs. image dependencies and not all of them can be resolved in a clear and clean way.

The most compatible environment is provided if the build host is of the same distribution than the target image. In other cases our recommendation is that the build host is of the same distribution than the target and near to the major version (+-1) compared to the target. Such an environment can be found in:

In general, our goal is to support any major distribution with KIWI NG. However for building images we rely on core tools which are not under our control. Also several design aspects of distributions like secure boot and working with upstream projects are different and not influenced by us. There are many side effects that can be annoying especially if the build host is not of the same distribution vendor than the image target.

5.2 Architectures

 

With regards to the information in Section 5.1, “Build Host Constraints” one requirement between the build host and the image when it comes to architecture support is, that the image architecture should match the build host architecture. Cross arch building would require any core tool that is used to build an image to be cross arch capable.

To patch e.g an x86_64 system such that it can build an aarch64 image would require some work on binutils and hacks as well as performance tweaks which is all not worth the effort and still can lead to broken results. Thus we recommend to provide native systems for the target architecture and build there. One possible alternative is to use the kiwi boxed plugin as mentioned above together with a box created for the desired architecture. However keep in mind the performance problematic when running a VM of a different architecture.

The majority of the image builds are based on the x86 architecture. As mentioned KIWI NG also supports other architectures, shown in the table below:

Architecture

Supported

x86_64

yes

ix86

yes note:distro

s390/s390x

yes note:distro

arm/aarch64

yes note:distro

ppc64

yes note:distro

5.3 Host Security Settings Conflicts with KIWI

 
Note
Note

Abstract

This page provides further information how to solve image build problems caused by selinux security policies.

Linux systems are protected against write/read or other operations depending on the application which wants to access or modify data. The rules for this protection are provided in security policies. There are several applications enforcing these security settings, e.g apparmor or selinux. In this troubleshooting chapter the focus is set on selinux

Protecting files, process groups, kernel filesystems, device nodes and more from unauthorized access and restrict it to a certain set of applications is a nice concept. However, if taken serious no other application except the ones configured in the security policy will function properly.

When building an appliance, the appliance builder has to have access to a wide range of services. It must be able to create a new package database elsewhere in the system. It must be able to create, read and write device nodes, create filesystems, partitions, bootloader configurations etc etc. The list is very long and no security policy could cover this in a way that it would not be open to everything which in the end leads to a pointless exercise and no security at all.

This means for users who would like to keep the security settings of the system enforced and unchanged, the only way to allow KIWI NG to do its job is to run it through boxbuild as explained in Section 6.1, “Building in a Self-Contained Environment”

For users who can afford to open the system security policy, the following procedure will make KIWI NG to work:

sudo setenforce 0

This action disables selinux temporary. To disable selinux permanently perform the following steps:

  1. Open the SELinux configuration file: /etc/selinux/config

  2. Locate the following line: SELINUX=enforcing

  3. Change the value to disabled:

    SELINUX=disabled

  4. On the next reboot, SELinux is permanently disabled.

Note
Note

similar instructions applies to other application security subsystems like apparmor. Due to the complexity of these systems this article just mentions the most common issue people run into when building images on systems protected through selinux.

5.4 Incompatible Filesystem Settings on Host vs. Image

 
Note
Note

Abstract

This page provides further information how to solve image boot problems if the filesystem tool chain on the image build host is incompatible with the image target distribution

When KIWI NG builds an image which requests the creation of a filesystem, the required filesystem creation tool, for example mkfs.xfs, is called from the host on which KIWI NG gets called. It is expected that the generated filesystem is compatible with the image target distribution. This expectation is not always correct and depends on the compatibility of the filesystem default settings between build host and image target. We know about the following settings that causes an incompatible filesystem which will not be able to be used on boot:

Ext[2,3,4]

Check /etc/mke2fs.conf on the build host and make sure the configured inode_size is the same as the setting used for the target image. To solve an issue of this type use the following filesystem creation option in your KIWI NG image configuration:

<type fscreateoptions="-I inode-size"/>
XFS

Check the XFS metadata setup on the build host and make sure the settings are compatible with the target image. XFS has the default settings compiled in, thus it might be needed to build the image first and use the xfs_info tool in a disk.sh script to fetch the settings at build time of the image. We know from community reports that the setting sparse=1 will cause issues on older versions of grub’s xfs module, which does not know how to handle this setting properly. To solve an issue of this type use the following filesystem creation option in your KIWI NG image configuration:

<type fscreateoptions="-i sparse=0"/>
Note
Note

There can be more inconsistencies in the area of filesystems which we haven’t listed here. In general it’s advisable to build the image in a compatible environment. At best the build host distribution is of the same major Linux version than the image target. For this purpose KIWI NG provides the so called boxed-plugin. Further details can be found in Section 6.1, “Building in a Self-Contained Environment”

5.5 Boxbuild Tweaks

 
Note
Note

Abstract

This document describes a few ways to modify box build VMs for testing/debugging.

5.5.1 Increase Box Build Image Size

 

In particularly large builds, you may find that the upstream build boxes aren’t quite large enough, and fail to build during the final few steps. While it is a bit of a kludge, it is possible to increase the size of the build box.

To do so, follow these steps:

Note
Note

For this example, we will assume the box increase question is an Ubuntu box, located in ~/.kiwi_boxes/ubuntu

  • While the VM is offline, locate the VM you want to modify and resize with qemu-img. Here we will increase the size by 20G. The VM will have to be told to utilize this space in the following steps.

$ qemu-img resize Ubuntu-Box.x86_64-1.22.04-System-BuildBox.qcow2 +20G

  • When relaunching your kiwi-ng box build, make sure you use --no-snapshot and --box-debug options within your build command/script. Example:

$ kiwi --debug --profile="Disk" --type oem system boxbuild --no-snapshot \
--box-memory=32G --box-smp-cpus=16 --box-debug --box ubuntu -- \
--description ./ubuntu-jammy --target-dir /build/kiwi/outputs/

  • When the build fails and drops you into the VM console, you will need to extend the partition of the VM rootfs, then resize with resize2fs. In this example, parted was used and the partition in question was /dev/vda3.

$ parted
# Can run parted print to check for relevant partitions if needed.
(parted) $ print
(parted) $ resizepart 3 100%
# Exit from parted
(parted) $ quit
# Run resize2fs to grow the filesystem to fill the space
$ resize2fs /dev/vda3

  • From this point, depending on where your build failed, it may be possible to continue your build from inside the box, using the existing 9p mount points defined by your build command. Using the command above as an example, /result within the box maps up to /build/kiwi/outputs on the host, and it’s possible to run

$ kiwi-ng --profile="Disk" --type oem  system create \
--root=/result/build/image-root/ --target-dir=/result

  • If the rebuild from within was successful, you can copy the files from /result to /bundle, from within the VM, where /bundle maps to your target-dir on the host.