3 Standalone clusters with Edge Image Builder #
Edge Image Builder (EIB) is a tool that streamlines the process of generating Customized, Ready-to-Boot (CRB) disk images for bootstrapping machines, even in fully air-gapped scenarios. EIB is used to create deployment images for use in all three of the SUSE Edge deployment footprints, as it’s flexible enough to offer the smallest customizations, e.g. adding a user or setting the timezone, through offering a comprehensively configured image that sets up, for example, complex networking configurations, deploys multi-node Kubernetes clusters, deploys customer workloads, and registers to the centralized management platform via Rancher/Elemental and SUSE Manager. EIB runs as in a container image, making it incredibly portable across platforms and ensuring that all of the required dependencies are self-contained, having a very minimal impact on the installed packages of the system that’s being used to operate the tool.
For more information, read the Edge Image Builder Introduction (Chapter 9, Edge Image Builder).
3.1 Prerequisites #
An x86_64 physical host (or virtual machine) running SLES 15 SP5, openSUSE Leap 15.5, or openSUSE Tumbleweed.
An available container runtime (e.g. Podman)
A downloaded copy of the latest SLE Micro 5.5 SelfInstall "GM2" ISO image found here.
Other operating systems may function so long as a compatible container runtime is available, but testing on other platforms has not been extensive. The documentation focuses on Podman, but the same functionality should be able to be achieved with Docker.
3.1.1 Getting the EIB Image #
The EIB container image is publicly available and can be downloaded from the SUSE Edge registry by running the following command on your image build host:
podman pull registry.suse.com/edge/edge-image-builder:1.0.2
3.2 Creating the image configuration directory #
As EIB runs within a container, we need to mount a configuration directory from the host, enabling you to specify your desired configuration, and during the build process EIB has access to any required input files and supporting artifacts. This directory must follow a specific structure. Let’s create it, assuming that this directory will exist in your home directory, and called "eib":
export CONFIG_DIR=$HOME/eib mkdir -p $CONFIG_DIR/base-images
In the previous step we created a "base-images" directory that will host the SLE Micro 5.5 input image, let’s ensure that the downloaded image is copied over to the configuration directory:
cp /path/to/downloads/SLE-Micro.x86_64-5.5.0-Default-SelfInstall-GM2.install.iso $CONFIG_DIR/base-images/slemicro.iso
During the EIB run, the original base image is not modified; a new and customized version is created with the desired configuration in the root of the EIB config directory.
The configuration directory at this point should look like the following:
└── base-images/ └── slemicro.iso
3.3 Creating the image definition file #
The definition file describes the majority of configurable options that the Edge Image Builder supports, a full example of options can be found here, and we would recommend that you take a look at the upstream building images guide for more comprehensive examples than the one we’re going to run through below. Let’s start with a very basic definition file for our OS image:
cat << EOF > $CONFIG_DIR/iso-definition.yaml apiVersion: 1.0 image: imageType: iso arch: x86_64 baseImage: slemicro.iso outputImageName: eib-image.iso EOF
This definition specifies that we’re generating an output image for an x86_64
based system. The image that will be used as the base for further modification is an iso
image named slemicro.iso
,
expected to be located at $CONFIG_DIR/base-images/slemicro.iso
. It also outlines that after EIB finishes modifying the image, the output image will be named eib-image.iso
, and by default will reside in $CONFIG_DIR
.
Now our directory structure should look like:
├── iso-definition.yaml └── base-images/ └── slemicro.iso
In the following sections we’ll walk through a few examples of common operations:
3.3.1 Configuring OS Users #
EIB allows you to preconfigure users with login information, such as passwords or SSH keys, including setting a fixed root password. As part of this example we’re going to fix the root password, and the first step is to use OpenSSL
to create a one-way encrypted password:
openssl passwd -6 SecurePassword
This will output something similar to:
$6$G392FCbxVgnLaFw1$Ujt00mdpJ3tDHxEg1snBU3GjujQf6f8kvopu7jiCBIhRbRvMmKUqwcmXAKggaSSKeUUOEtCP3ZUoZQY7zTXnC1
We can then add a section in the definition file called operatingSystem
with a users
array inside it. The resulting file should look like:
apiVersion: 1.0
image:
imageType: iso
arch: x86_64
baseImage: slemicro.iso
outputImageName: eib-image.iso
operatingSystem:
users:
- username: root
encryptedPassword: $6$G392FCbxVgnLaFw1$Ujt00mdpJ3tDHxEg1snBU3GjujQf6f8kvopu7jiCBIhRbRvMmKUqwcmXAKggaSSKeUUOEtCP3ZUoZQY7zTXnC1
It’s also possible to add additional users, create the home directories, set user-id’s, add ssh-key authentication, and modify group information. Please refer to the upstream building images guide for further examples.
3.3.2 Configuring RPM packages #
One of the major features of EIB is to provide a mechanism to add additional software packages to the image, so when the installation completes the system is able to leverage the installed packages right away. EIB permits users to specify the following:
Packages by their name within a list in the image definition
Network repositories to search for these packages in
SUSE Customer Center (SCC) credentials to search official SUSE repositories for the listed packages
Via an
$CONFIG_DIR/rpms
directory, side-load custom RPM’s that don’t exist in network repositoriesVia the same directory (
$CONFIG_DIR/rpms/gpg-keys
), GPG-keys to enable validation of third party packages
EIB will then run through a package resolution process at image build time, taking the base image as the input, and attempts to pull and install all supplied packages, either specified via the list or provided locally. EIB downloads all of the packages, including any dependencies into a repository that exists within the output image and instructs the system to install these during the first boot process. Doing this process during the image build guarantees that the packages will successfully install during first-boot on the desired platform, e.g. the node at the edge. This is also advantageous in environments where you want to bake the additional packages into the image rather than pull them over the network when in operation, e.g. for air-gapped or restricted network environments.
As a simple example to demonstrate this, we are going to install the nvidia-container-toolkit
RPM package found in the third party vendor-supported NVIDIA repository:
packages:
packageList:
- nvidia-container-toolkit
additionalRepos:
- url: https://nvidia.github.io/libnvidia-container/stable/rpm/x86_64
The resulting definition file looks like:
apiVersion: 1.0
image:
imageType: iso
arch: x86_64
baseImage: slemicro.iso
outputImageName: eib-image.iso
operatingSystem:
users:
- username: root
encryptedPassword: $6$G392FCbxVgnLaFw1$Ujt00mdpJ3tDHxEg1snBU3GjujQf6f8kvopu7jiCBIhRbRvMmKUqwcmXAKggaSSKeUUOEtCP3ZUoZQY7zTXnC1
packages:
packageList:
- nvidia-container-toolkit
additionalRepos:
- url: https://nvidia.github.io/libnvidia-container/stable/rpm/x86_64
The above is a simple example, but for completeness, download the NVIDIA package signing key before running the image generation:
$ mkdir -p $CONFIG_DIR/rpms/gpg-keys
$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey > rpms/gpg-keys/nvidia.gpg
Adding in additional RPM’s via this method is meant for the addition of supported third party components or user-supplied (and maintained) packages; this mechanism should not be used to add packages that would not usually be supported on SLE Micro. If this mechanism is used to add components from openSUSE repositories (which are not supported), including from newer releases or service packs, you may end up with an unsupported configuration, especially when dependency resolution results in core parts of the operating system being replaced, even though the resulting system may appear to function as expected. If you’re unsure, contact your SUSE representative for assistance in determining the supportability of your desired configuration.
A more comprehensive guide with additional examples can be found in the upstream installing packages guide.
3.3.3 Configuring Kubernetes cluster and user workloads #
Another feature of EIB is the ability to use it to automate the deployment of both single-node and multi-node highly-available Kubernetes clusters that "bootstrap in place", i.e. don’t require any form of centralized management infrastructure to coordinate. The primary driver behind this approach is for air-gapped deployments, or network restricted environments, but it also serves as a way of quickly bootstrapping standalone clusters, even if full and unrestricted network access is available.
This method enables not only the deployment of the customized operating system, but also the ability to specify Kubernetes configuration, any additional layered components via Helm charts, and any user workloads via supplied Kubernetes manifests. However, the design principle behind using this method is that we default to assuming that the user is wanting to air-gap and therefore any items specified in the image definition will be pulled into the image, which includes user-supplied workloads, where EIB will make sure that any discovered images that are required by definitions supplied are copied locally, and are served by the embedded image registry in the resulting deployed system.
In this next example, we’re going to take our existing image definition and will specify a Kubernetes configuration (in this example it doesn’t list the systems and their roles, so we default to assuming single-node), which will instruct EIB to provision a single-node RKE2 Kubernetes cluster. To show the automation of both the deployment of both user-supplied workloads (via manifest) and layered components (via Helm), we are going to install KubeVirt via the SUSE Edge Helm chart, as well as NGINX via a Kubernetes manifest. The additional configuration we need to append to the existing image definition is as follows:
kubernetes:
version: v1.28.10+rke2r1
manifests:
urls:
- https://k8s.io/examples/application/nginx-app.yaml
helm:
charts:
- name: kubevirt-chart
version: 0.2.4
repositoryName: suse-edge
repositories:
- name: suse-edge
url: oci://registry.suse.com/edge
The resulting full definition file should now look like:
apiVersion: 1.0
image:
imageType: iso
arch: x86_64
baseImage: slemicro.iso
outputImageName: eib-image.iso
operatingSystem:
users:
- username: root
encryptedPassword: $6$G392FCbxVgnLaFw1$Ujt00mdpJ3tDHxEg1snBU3GjujQf6f8kvopu7jiCBIhRbRvMmKUqwcmXAKggaSSKeUUOEtCP3ZUoZQY7zTXnC1
packages:
packageList:
- nvidia-container-toolkit
additionalRepos:
- url: https://nvidia.github.io/libnvidia-container/stable/rpm/x86_64
kubernetes:
version: v1.28.10+rke2r1
manifests:
urls:
- https://k8s.io/examples/application/nginx-app.yaml
helm:
charts:
- name: kubevirt-chart
version: 0.2.4
repositoryName: suse-edge
repositories:
- name: suse-edge
url: oci://registry.suse.com/edge
Further examples of options such as multi-node deployments, custom networking, and Helm chart options/values can be found in the upstream documentation.
3.3.4 Configuring the network #
In the last example in this quickstart, let’s configure the network that will be brought up when a system is provisioned with the image generated by EIB. It’s important to understand that unless a network configuration is supplied, the default model is that DHCP will be used on all interfaces discovered at boot time. However, this is not always a desirable configuration, especially if DHCP is not available and you need to provide static configurations, or you need to set up more complex networking constructs, e.g. bonds, LACP, and VLAN’s, or need to override certain parameters, e.g. hostnames, DNS servers, and routes.
EIB provides the ability to provide either per-node configurations (where the system in question is uniquely identified by its MAC address), or an override for supplying an identical configuration to each machine, which is more useful when the system MAC addresses aren’t known. An additional tool is used by EIB called Network Manager Configurator, or nmc
for short, which is a tool built by the SUSE Edge team to allow custom networking configurations to be applied based on the nmstate.io declarative network schema, and at boot time will identify the node it’s booting on and will apply the desired network configuration prior to any services coming up.
We’ll now apply a static network configuration for a system with a single interface by describing the desired network state in a node-specific file (based on the desired hostname) in the required network
directory:
mkdir $CONFIG_DIR/network cat << EOF > $CONFIG_DIR/network/host1.local.yaml routes: config: - destination: 0.0.0.0/0 metric: 100 next-hop-address: 192.168.122.1 next-hop-interface: eth0 table-id: 254 - destination: 192.168.122.0/24 metric: 100 next-hop-address: next-hop-interface: eth0 table-id: 254 dns-resolver: config: server: - 192.168.122.1 - 8.8.8.8 interfaces: - name: eth0 type: ethernet state: up mac-address: 34:8A:B1:4B:16:E7 ipv4: address: - ip: 192.168.122.50 prefix-length: 24 dhcp: false enabled: true ipv6: enabled: false EOF
The above example is set up for the default 192.168.122.0/24
subnet assuming that testing is being executed on a virtual machine, please adapt to suit your environment, not forgetting the MAC address. As the same image can be used to provision multiple nodes, networking configured by EIB (via nmc
) is dependent on it being able to uniquely identify the node by its MAC address, and hence during boot nmc
will apply the correct networking configuration to each machine. This means that you’ll need to know the MAC addresses of the systems you want to install onto. Alternatively, the default behavior is to rely on DHCP, but you can utilize the configure-network.sh
hook to apply a common configuration to all nodes - see the networking guide (Chapter 10, Edge Networking) for further details.
The resulting file structure should look like:
├── iso-definition.yaml ├── base-images/ │ └── slemicro.iso └── network/ └── host1.local.yaml
The network configuration we just created will be parsed and the necessary NetworkManager connection files will be automatically generated and inserted into the new installation image that EIB will create. These files will be applied during the provisioning of the host, resulting in a complete network configuration.
Please refer to the Edge Networking component (Chapter 10, Edge Networking) for a more comprehensive explanation of the above configuration and examples of this feature.
3.4 Building the image #
Now that we’ve got a base image and an image definition for EIB to consume, let’s go ahead and build the image. For this, we simply use podman
to call the EIB container with the "build" command, specifying the definition file:
podman run --rm -it --privileged -v $CONFIG_DIR:/eib \
registry.suse.com/edge/edge-image-builder:1.0.2 \
build --definition-file iso-definition.yaml
The output of the command should be similar to:
Setting up Podman API listener... Generating image customization components... Identifier ................... [SUCCESS] Custom Files ................. [SKIPPED] Time ......................... [SKIPPED] Network ...................... [SUCCESS] Groups ....................... [SKIPPED] Users ........................ [SUCCESS] Proxy ........................ [SKIPPED] Resolving package dependencies... Rpm .......................... [SUCCESS] Systemd ...................... [SKIPPED] Elemental .................... [SKIPPED] Suma ......................... [SKIPPED] Downloading file: dl-manifest-1.yaml 100% (498/498 B, 5.9 MB/s) Populating Embedded Artifact Registry... 100% (3/3, 11 it/min) Embedded Artifact Registry ... [SUCCESS] Keymap ....................... [SUCCESS] Configuring Kubernetes component... The Kubernetes CNI is not explicitly set, defaulting to 'cilium'. Downloading file: rke2_installer.sh Downloading file: rke2-images-core.linux-amd64.tar.zst 100% (782/782 MB, 98 MB/s) Downloading file: rke2-images-cilium.linux-amd64.tar.zst 100% (367/367 MB, 100 MB/s) Downloading file: rke2.linux-amd64.tar.gz 100% (34/34 MB, 101 MB/s) Downloading file: sha256sum-amd64.txt 100% (3.9/3.9 kB, 1.5 MB/s) Downloading file: dl-manifest-1.yaml 100% (498/498 B, 7.2 MB/s) Kubernetes ................... [SUCCESS] Certificates ................. [SKIPPED] Building ISO image... Kernel Params ................ [SKIPPED] Image build complete!
The built ISO image is stored at $CONFIG_DIR/eib-image.iso
:
├── iso-definition.yaml ├── eib-image.iso ├── _build │ └── cache/ │ └── ... │ └── build-<timestamp>/ │ └── ... ├── base-images/ │ └── slemicro.iso └── network/ └── host1.local.yaml
Each build creates a time-stamped folder in $CONFIG_DIR/_build/
that includes the logs of the build, the artifacts used during the build,
and the combustion
and artefacts
directories which contain all the scripts and artifacts that are added to the CRB image.
The contents of this directory should look like:
├── build-<timestamp>/ │ │── combustion/ │ │ ├── 05-configure-network.sh │ │ ├── 10-rpm-install.sh │ │ ├── 12-keymap-setup.sh │ │ ├── 13b-add-users.sh │ │ ├── 20-k8s-install.sh │ │ ├── 26-embedded-registry.sh │ │ ├── 48-message.sh │ │ ├── network/ │ │ │ ├── host1.local/ │ │ │ │ └── eth0.nmconnection │ │ │ └── host_config.yaml │ │ ├── nmc │ │ └── script │ │── artefacts/ │ │ │── registry/ │ │ │ ├── hauler │ │ │ ├── nginx:1.14.2-registry.tar.zst │ │ │ ├── rancher_kubectl:v1.28.7-registry.tar.zst │ │ │ └── registry.suse.com_suse_sles_15.5_virt-operator:1.1.1-150500.8.12.1-registry.tar.zst │ │ │── rpms/ │ │ │ └── rpm-repo │ │ │ ├── addrepo0 │ │ │ │ └── x86_64 │ │ │ │ ├── nvidia-container-toolkit-1.15.0-1.x86_64.rpm │ │ │ │ ├── ... │ │ │ ├── repodata │ │ │ │ ├── ... │ │ │ └── zypper-success │ │ └── kubernetes/ │ │ ├── rke2_installer.sh │ │ ├── registries.yaml │ │ ├── server.yaml │ │ ├── images/ │ │ │ ├── rke2-images-cilium.linux-amd64.tar.zst │ │ │ └── rke2-images-core.linux-amd64.tar.zst │ │ ├── install/ │ │ │ ├── rke2.linux-amd64.tar.gz │ │ │ └── sha256sum-amd64.txt │ │ └── manifests/ │ │ ├── dl-manifest-1.yaml │ │ └── kubevirt-chart.yaml │ ├── createrepo.log │ ├── eib-build.log │ ├── embedded-registry.log │ ├── helm │ │ └── kubevirt-chart │ │ └── kubevirt-0.2.4.tgz │ ├── helm-pull.log │ ├── helm-template.log │ ├── iso-build.log │ ├── iso-build.sh │ ├── iso-extract │ │ └── ... │ ├── iso-extract.log │ ├── iso-extract.sh │ ├── modify-raw-image.sh │ ├── network-config.log │ ├── podman-image-build.log │ ├── podman-system-service.log │ ├── prepare-resolver-base-tarball-image.log │ ├── prepare-resolver-base-tarball-image.sh │ ├── raw-build.log │ ├── raw-extract │ │ └── ... │ └── resolver-image-build │ └──... └── cache └── ...
If the build fails, eib-build.log
is the first log that contains information. From there, it will direct you to the component that failed for debugging.
At this point, you should have a ready-to-use image that will:
Deploy SLE Micro 5.5
Configure the root password
Install the
nvidia-container-toolkit
packageConfigure an embedded container registry to serve content locally
Install single-node RKE2
Configure static networking
Install KubeVirt
Deploy a user-supplied manifest
3.5 Debugging the image build process #
If the image build process fails, refer to the upstream debugging guide.
3.6 Testing your newly built image #
For instructions on how to test the newly built CRB image, refer to the upstream image testing guide.