Jump to content

Keg Reference Guide

Publication Date: 07/20/2021

Preface

Note
Note

Template-Based KIWI Description Builder

1 Overview

Note
Note

Abstract

This document provides a conceptual overview about the steps of creating an image description with keg which can be used to build an appliance with the KIWI appliance builder.

1.1 Conceptual Overview

Keg is a tool which helps to create and manage image descriptions suitable for the KIWI appliance builder. While keg can be used to manage a single image definition the tool provides no considerable advantage in such a use case. The primary use case for keg are situations where many image descriptions must be managed and the image descriptions have considerable overlap with respect to content and setup.

The key component for keg is a data structure called recipes. This data structure is expected to contain all information necessary to create KIWI image descriptions. keg is implemented such that data inheritance is possible to reduce data duplication in the recipes.

The recipes consist of three major components:

Data Building Blocks: data

Independent collection of components used in KIWI image descriptions. This includes for example information about packages, repositories or custom script code and more. A building block should be created to represent a certain functionality or to provide a capability for a certain target distribution such that it can be used in a variety of different image descriptions.

Image Definitions: images

Formal instructions which building blocks should be used for the specified image

Schema Templates: schemas

Templates to implement Syntax and Semantic of image description files as required by KIWI

The setup of the recipes is the most time consuming part when using Keg. Example definitions for the recipes can be found here: Public Cloud Keg Recipes

For more details on how keg processes recipes data, see section Chapter 4, Recipes Basics (ff).

1.2 Working With Keg

Using keg is a two step process:

  1. Fetch or create an image definition tree

  2. Call the keg commandline utility to create a KIWI image description

For the above to work, Keg needs to be installed as described in Chapter 2, Installation. In addition install KIWI, see https://osinside.github.io/kiwi/installation.html.

If all software components are installed, keg can be utilized like the following example shows:

$ git clone https://github.com/SUSE-Enceladus/keg-recipes.git

$ keg --recipes-root keg-recipes --dest-dir leap_description \
      leap/jeos/15.2

After the keg command completes the destination directory specified with --dest-dir contains and image description that can be processed with KIWI to build an image. For more details about KIWI image descriptions, see https://osinside.github.io/kiwi/image_description.html.

With KIWI installed you can build the image with the keg created image description as follows:

$ sudo kiwi-ng system build --description leap_description \
      --target-dir leap_image

2 Installation

Note
Note

This document describes how to install Keg. Currently keg is provided from PyPi and further install methods for Linux distributions will follow soon.

2.1 Installation from PyPI

Keg can be obtained from the Python Package Index (PyPi) via Python’s package manager pip:

$ pip install kiwi_keg

3 Command Line

3.1 keg

3.1.1 SYNOPSIS

keg (-l|--list-recipes) (-r RECIPES_ROOT|--recipes-root=RECIPES_ROOT)

keg (-r RECIPES_ROOT|--recipes-root=RECIPES_ROOT)
    [--format-xml|--format-yaml] [--dump-dict]
    [-a ADD_DATA_ROOT] ... [-d DEST_DIR] [-fv]
    SOURCE

keg -h | --help

3.1.2 DESCRIPTION

Keg is a tool which helps to create and manage image descriptions suitable for the KIWI appliance builder. While keg can be used to manage a single image definition the tool provides no considerable advantage in such a use case. The primary use case for keg are situations where many image descriptions must be managed and the image descriptions have considerable overlap with respect to content and setup.

Keg requires source data called recipes which provides all information necessary for keg to create KIWI image descriptions. See Chapter 4, Recipes Basics for more information about recipes.

The recipes used for generating SUSE Public Cloud image descriptions can be found in the Public Cloud Keg Recipes repository.

3.1.3 ARGUMENTS

SOURCE

Path to image source, expected under RECIPES_ROOT/images

3.1.4 OPTIONS

-r --recipes-root

Root directory of Keg recipes

-a --add-data-root

Additional data root directory of recipes (multiples allowed)

-d --dest-dir

Destination directory for generated description, default cwd

-l --list-recipes

List available images that can be created with the current recipes

-f --force

Force mode (ignore errors, overwrite files)

--format-yaml

Format/Update Keg written image description to installed KIWI schema and write the result description in YAML markup.

Note
Note

Currently no translation of comment blocks from the Keg generated KIWI description to the YAML markup will be performed

--format-xml

Format/Update Keg written image description to installed KIWI schema and write the result description in XML markup

Note
Note

Currently only toplevel header comments from the Keg written image description will be preserved into the formatted/updated KIWI XML file. Inline comments will not be preserved.

--dump-dict

Parse input data and build image data dictionary, but instead of running the generator, dump data dictionary and exit. Useful for debugging.

-v --verbose

Enable verbose output

3.1.5 EXAMPLE

$ git clone https://github.com/SUSE-Enceladus/keg-recipes.git

$ keg --recipes-root keg-recipes --dest-dir leap_description leap/jeos/15.2

4 Recipes Basics

To produce image descriptions, keg must be provided with source data, also called keg recipes. Unlike kiwi descriptions, keg recipes can be composed of an arbitrary number of files, which allows to create building blocks for images. Keg does not mandate a specific structure of the recipes data, with the exception that it expects certain types of source data in specific directories, but how you want to structure the data is up to you.

This document describes the fundamental keg recipes structure and how keg processes input data to generate an image definition.

4.1 Recipes Data Types

There are several types of source data in keg recipes:

  • Image Definitions

Defines image properties and composition. Image definitions must be placed in the images directory in the recipes root directory. Input format is yaml.

See Chapter 5, Image Definition for details.

  • Data Specifications

Specifies profile parameters, package lists, image setup configuration, and overlay data configuration. Data specifications must be placed in the data directory. The sub directories data/scripts and data/overlayfiles are reserved for configuration scriptlets and overlay files (see below). Everything else under data is considered data module specification with input format yaml.

2.1 Image Configuration Scripts

Image descriptions may include configuration scripts, which keg can compose from scriptlets. Those must be placed in data/scripts. All files need to have a sh suffix. Format is bash.

2.2 Overlay Data

Image description may include overlay files that get copied into the target image. Keg can create overlay archives from overlay data directories. Those must be placed in data/overlayfiles.

See Chapter 6, Data Modules for details on data modules.

  • Schema Templates

Keg uses jinja2 templates to produce the target config.kiwi file. The template defines the output structure, which is provided with the full image description as composed by keg. Templates need to be in data/schemas.

4.2 Source Data Format and Processing

This section contains some general information about how keg handles its source data. An image description is internally represented by a data dictionary with a certain structure. This dictionary gets composed by parsing source image definition and data files referenced by the image definition and merging them into a dictionary. Image definitions as well as data modules are used by referencing a directory (under images or data respectively), which may be several layers of directories under the root directory. When parsing those, keg will also read any yaml file that is in a directory above the referenced one, and merge all source data into one dictionary, with the lower (i.e. more specific) layers taking precedence over upper (i.e. more generic) ones. This inheritance mechanism is intended to reduce data duplication.

5 Image Definition

Keg considers all leaf directories in images to be image definitions. This means by parsing any yaml file from those directories and all yaml files in any parent directory and merging their data into a dictionary, a complete image definition needs to be available in the resulting dictionary. There is no specific hierarchy required in images. You can use any level of sub directories to use any level of inheritance, or simply just to group image definitions. Example directory layout:

images/
       opensuse/
                defaults.yaml
                leap/
                     profiles.yaml
                     15.2/
                          image.yaml
                     15.3/
                          image.yaml

This example layout defines two images, opensuse/leap/15.2 and opensuse/leap/15.3. It uses inheritance to define a common profile for both image definitions, and to set some opensuse specific defaults. Running keg -d output_dir opensuse/leap/15.3 would merge data from the following files in the show order:

images/opensuse/defaults.yaml
images/opensuse/leap/profiles.yaml
images/opensuse/leap/15.3/image.yaml

5.1 Image Definition Structure

To properly define an image, the dictionary produced from merging the dictionaries from a given input path need to have the following structure:

archs:
  - string
  ...
include-paths:
  - string
  ...
image:
  author: string
  contact: string
  name: string
  specification: string
  version: integer.integer.integer
profiles:
  common:
    include:
      - string
      ...
  profile1:
    include:
      - string
      ...
  ...
schema: string
users:
  - name: string
    groups:
      - string
      ...
    home: string
    password: string
  ...
Note
Note

schema corresponds to a template file in schema (with .kiwi.templ extension added). The schema defines the output structure and hence the input structure is dependent on what schema is used.

Some of the listed dictionary items are not strictly required by keg but they are used by the template provided in the keg-recipes repository.

Note
Note

Image definitions that define a common profile only in the profiles section are considered single-build and definitions with additional profiles are considered multi-build. Single-build image descriptions produce a single image binary, with all configuration properties included in the common profile. Multi-build image descriptions produce an image binary per profile, with each profile using all configuration properties included in the corresponding profile section and the common profile. Since the generated image description depends on the used template this may not apply for custom templates.

The profiles section is what defines the image configuration and data composition. Any list item in include refers to a directory under data.

6 Data Modules

Data modules are essentially directories in the data tree. Inheritance rules apply similarly to the image definition tree, but additionally, keg supports cross inheritance for data modules. Cross inheritance is useful to inherit configuration changes from previous versions. This can be specified in the image definition using the include-paths list. Include paths are paths that get appended to any source path and those get scanned for input files as well. For example, let’s assume you have the following configuration in your image definition:

include-paths:
  leap15/1
  leap15/2
profiles:
  common:
    include:
      base/common

This tells keg, when adding data from directory data/base/common to the image data dictionary, to also look into sub directories leap15/2, leap15/1, and leap15 (through inheritance). This would lead to the following directories being scanned:

data/common
data/common/base
data/common/base/leap15
data/common/base/leap15/1
data/common/base/leap15/2

This allows for example to put generic configuration bits in common/base, Leap 15 specific configuration in common/base/leap15, and adjust the configuration for minor versions, if necessary.

When building the dictionary, keg will parse all input files referenced in the profiles section and merge them into the main dictionary. The following section describes the structure used in the data section.

6.1 Data Module Dictionary Structure

This section describes the input parameters used by the data modules.

Note
Note

This assumes the schema template provided in the keg-recipes repository. Custom templates may require different input data.

profile:
  bootloader:
    kiwi_bootloader_param: string
    ...
  parameters:
    kiwi_preferences_type_param: string
    ...
    kernelcmdline:
      kernel_param: value
      kernel_multi_param: [value, value]
      ...
  size: integer
packages:
  packages_type:
    namespace:
      - string
      - name: string
        arch: string
config:
  files:
    namespace:
      - path: /path/to/file
        append: bool
        content: string
      ...
  scripts:
    namespace:
      - string
      ....
  sysconfig:
    namespace:
      - file: /etc/sysconfig/file
        name: VARIABLE_NAME
        value: VARIABLE_VALUE
      ...
  services:
    namespace:
      - string
      - name: string
        enable: bool
      ...
setup:
  (same as config)
overlayfiles:
  namespace:
    include:
      - string
      ...
  namespace_named_archive:
    archivename: string
    include:
      - string
      ...
Note
Note

For multi-build image definitions, any module that defines parameters must be included in the profile specific section of the image definition. Inclusion in the common profile only works for single-build image definitions.

Namespace may be any name. Namespaces exist to allow for dictionaries to be merged without overwriting keys from inherited dictionaries, except where this is wanted. Using the same namespace in a more specific dictionary (i.e. a lower level directory) can be used to change or even remove that namespace (for the latter set it to Null).

kiwi_bootloader_param refers to any bootloader type parameter supported by kiwi.

kiwi_preferences_type_param refers to any preferences type parameter supported by kiwi (see <preferences><type> in kiwi documentation).

kernelcmdline is not a string that is direct copied into the appropriate kiwi parameter but a dictionary that defines kernel parameters individually, with each key representing a kernel parameter. This allows to inherit parts of the kernel command line from other modules. There are two notations for parameters. kernel_param: value will be translated into a single kernel_param=value, and kernel_multi_param: [value, value, ...] will add kernel_multi_param multiple times for each value from the given list.

packages_type can be bootstrap or image (see kiwi documentation). The items in the package list have two possible notations. Either just a plain string, which is considered to be the package name, or a dictionary with keys name (the package name) and arch (the build architecture for which the package should be included).

List items in configscript refer to files in data/scripts (with .sh appended by keg) and the content of those will be added to the config.sh script.

List items in configservices refer to system service that should be enabled or disabled in the image. Analogue to packages, there are two supported version, a short one containing only the service name, or a long one that allows to specify whether the service should be enabled or disabled).

setup has the same structure as config but the data will be used to generated images.sh instead of config.sh.

List items in overlayfiles refer to directories under data/overlayfiles. Files from those directories will be copied into an overlay archive to be included in the image, either a generic or a profile specific one (depending on where the data module was included), or a named one in case archivename tag is used.

Print this page