Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to SUSE CaaS Platform 4.5.2

14 Miscellaneous

14.1 Configuring HTTP/HTTPS Proxy for CRI-O

In some cases you must configure the container runtime to use a proxy to pull container images.

The CRI-O runtime uses the system-wide proxy configuration, defined at /etc/sysconfig/proxy.

This file can be edited a number of ways. It can be pre-configured at build time via AutoYaST, as described in the AutoYaST documentation. On an existing system, the file can be edited via YaST by running yast2 proxy.

If preferred, it can alternatively be edited manually as described in the SUSE Knowledge Base article

Note
Note

CRI-O and skuba both support four types of comma-separated entries in the NO_PROXY variable:

  • An exact IP address (1.2.3.4)

  • CIDR IP range (1.2.3.4/16)

  • DNS domain name (eg.com matches www.eg.com and eg.com)

  • Restricted DNS subdomain (.eg.com matches www.eg.com but not eg.com)

All standard programs should ignore unsupported values in that variable and continue to work (albeit without the configured proxy) when encountering an unsupported value.

Tip
Tip

Not all programs on all systems will respect CIDR ranges or restricted subdomains.

After you have configured the system proxy for your environment, restart the container runtime with:

systemctl restart crio

14.2 Configuring Container Registries for CRI-O

Important
Important

The configuration example in this text uses VERSION 2 of the CRI-O registries configuration syntax. It is not compatible with the VERSION 1 syntax present in some upstream examples.

Please refer to: https://raw.githubusercontent.com/containers/image/master/docs/containers-registries.conf.5.md

Tip
Tip

You can create and deploy a custom registries.conf during the initial bootstrap of the cluster with skuba. Create the file <CLUSTER_NAME>/addons/containers/registries.conf and apply your changes there. This file will be rolled out during node bootstrapping and upgrading.

Every registry-related configuration needs to be done in the TOML file /etc/containers/registries.conf. After any change of this file, CRI-O needs to be restarted.

The configuration is a sequence of [[registry]] entries. For example, a single registry entry within that configuration could be added like this:

/etc/containers/registries.conf

[[registry]]
blocked = false
insecure = false
location = "example.net/bar"
prefix = "example.com/foo/images"
mirror = [
    { location = "example-mirror-0.local", insecure = false },
    { location = "example-mirror-1.local", insecure = true, mirror-by-digest-only = true }
]

[[registry]]
blocked = false
insecure = false
location = "example.net/mymirror"
prefix = "example.com/mirror/images"
mirror = [
    { location = "example-mirror-2.local", insecure = false, mirror-by-digest-only = true },
    { location = "example-mirror-3.local", insecure = true }
]
unqualified-search = false

Given an image name, a single [[registry]] TOML table is chosen based on its prefix field.

A prefix is mainly a user-specified image name and can have one of the following formats:

  • host[:port]

  • host[:port]/namespace[/namespace…]

  • host[:port]/namespace[/namespace…]/repo

  • host[:port]/namespace[/namespace…]/repo[:tag|@digest]

The user-specified image name must start with the specified prefix (and continue with the appropriate separator) for a particular [[registry]] TOML table to be considered. Only the TOML entry with the longest match is used.

As a special case, the prefix field can be missing. If so, it defaults to the value of the location field.

14.2.1 Per-namespace Settings

  • insecure (true or false): By default, container runtimes require TLS when retrieving images from a registry. If insecure is set to true, unencrypted HTTP as well as TLS connections with untrusted certificates are allowed.

  • blocked (true or false): If true, pulling images with matching names is forbidden.

14.2.2 Remapping and Mirroring Registries

The user-specified image reference is, primarily, a "logical" image name, always used for naming the image. By default, the image reference also directly specifies the registry and repository to use, but the following options can be used to redirect the underlying accesses to different registry servers or locations. This can be used to support configurations with no access to the Internet without having to change Dockerfiles, or to add redundancy.

14.2.2.1 location

Accepts the same format as the prefix field, and specifies the physical location of the prefix-rooted namespace. By default, this is equal to prefix (in which case prefix can be omitted and the [[registry]] TOML table can just specify location).

14.2.2.1.1 Example
prefix = "example.com/foo"
location = "internal-registry-for-example.net/bar"

Requests for the image example.com/foo/myimage:latest will actually work with the internal-registry-for-example.net/bar/myimage:latest image.

14.2.2.2 mirror

An array of TOML tables specifying (possibly partial) mirrors for the prefix-rooted namespace.

The mirrors are attempted in the specified order. The first one that can be contacted and contains the image will be used (and if none of the mirrors contains the image, the primary location specified by the registry.location field, or using the unmodified user-specified reference, is tried last).

Each TOML table in the mirror array can contain the following fields, with the same semantics as if specified in the [[registry]] TOML table directly:

  • location

  • insecure

14.2.2.3 mirror-by-digest-only

Can be true or false. If true, mirrors will only be used during pulling if the image reference includes a digest. Referencing an image by digest ensures that the same one is always used (whereas referencing an image by a tag may cause different registries to return different images if the tag mapping is out of sync).

Note that if this is true, images referenced by a tag will only use the primary registry, failing if that registry is not accessible.

14.3 FlexVolume Configuration

FlexVolume drivers are external (out-of-tree) drivers usually provided by a specific vendor. They are executable files that are placed in a predefined directory in the cluster on both worker and master nodes. Pods interact with FlexVolume drivers through the flexvolume in-tree plugin.

The vendor driver first has to be installed on each worker and master node in a Kubernetes cluster. On SUSE CaaS Platform 4, the path to install the drivers is /usr/lib/kubernetes/kubelet-plugins/volume/exec/.

If the drivers are deployed with DaemonSet, this will require changing the FlexVolume directory path, which is usually stored as an environment variable, for example for rook:

FLEXVOLUME_DIR_PATH=/usr/lib/kubernetes/kubelet-plugins/volume/exec/

For a general guide to the FlexVolume configuration, see https://github.com/kubernetes/community/blob/master/contributors/devel/sig-storage/flexvolume.md

14.4 Configuring kubelet

Warning
Warning

Modifying the file /etc/sysconfig/kubelet directly is not supported.

The changes made to this file will not persist through an update/upgrade of the software. Please follow the instructions below to change the configuration for kubelet persistently.

Note
Note

This procedure does not override the default configuration but amends the changes from the "drop-in" configuration.

Please refer to: https://www.freedesktop.org/software/systemd/man/systemd.unit.html

If you wish to modify the configuration for kubelet you must use the "drop-in" configuration feature of systemd. The steps need to be performed on each cluster node whose kubelet you wish to reconfigure.

  1. Create an appropriate .conf file (e.g. resource-handling.conf) in /usr/lib/systemd/system/kubelet.service.d/ with your desired changes. .

  2. Reload the service definitions

    sudo systemctl daemon-reload
  3. Restart kubelet

    sudo systemctl restart kubelet

14.5 Changes from Kubernetes 1.17 to 1.18

This documentation page lists all the behavioral changes and API deprecations that will happen from Kubernetes 1.17 to 1.18. You will require this information when migrating to SUSE CaaS Platform 5.x shipping with Kubernetes 1.18.

14.5.1 API deprecations

The following APIs have been deprecated:

  • All resources under apps/v1beta1 and apps/v1beta2: please use apps/v1 instead.

  • daemonsets, deployments, replicasets under extensions/v1beta1: please use apps/v1 instead.

  • networkpolicies under extensions/v1beta1: please use networking.k8s.io/v1 instead.

  • podsecuritypolicies resources under extensions/v1beta1: please use policy/v1beta1 instead.

14.5.2 Behavioral changes

In this section we will highlight some relevant changes that might interest you for the new Kubernetes version.

14.5.2.1 Core

  • #853 Configurable scale velocity for HPA.

  • #1393 Provide OIDC discovery for service account token issuer.

  • #1513 CertificateSigningRequest API.

14.5.2.2 Scheduling

  • #1451 Run multiple Scheduling Profiles [Alpha]

  • #895 Even pod spreading across failure domains [Beta]

  • #1258 Add a configurable default Even Pod Spreading rule [Alpha]

  • #166 Taint Based Eviction [Stable]

14.5.2.3 Nodes

  • #1539 Extending Hugepage Feature [Stable]

  • #688 Pod Overhead: account resources tied to the pod sandbox, but not specific containers [Beta]

  • #693 Node Topology Manager [Beta]

  • #950 Add pod-startup liveness-probe holdoff for slow-starting pods [Beta]

14.5.2.4 Networking

  • #752 EndpointSlice API [Beta]

  • #508 IPv6 support added [Beta]

  • #1024 Graduate NodeLocal DNSCache to GA [Stable]

  • #1453 Graduate Ingress to V1 [Beta]

  • #1507 Adding AppProtocol to Services and Endpoints [Stable]

14.5.2.5 API

  • #1040 Priority and Fairness for API Server Requests [Alpha]

  • #1601 client-go signature refactor to standardize options and context handling [Stable]

  • #576 APIServer DryRun [Stable]

  • #1281 API Server Network Proxy KEP to Beta [Beta]

14.5.2.6 Storage

  • #695 Skip Volume Ownership Change [Alpha]

  • #1412 Immutable Secrets and ConfigMaps [Alpha]

  • #1495 Generic data populators [Alpha]

  • #770 Skip attach for non-attachable CSI volumes [Stable]

  • #351 Raw block device using persistent volume source [Stable]

  • #565 CSI Block storage support [Stable]

  • #603 Pass Pod information in CSI calls [Stable]

  • #989 Extend allowed PVC DataSources [Stable]

14.5.2.7 Features

  • #1441 kubectl debug [Alpha]

  • #491 kubectl diff [Stable]

  • #670 Support Out-of-Tree vSphere Cloud Provider [Stable]

Print this page