SUSE Multi-Linux Manager 5.2 RC Proxy Deployment on Kubernetes
1. Proxy on Kubernetes changes
There were multiple changes in how to install SUSE Multi-Linux Manager proxies running on Kubernetes:
-
mgrpxyis no longer handling proxies on Kubernetes, helm and theproxy-helmchart need to be used instead. -
The TLS certificates have to be in secrets, rather than in the configuration tarball. This aims at allowing cloud-native TLS certificates management for the proxies.
-
The proxy queries the server at the start of the container to verify that the versions are compatible.
-
The needed persistent volume claims has been reduced to the squid cache only.
-
The SUSE Multi-Linux Manager proxy is supported when running on an RKE2 or K3S cluster.
2. Prerequisites
Installing the Kubernetes cluster and configuring it is out of the scope of this document.
The cluster is assumed to be ready to be used with a user having rights on a namespace dedicated to SUSE Multi-Linux Manager.
Create Role and RoleBinding if they do not exist already.
The minimum rights required to deploy proxy-helm are defined as:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: example-resource-manager
namespace: uyuni-proxy
rules:
- apiGroups: [""]
resources: ["pods", "pods/log", "services", "secrets", "configmaps", "persistentvolumeclaims"]
verbs: ["*"]
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["*"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: example-resource-manager-binding
namespace: uyuni-proxy
subjects:
- kind: User
name: $USERNAME
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: example-resource-manager
apiGroup: rbac.authorization.k8s.io
|
This guide assumes the reader knows how to work with Kubernetes: the concepts will not be explained here as they are extensively documented in the official Kubernetes documentation. |
The SUSE Multi-Linux Manager administrator needs to deploy the proxy-helm Helm chart.
However, this chart requires to prepare:
-
TLS certificates chain for the proxy,
-
a
ConfigMapfor the proxy root CA certificate, -
a persistent volumes for the claim the chart will create or a storage class automatically creating it,
-
Load balancers or other mechanisms to expose the Salt, SSH and TFTP ports.
|
The TLS certificate and the root CA |
Run the following command to read the full details on how to use the proxy Helm chart:
helm show readme --version 5.2.0 \
oci://registry.suse.com/suse/multi-linux-manager/5.2/proxy-helm
2.1. Credentials
A secret with the SCC credentials needs to be defined in order to pull the images from registry.suse.com.
Refer to https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ for the instructions to prepare the secret.
Set the registrySecret proxy-helm chart value to the name of the secret containing those credentials to use it.
2.2. TLS setup
The proxy needs the following objects in its namespace:
-
A TLS secret named
proxy-certcontaining the proxy server certificate and key, with the public FQDN as Subject Alternate Name. -
A
ConfigMapnameduyuni-cacontaining the root CA in theca.crtkey.
In a cloud-native setup the certificates are typically managed externally (for example by cert-manager); create the objects beforehand:
kubectl create secret tls proxy-cert -n uyuni-proxy \
--cert=server.crt --key=server.key
kubectl create configmap uyuni-ca -n uyuni-proxy \
--from-file=ca.crt=/path/to/uyuni-ca.crt
The certificate file passed to kubectl create secret tls needs to start with the server certificate followed by the chain of intermediary CA certificates if any.
The root CA is not needed in this secret as it is expected in the uyuni-ca ConfigMap.
|
Both objects can also be created by the
This requires the configuration tarball to actually contain those fields.
If the proxy configuration was generated with the SSL part skipped (for example via the When
The toggle is required because the default skip-if-exists behavior leaves nothing for |
2.3. Storage
The proxy chart defines a volume as a Persistent Volume Claim (PVC).
|
The created PVC can be tuned Helm chart values, it can have the following values:
-
size: to set the requested size of the PVC. -
storageClass: can be used to select the storage class to use for the PVC. -
extraLabels: can be used to add custom labels to the PVC. -
annotations: can be used to set custom annotations on the PVC. -
volumeName: can be used to hard code which volume the PVC should be bound to. -
selector: is the YAML fragment of the PVC selector to use to find the PV to bind to.
Refer to https://kubernetes.io/docs/concepts/storage/persistent-volumes/ for more information on persistent volumes and their claims.
Refer to the proxy-helm README for the list of persistent volume claims which will be created and will need to be bound to persistent volumes.
|
While the default sizes are provided, it is highly recommended to change them based on the distributions you plan to synchronize. For more information on storage requirements see General Requirements. |
2.4. Exposing ports
SUSE Multi-Linux Manager proxy requires some TCP and UDP ports to be routed to its services.
Refer to the proxy-helm README for the list of ports to be exposed.
|
RKE2 ships with nginx as the default ingress controller.
However, as this is deprecated and soon to be unsupported, the K3S ships Traefik by default, so no controller swap is needed. |
|
The |
There are multiple ways to expose the ports, but this documentation will only mention how to configure the bundled Traefik on RKE2 and K3S. This is not a task for the SUSE Multi-Linux Manager administrator, but the Kubernetes cluster administrator as it requires configuration to be set on the cluster nodes.
2.4.1. On RKE2
Create /var/lib/rancher/rke2/server/manifests/uyuni-traefik.yaml on each node with the following content.
Note that Traefik takes a few seconds to be reinstalled after saving the file.
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: rke2-traefik
namespace: kube-system
spec:
valuesContent: |-
ports:
ssh:
port: 8022
expose:
default: true
exposedPort: 8022
protocol: TCP
hostPort: 8022
salt-publish:
port: 4505
expose:
default: true
exposedPort: 4505
protocol: TCP
hostPort: 4505
containerPort: 4505
salt-request:
port: 4506
expose:
default: true
exposedPort: 4506
protocol: TCP
hostPort: 4506
containerPort: 4506
2.4.2. On K3S
K3S ships Traefik in the kube-system namespace. Create /var/lib/rancher/k3s/server/manifests/uyuni-traefik.yaml on each node with the following content.
The HelmChartConfig name is traefik and hostPort/containerPort are not required:
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
ports:
ssh:
port: 8022
exposedPort: 8022
protocol: TCP
expose:
default: true
salt-publish:
port: 4505
exposedPort: 4505
protocol: TCP
expose:
default: true
salt-request:
port: 4506
exposedPort: 4506
protocol: TCP
expose:
default: true
|
On a vanilla K3S install, the bundled Traefik picks up every |
|
K3S’s built-in load balancer ( |
|
The Traefik entryPoint names (
|
If Traefik is used as the Ingress controller, the user needs access to additional resources. Add the following to the rules of the previously defined role:
- apiGroups: ["traefik.io", "traefik.containo.us"]
resources: ["ingressroutetcps"]
verbs: ["*"]
If Gateway API is used instead, add the following to the rules of the previously defined role:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gateways", "httproutes", "tcproutes"]
verbs: ["*"]
TFTP is complex to expose from a Kubernetes pod due to the nature of the protocol: the TFTP server receives requests on port 69, but negotiates another random port to continue. This port also needs to stay the same through the whole session for the server to recognize the client as being the same. This means that there are only two possible ways to use the TFTP server:
-
using a load balancer compatible with TFTP,
-
using the host network for the TFTP pod. This can be achieved by setting the
tftp.hostNetworkhelm chart value totrue.
2.5. DNS configuration
The dnsConfig chart value is a passthrough to the pod’s https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-configPodDNSConfig].
It is unset by default so the pod inherits the cluster’s normal DNS behavior.
If external FQDN lookups from the proxy hang for around 10 seconds (the default urlopen timeout), the cluster’s default ndots:5 is likely expanding the name through several cluster search domains before falling back to the bare lookup.
Override with:
dnsConfig:
options:
- name: ndots
value: "1"
3. Configuration generation
Before deploying the SUSE Multi-Linux Manager proxy, a configuration archive needs to be generated.
3.1. Generate the Proxy Configuration with Web UI
In the Web UI, navigate to and fill the required data:
In the
Proxy FQDNfield type fully qualified domain name for the proxy.In the
Parent FQDNfield type fully qualified domain name for the SUSE Multi-Linux Manager Server or another SUSE Multi-Linux Manager Proxy.In the
Proxy SSH portfield type SSH port on which SSH service is listening on SUSE Multi-Linux Manager Proxy. Recommended is to keep default 8022.In the
Max Squid cache size [MB]field type maximal allowed size for Squid cache. Recommended is to use at most 80% of available storage for the containers.
2 GB represents the default proxy squid cache size. This will need to be adjusted for your environment.
In the
SSL certificateselection list choose if a new server certificate should be generated for SUSE Multi-Linux Manager Proxy, an existing one should be used, or the SSL part should be skipped entirely. You can consider generated certificates as SUSE Multi-Linux Manager builtin (self signed) certificates. If SUSE Multi-Linux Manager server runs on Kubernetes, the generated certificate option is not possible and replaced with no SSL certificate as they are managed outside the containers.Depending on the choice then provide either path to signing CA certificate to generate a new certificate or path to an existing certificate and its key to be used as proxy certificate. If the SSL part is skipped, the resulting configuration archive does not carry
server_crt,server_keyorca_crt; the deployment is then responsible for providing the proxy certificate and root CA out-of-band (for Kubernetes deployments this means creatingproxy-certanduyuni-camanually — see the TLS setup section of the Kubernetes proxy deployment page).The CA certificates generated by the server are stored in the
/var/lib/containers/storage/volumes/root/_data/ssl-builddirectory.For more information about existing or custom certificates and the concept of corporate and intermediate certificates, see SSL 인증서 임포트.
Click Generate to register a new proxy FQDN in the SUSE Multi-Linux Manager Server and generate a configuration archive (
config.tar.gz) containing details for the container host.After a few moments you are presented with file to download. Save this file locally.
3.2. Generate Proxy Configuration With spacecmd and Self-Signed Certificate
You can generate a Proxy configuration using spacecmd.
This is only possible if SUSE Multi-Linux Manager server runs on podman and has a self-signed root CA certificate.
SSH into your container host.
Execute the following command replacing the Server and Proxy FQDN:
mgrctl exec -ti 'spacecmd proxy_container_config_generate_cert -- dev-pxy.example.com dev-srv.example.com 2048 email@example.com -o /tmp/config.tar.gz'Copy the generated configuration from the server container:
mgrctl cp server:/tmp/config.tar.gz .
3.3. Generate Proxy Configuration With spacecmd and Custom Certificate
You can generate a Proxy configuration using spacecmd for custom certificates rather than the default self-signed certificates.
SSH into your Server container host.
Execute the following commands, replacing the Server and Proxy FQDN:
for f in ca.crt proxy.crt proxy.key; do mgrctl cp $f server:/tmp/$f done mgrctl exec -ti 'spacecmd proxy_container_config -- -p 8022 pxy.example.com srv.example.com 2048 email@example.com /tmp/ca.crt /tmp/proxy.crt /tmp/proxy.key -o /tmp/config.tar.gz'If your setup uses an intermediate CA, copy it as well and include it in the command with the
-ioption (can be provided multiple times if needed) :mgrctl cp intermediateCA.pem server:/tmp/intermediateCA.pem mgrctl exec -ti 'spacecmd proxy_container_config -- -p 8022 -i /tmp/intermediateCA.pem pxy.example.com srv.example.com 2048 email@example.com /tmp/ca.crt /tmp/proxy.crt /tmp/proxy.key -o /tmp/config.tar.gz'Copy the generated configuration from the server container:
mgrctl cp server:/tmp/config.tar.gz .
3.4. Generate Proxy Configuration With spacecmd and no Certificate
You can generate a Proxy configuration using spacecmd with no TLS certificates.
This is needed for SUSE Multi-Linux Manager running on Kubernetes as the certificates are handled outside of the containers.
SSH into your Server container host.
Execute the following commands, replacing the Server and Proxy FQDN:
for f in ca.crt proxy.crt proxy.key; do mgrctl cp $f server:/tmp/$f done mgrctl exec -ti 'spacecmd proxy_container_config_nossl -- -p 8022 pxy.example.com srv.example.com 2048 email@example.com -o /tmp/config.tar.gz'Copy the generated configuration from the server container:
mgrctl cp server:/tmp/config.tar.gz .
4. Deploying the SUSE Multi-Linux Manager Proxy Helm Chart
Copy and extract the generated configuration tar.gz file and then install using helm:
helm install smlm-proxy \
oci://registry.suse.com/suse/multi-linux-manager/5.2/proxy-helm \
-n uyuni-proxy \
--description "Proxy installation" \
--set "registrySecret=the-scc-secret" \
--set-file global.config=path/to/config.yaml \
--set-file global.ssh=path/to/ssh.yaml \
--set-file global.httpd=path/to/httpd.yaml \
When setting multiple values, using a YAML values file is recommended instead of passing several --set parameters.
Refer to the helm command help for more details.
5. Example helm charts
Some helm charts using the proxy-helm chart can be found in the Manager-5.2 branch of the uyuni-charts git repository.
They show case how the TLS certificate can be generated using cert-manager and trust-manager.
Those examples may assume to have Kubernetes cluster administrator permissions.
|
These examples are not supported, only provided for documentation purpose. |