SUSE Multi-Linux Manager Proxy Deployment on Kubernetes

1. Proxy on Kubernetes changes

There were multiple changes in how to install SUSE Multi-Linux Manager proxies running on Kubernetes:

  • mgrpxy is no longer handling proxies on Kubernetes, helm and the proxy-helm chart need to be used instead.

  • The TLS certificates have to be in secrets, rather than in the configuration tarball. This aims at allowing cloud-native TLS certificates management for the proxies.

  • The proxy queries the server at the start of the container to verify that the versions are compatible.

  • The needed persistent volume claims has been reduced to the squid cache only.

  • The SUSE Multi-Linux Manager proxy is supported when running on an RKE2 or K3S cluster.

2. Prerequisites

Installing the Kubernetes cluster and configuring it is out of the scope of this document.

The cluster is assumed to be ready to be used with a user having rights on a namespace dedicated to SUSE Multi-Linux Manager.

Create Role and RoleBinding if they do not exist already. The minimum rights required to deploy proxy-helm are defined as:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: example-resource-manager
  namespace: uyuni-proxy
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log", "services", "secrets", "configmaps", "persistentvolumeclaims"]
  verbs: ["*"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["*"]
- apiGroups: ["networking.k8s.io"]
  resources: ["ingresses"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: example-resource-manager-binding
  namespace: uyuni-proxy
subjects:
- kind: User
  name: $USERNAME
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: example-resource-manager
  apiGroup: rbac.authorization.k8s.io

This guide assumes the reader knows how to work with Kubernetes: the concepts will not be explained here as they are extensively documented in the official Kubernetes documentation.

The SUSE Multi-Linux Manager administrator needs to deploy the proxy-helm Helm chart. However, this chart requires to prepare:

  • TLS certificates chain for the proxy,

  • a ConfigMap for the proxy root CA certificate,

  • a persistent volumes for the claim the chart will create or a storage class automatically creating it,

  • Load balancers or other mechanisms to expose the Salt, SSH and TFTP ports.

The TLS certificate and the root CA ConfigMap are optional inputs: the proxy-helm chart can create proxy-cert and uyuni-ca itself from the values carried in the configuration tarball. See the TLS setup section below for the trade-offs and the adoption procedure for pre-existing objects.

Run the following command to read the full details on how to use the proxy Helm chart:

helm show readme --version 5.2.0 \
    oci://registry.suse.com/suse/multi-linux-manager/5.2/proxy-helm

2.1. Credentials

A secret with the SCC credentials needs to be defined in order to pull the images from registry.suse.com. Refer to https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ for the instructions to prepare the secret. Set the registrySecret proxy-helm chart value to the name of the secret containing those credentials to use it.

2.2. TLS setup

The proxy needs the following objects in its namespace:

  • A TLS secret named proxy-cert containing the proxy server certificate and key, with the public FQDN as Subject Alternate Name.

  • A ConfigMap named uyuni-ca containing the root CA in the ca.crt key.

In a cloud-native setup the certificates are typically managed externally (for example by cert-manager); create the objects beforehand:

kubectl create secret tls proxy-cert -n uyuni-proxy \
    --cert=proxy.crt --key=proxy.key

kubectl create configmap uyuni-ca -n uyuni-proxy \
    --from-file=ca.crt=/path/to/uyuni-ca.crt

The certificate file passed to kubectl create secret tls needs to start with the server certificate followed by the chain of intermediary CA certificates if any. The root CA is not needed in this secret as it is expected in the uyuni-ca ConfigMap.

Both objects can also be created by the proxy-helm chart itself from the values in the configuration tarball, with no manual kubectl create step:

  • proxy-cert is rendered from the server_crt and server_key fields of httpd.yaml.

  • uyuni-ca is rendered from the ca_crt field of config.yaml.

This requires the configuration tarball to actually contain those fields. If the proxy configuration was generated with the SSL part skipped (for example via the SSL certificate selection list in the Web UI, or with spacecmd proxy_container_config_nossl), the tarball carries no certificates and the auto-render path produces nothing — proxy-cert and uyuni-ca must then be created manually as shown above.

When proxy-cert or uyuni-ca already exist in the namespace (created manually or by a previous tool), the chart leaves them alone and the proxy uses the values already present. To switch a pre-existing object over to helm management from the tarball values, set the cert.takeOwnership chart value to true and pass helm install --take-ownership (Helm 3.14+) once:

helm upgrade --install uyuni-proxy \
    --take-ownership \
    --set cert.takeOwnership=true \
    ...

The toggle is required because the default skip-if-exists behavior leaves nothing for --take-ownership to adopt. After adoption the objects carry the helm managed-by label and the chart treats them as helm-managed: value changes are applied on subsequent upgrades, and they are removed on helm uninstall like any other chart resource. On later upgrades cert.takeOwnership can be left at true or dropped back to false — the default render condition already covers helm-managed objects.

2.3. Storage

The proxy chart defines a volume as a Persistent Volume Claim (PVC).

  • The creation of the underlying PV is the responsibility of the cluster administrators.

  • The PVC use the ReadWriteOnce access mode.

The created PVC can be tuned Helm chart values, it can have the following values:

  • size: to set the requested size of the PVC.

  • storageClass: can be used to select the storage class to use for the PVC.

  • extraLabels: can be used to add custom labels to the PVC.

  • annotations: can be used to set custom annotations on the PVC.

  • volumeName: can be used to hard code which volume the PVC should be bound to.

  • selector: is the YAML fragment of the PVC selector to use to find the PV to bind to.

Refer to https://kubernetes.io/docs/concepts/storage/persistent-volumes/ for more information on persistent volumes and their claims.

Refer to the proxy-helm README for the list of persistent volume claims which will be created and will need to be bound to persistent volumes.

While the default sizes are provided, it is highly recommended to change them based on the distributions you plan to synchronize.

For more information on storage requirements see 일반 요구사항.

2.4. Exposing ports

SUSE Multi-Linux Manager proxy requires some TCP and UDP ports to be routed to its services. Refer to the proxy-helm README for the list of ports to be exposed.

RKE2 ships with nginx as the default ingress controller. However, as this is deprecated and soon to be unsupported, the proxy-helm chart defaults to use Traefik as ingress controller. Using the nginx ingress controller might work and will not be documented, use at your own risk.

K3S ships Traefik by default, so no controller swap is needed.

The proxy-helm chart supports Gateway API version 1.4. Since this requires experimental CRDs which are not shipped with RKE2 1.35 or the bundled K3S Traefik, it is not recommended to be used in production.

There are multiple ways to expose the ports, but this documentation will only mention how to configure the bundled Traefik on RKE2 and K3S. This is not a task for the SUSE Multi-Linux Manager administrator, but the Kubernetes cluster administrator as it requires configuration to be set on the cluster nodes.

2.4.1. On RKE2

Create /var/lib/rancher/rke2/server/manifests/uyuni-traefik.yaml on each node with the following content. Note that Traefik takes a few seconds to be reinstalled after saving the file.

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-traefik
  namespace: kube-system
spec:
  valuesContent: |-
    ports:
      ssh:
        port: 8022
        expose:
          default: true
        exposedPort: 8022
        protocol: TCP
        hostPort: 8022
      salt-publish:
        port: 4505
        expose:
          default: true
        exposedPort: 4505
        protocol: TCP
        hostPort: 4505
        containerPort: 4505
      salt-request:
        port: 4506
        expose:
          default: true
        exposedPort: 4506
        protocol: TCP
        hostPort: 4506
        containerPort: 4506

2.4.2. On K3S

K3S ships Traefik in the kube-system namespace. Create /var/lib/rancher/k3s/server/manifests/uyuni-traefik.yaml on each node with the following content. The HelmChartConfig name is traefik and hostPort/containerPort are not required:

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    ports:
      ssh:
        port: 8022
        exposedPort: 8022
        protocol: TCP
        expose:
          default: true
      salt-publish:
        port: 4505
        exposedPort: 4505
        protocol: TCP
        expose:
          default: true
      salt-request:
        port: 4506
        exposedPort: 4506
        protocol: TCP
        expose:
          default: true

On a vanilla K3S install, the bundled Traefik picks up every Ingress regardless of class. Set ingress.class: "" in the proxy-helm values so the chart does not emit a class annotation that would be ignored. If the Traefik install is customized to filter by class (for example via --providers.kubernetescrd.ingressclass=traefik), keep ingress.class: "traefik".

K3S’s built-in load balancer (klipper-lb / serviceLB) does not support the TFTP protocol. Expose TFTP via the host network instead by setting tftp.hostNetwork: true in the chart values.

The Traefik entryPoint names (ssh, salt-publish and salt-request) must match the ingress.entryPoints values configured in the proxy-helm chart (the names above are the chart defaults). If the surrounding Traefik installation uses different entryPoint names, override the chart defaults via the ingress.entryPoints value:

ingress:
  entryPoints:
    ssh: "uyuni-ssh"
    saltPublish: "uyuni-publish"
    saltRequest: "uyuni-request"

If Traefik is used as the Ingress controller, the user needs access to additional resources. Add the following to the rules of the previously defined role:

- apiGroups: ["traefik.io", "traefik.containo.us"]
  resources: ["ingressroutetcps"]
  verbs: ["*"]

If Gateway API is used instead, add the following to the rules of the previously defined role:

- apiGroups: ["gateway.networking.k8s.io"]
  resources: ["gateways", "httproutes", "tcproutes"]
  verbs: ["*"]

TFTP is complex to expose from a Kubernetes pod due to the nature of the protocol: the TFTP server receives requests on port 69, but negotiates another random port to continue. This port also needs to stay the same through the whole session for the server to recognize the client as being the same. This means that there are only two possible ways to use the TFTP server:

  • using a load balancer compatible with TFTP,

  • using the host network for the TFTP pod. This can be achieved by setting the tftp.hostNetwork helm chart value to true.

2.5. DNS configuration

The dnsConfig chart value is a passthrough to the pod’s Kubernetes PodDNSConfig. It is unset by default so the pod inherits the cluster’s normal DNS behavior.

If external FQDN lookups from the proxy hang for around 10 seconds (the default urlopen timeout), the cluster’s default ndots:5 is likely expanding the name through several cluster search domains before falling back to the bare lookup. Override with:

dnsConfig:
  options:
    - name: ndots
      value: "1"

3. Configuration generation

Before deploying the SUSE Multi-Linux Manager proxy, a configuration archive needs to be generated.

3.1. Web UI를 사용하여 프록시 구성 생성

Procedure: Generating a Proxy Container Configuration Using Web UI

  1. Web UI에서 Systems  프록시 구성으로 이동하여 필요한 데이터를 입력합니다.

  2. In the Proxy FQDN field type fully qualified domain name for the proxy.

  3. In the Parent FQDN field type fully qualified domain name for the SUSE Multi-Linux Manager Server or another SUSE Multi-Linux Manager Proxy.

  4. In the Proxy SSH port field type SSH port on which SSH service is listening on SUSE Multi-Linux Manager Proxy. Recommended is to keep default 8022.

  5. In the Max Squid cache size [MB] field type maximal allowed size for Squid cache. Recommended is to use at most 80% of available storage for the containers.

    2GB는 기본 프록시 squid 캐시 크기입니다. 사용자의 환경에 적합하도록 조정해야 합니다.

    In the SSL certificate selection list choose if a new server certificate should be generated for SUSE Multi-Linux Manager Proxy, an existing one should be used, or the SSL part should be skipped entirely. You can consider generated certificates as SUSE Multi-Linux Manager builtin (self signed) certificates. If SUSE Multi-Linux Manager server runs on Kubernetes, the generated certificate option is not possible and replaced with no SSL certificate as they are managed outside the containers.

    Depending on the choice then provide either path to signing CA certificate to generate a new certificate or path to an existing certificate and its key to be used as proxy certificate. If the SSL part is skipped, the resulting configuration archive does not carry server_crt, server_key or ca_crt; the deployment is then responsible for providing the proxy certificate and root CA out-of-band (for Kubernetes deployments this means creating proxy-cert and uyuni-ca manually — see the TLS setup section of the Kubernetes proxy deployment page).

    The CA certificates generated by the server are stored in the /var/lib/containers/storage/volumes/root/_data/ssl-build directory.

    기존 또는 사용자 정의 인증서와 기업 및 중간 인증서의 개념에 대한 자세한 내용은 SSL 인증서 임포트에서 확인할 수 있습니다.

  6. Click Generate to register a new proxy FQDN in the SUSE Multi-Linux Manager Server and generate a configuration archive (config.tar.gz) containing details for the container host.

  7. 잠시 후 다운로드할 파일이 표시됩니다. 이 파일을 로컬에 저장합니다.

3.2. Generate Proxy Configuration With spacecmd and Self-Signed Certificate

You can generate a Proxy configuration using spacecmd. This is only possible if SUSE Multi-Linux Manager server runs on podman and has a self-signed root CA certificate.

절차: spacecmd 및 자체 서명 인증서를 사용하여 프록시 구성 생성

  1. 컨테이너 호스트에 SSH로 연결합니다.

  2. 서버 및 프록시 FQDN을 바꾸는 다음 명령을 실행합니다.

    mgrctl exec -ti 'spacecmd proxy_container_config_generate_cert -- dev-pxy.example.com dev-srv.example.com 2048 email@example.com -o /tmp/config.tar.gz'

  3. 서버 컨테이너에서 생성된 구성을 복사합니다.

    mgrctl cp server:/tmp/config.tar.gz .

3.3. Generate Proxy Configuration With spacecmd and Custom Certificate

You can generate a Proxy configuration using spacecmd for custom certificates rather than the default self-signed certificates.

절차: spacecmd 및 사용자 정의 인증서를 사용하여 프록시 구성 생성

  1. 서버 컨테이너 호스트에 SSH로 연결합니다.

  2. Execute the following commands, replacing the Server and Proxy FQDN:

    for f in ca.crt proxy.crt proxy.key; do
  mgrctl cp $f server:/tmp/$f
done
mgrctl exec -ti 'spacecmd proxy_container_config -- -p 8022 pxy.example.com srv.example.com 2048 email@example.com /tmp/ca.crt /tmp/proxy.crt /tmp/proxy.key -o /tmp/config.tar.gz'

  3. 설정에서 중간 CA를 사용하는 경우, 이를 복사하여 -i 옵션과 함께 명령에 포함시킵니다(필요한 경우 여러 번 제공 가능).

    mgrctl cp intermediateCA.pem server:/tmp/intermediateCA.pem
mgrctl exec -ti 'spacecmd proxy_container_config -- -p 8022 -i /tmp/intermediateCA.pem pxy.example.com srv.example.com 2048 email@example.com /tmp/ca.crt /tmp/proxy.crt /tmp/proxy.key -o /tmp/config.tar.gz'

  4. 서버 컨테이너에서 생성된 구성을 복사합니다.

    mgrctl cp server:/tmp/config.tar.gz .

3.4. Generate Proxy Configuration With spacecmd and no Certificate

You can generate a Proxy configuration using spacecmd with no TLS certificates. This is needed for SUSE Multi-Linux Manager running on Kubernetes as the certificates are handled outside of the containers.

Procedure: Generating Proxy Configuration with spacecmd and no Certificate

  1. 서버 컨테이너 호스트에 SSH로 연결합니다.

  2. Execute the following commands, replacing the Server and Proxy FQDN:

    for f in ca.crt proxy.crt proxy.key; do
  mgrctl cp $f server:/tmp/$f
done
mgrctl exec -ti 'spacecmd proxy_container_config_nossl -- -p 8022 pxy.example.com srv.example.com 2048 email@example.com -o /tmp/config.tar.gz'

  3. 서버 컨테이너에서 생성된 구성을 복사합니다.

    mgrctl cp server:/tmp/config.tar.gz .

4. Deploying the SUSE Multi-Linux Manager Proxy Helm Chart

Copy and extract the generated configuration tar.gz file and then install using helm:

The proxy-helm chart requires the repository value to be set. This is the path to the images to use and its value cannot be computed automatically as it contains the target architecture. The other values have sensible defaults, refer to the proxy-helm chart README for more details on those.

Replace $ARCH$ in the repository path with the target architecture, which can be x86_64 or aarch64.

helm install smlm-proxy \
    oci://registry.suse.com/suse/multi-linux-manager/5.2/proxy-helm \
    -n uyuni-proxy \
    --description "Proxy installation" \
    --set "registrySecret=the-scc-secret" \
    --set "repository=registry.suse.com/suse/multi-linux-manager/5.2/$ARCH$/"
    --set-file global.config=path/to/config.yaml \
    --set-file global.ssh=path/to/ssh.yaml \
    --set-file global.httpd=path/to/httpd.yaml \

When setting multiple values, using a YAML values file is recommended instead of passing several --set parameters. Refer to the helm command help for more details.

5. Example helm charts

Some helm charts using the proxy-helm chart can be found in the Manager-5.2 branch of the uyuni-charts git repository. They show case how the TLS certificate can be generated using cert-manager and trust-manager. Those examples may assume to have Kubernetes cluster administrator permissions.

These examples are not supported, only provided for documentation purpose.