How-to: Expose Virtual Cluster Kubernetes API

This guide explains how to expose the Kubernetes API of the virtual clusters to external networks. Practices apply to both virtual mode and shared mode.

It is important to add the TLS-SAN arguments to the cluster, such that the IP address or host are not rejected.

Running k3kcli cluster delete clustername will delete the virtual cluster but will not cascade to the ingresses or services created using the following methods.

Ingress

It is possible to expose the virtual cluster’s Kubernetes API through an ingress controller present on the host. However, it is important to note that TLS termination must be performed at the virtual cluster, requiring some changes from normal ingress objects.

Traefik

In case of Traefik, the virtual cluster’s API is exposed through an IngressRouteTCP resource. Here’s an example configuration:

apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: cluster-ingress-rt
spec:
  entryPoints:
    - websecure
  routes:
    - match: HostSNI(`cluster.example.com`)
      services:
        - name: k3k-clustername-service
          port: 443
  tls:
    passthrough: true

Ingress Nginx

Ingress Nginx does not have SSL passthrough enabled out of the box and must be enabled with the --enable-ssl-passthrough flag. Refer to the ingress-nginx documentation {https://kubernetes.github.io/ingress-nginx/user-guide/tls/#ssl-passthrough}[here].

Afterwards, it is possible to create a normal ingress resource with the annotation "nginx.ingress.kubernetes.io/ssl-passthrough". For example:

apiVersion: networking.k8s.io
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "https"
  name: cluster-ingress
spec:
  rules:
  - host: cluster.example.com
    http:
      paths:
      - backend:
          service:
            name: k3k-clustername-service
            port:
              number: 443
        path: /
        pathType: Prefix

Service

The services created at creation time of the virtual cluster cannot be modified. It is necessary to create a new service of the desired type (LoadBalancer or NodePort).

LoadBalancer

If a load balancer is available to the host cluster, either from a cloud service provider or from an add-on such as MetalLB, it is possible to expose the virtual cluster’s API over its own unique IP address.

For an example cluster called test in the namespace k3k_test, create a LoadBalancer service with the following command:

kubectl -n k3k_test expose service k3k-test-service --type=LoadBalancer --name=k3k-test-servicelb

NodePort

In the absence of the above solutions, the virtual Kubernetes API can be exposed over a NodePort service using the IP address of any of the host nodes.

For an example cluster named test in the namespace k3k_test, create a NodePort service using the following command:

kubectl -n k3k_test expose service k3k-test-service --type=NodePort --name=k3k-test-servicenp