Quick Start

K3k, Kubernetes in Kubernetes, is a tool that enables you to create and manage isolated K3s clusters within your existing Kubernetes environment. It enables multi-tenancy, experimentation, and resource isolation, reducing infrastructure costs by allowing you to run multiple lightweight Kubernetes clusters on the same physical host. K3k offers both "shared" mode, optimizing resource utilization, and "virtual" mode, providing complete isolation with dedicated K3s server pods. This allows you to access a full Kubernetes experience without the overhead of managing separate physical resources.

K3k integrates with Rancher for simplified management of your embedded clusters.

Installation

This section provides instructions on how to install K3k and the k3kcli.

Prerequisites

Helm must be installed to use the charts. Please refer to Helm’s documentation to get started.
An existing RKE2 Kubernetes cluster (recommended).
A configured storage provider with a default storage class.

If you do not have a storage provider, you can configure the cluster to use ephemeral or static storage. Please consult the k3kcli Advanced Usage for instructions on using these options.

Install the K3k controller

Rancher Prime customers, please see Quick Start.

Add the K3k Helm repository:

helm repo add k3k https://rancher.github.io/k3k
helm repo update

Install the K3k controller:

helm install --namespace k3k-system --create-namespace k3k k3k/k3k

Install the `k3kcli`

The k3kcli provides a quick and easy way to create K3k clusters and automatically exposes them via a kubeconfig.

To install it, simply download the latest available version for your architecture from the GitHub Releases page.

For example, you can download the Linux amd64 version with:

wget -qO k3kcli https://github.com/rancher/k3k/releases/download/v1.0.0/k3kcli-linux-amd64 && \
  chmod +x k3kcli && \
  sudo mv k3kcli /usr/local/bin

Now run:

-> % k3kcli --version
k3kcli version v0.3.4

Usage

This section provides examples of how to use the k3kcli to manage your K3k clusters.

K3k operates within the context of your currently configured kubectl context. This means that K3k respects the standard Kubernetes mechanisms for context configuration, including the --kubeconfig flag, the $KUBECONFIG environment variable, and the default $HOME/.kube/config file. Any K3k clusters you create will reside within the Kubernetes cluster that your kubectl is currently pointing to.

Creating a K3k Cluster

To create a new K3k cluster, use the following command:

k3kcli cluster create mycluster

Creating a K3k Cluster on a Rancher-Managed Host Cluster

If your host Kubernetes cluster is managed by Rancher (for example, your kubeconfig’s server address includes a Rancher URL), use the --kubeconfig-server flag when creating your K3k cluster:

k3kcli cluster create --kubeconfig-server <host_node_IP_or_load_balancer_IP> mycluster

This ensures the generated kubeconfig connects to the correct endpoint.

When the K3s server is ready, k3kcli will generate the necessary kubeconfig file and print instructions on how to use it.

Here’s an example of the output:

INFO[0000] Creating a new cluster [mycluster]
INFO[0000] Extracting Kubeconfig for [mycluster] cluster
INFO[0000] waiting for cluster to be available..
INFO[0073] certificate CN=system:admin,O=system:masters signed by CN=k3s-client-ca@1738746570: notBefore=2025-02-05 09:09:30 +0000 UTC notAfter=2026-02-05 09:10:42 +0000 UTC
INFO[0073] You can start using the cluster with:

        export KUBECONFIG=/my/current/directory/mycluster-kubeconfig.yaml
        kubectl cluster-info

After exporting the generated kubeconfig, you can reach your Kubernetes cluster:

export KUBECONFIG=/my/current/directory/mycluster-kubeconfig.yaml
kubectl get nodes
kubectl get pods -A

You can also directly create a Cluster resource in some namespace, to create a K3k cluster:

kubectl apply -f - <<EOF
apiVersion: k3k.io/v1alpha1
kind: Cluster
metadata:
  name: mycluster
  namespace: k3k-mycluster
EOF

and use the k3kcli to retrieve the kubeconfig:

k3kcli kubeconfig generate --namespace k3k-mycluster --name mycluster

Deleting a K3k Cluster

To delete a K3k cluster, use the following command:

k3kcli cluster delete mycluster