v1.26.X

Fix: use correct wasm shims names

The embedded flannel cni-plugin binary is now built and versioned separate from the rest of the cni plugins and the embedded flannel controller.

Bump spegel to v0.0.18-k3s3

Adds wildcard registry support

Fixes issue with excessive CPU utilization while waiting for containerd to start

Add env var to allow spegel mirroring of latest tag

Tweak netpol node wait logs

Fix coredns NodeHosts on dual-stack clusters

Bump helm-controller/klipper-helm versions

Fix snapshot prune

Fix issue with etcd node name missing hostname

Rootless mode should also bind service nodePort to host for LoadBalancer type, matching UX of rootful mode.

To enable raw output for the check-config subcommand, you may now set NO_COLOR=1

Fix additional corner cases in registries handling

Bump metrics-server to v0.7.0

K3s will now warn and suppress duplicate entries in the mirror endpoint list for a registry. Containerd does not support listing the same endpoint multiple times as a mirror for a single upstream registry.

Directories containing runtimes need to be included in the $PATH environment variable for effective runtime detection.

Bump runc to v1.1.12 and helm-controller to v0.15.7

Fix handling of bare hostname or IP as endpoint address in registries.yaml

Added runtime classes for wasm/nvidia/crun

Added default runtime flag for containerd

New timezone info in Docker image allows the use of spec.timeZone in CronJobs

Bumped kine to v0.11.0 to resolve issues with postgres and NATS, fix performance of watch channels under heavy load, and improve compatibility with the reference implementation.

Containerd may now be configured to use rdt or blockio configuration by defining rdt_config.yaml or blockio_config.yaml files.

Add agent flag disable-apiserver-lb, agent will not start load balance proxy.

Improved ingress IP ordering from ServiceLB

Disable helm CRD installation for disable-helm-controller

Omit snapshot list configmap entries for snapshots without extra metadata

Add jitter to client config retry to avoid hammering servers when they are starting up

Bumped dynamiclistener to address a race condition that could cause a server to fail to sync its certificates into the Kubernetes secret

Reduced etcd snapshot log spam during initial cluster startup

Don’t apply S3 retention if S3 client failed to initialize

Don’t request metadata when listing S3 snapshots

Print key instead of file path in snapshot metadata log message

Fixed an issue with identifying additional container runtimes

The user will receive a error when --cluster-reset with the --server flag

Users can now rotate the server token using k3s token rotate -t <OLD_TOKEN> --new-token <NEW_TOKEN>. After command succeeds, all server nodes must be restarted with the new token.

Fixed an issue that could cause k3s to attempt to remove members from the etcd cluster immediately following a cluster-reset/restore, if they were queued for removal at the time the snapshot was taken.

Add --image-service-endpoint flag to specify an external image service socket.

Re-enable etcd endpoint auto-sync

Manually requeue configmap reconcile when no nodes have reconciled snapshots

Bump embedded containerd to v1.7.6

Bump embedded stargz-snapshotter plugin to latest

Fixed intermittent drone CI failures due to race conditions in test environment setup scripts

Fixed CI failures due to changes to api discovery changes in Kubernetes 1.28

K3s’s external apiserver listener now declines to add to its certificate any subject names not associated with the kubernetes apiserver service, server nodes, or values of the --tls-san option. This prevents the certificate’s SAN list from being filled with unwanted entries.

K3s no longer enables the apiserver’s enable-aggregator-routing flag when the egress proxy is not being used to route connections to in-cluster endpoints.

Updated the embedded containerd to v1.7.3+k3s1

Updated the embedded runc to v1.1.8

Updated the embedded etcd to v3.5.9+k3s1

User-provided containerd config templates may now use {{ template "base" . }} to include the default K3s template content. This makes it easier to maintain user configuration if the only need is to add additional sections to the file.

Bump docker/docker module version to fix issues with cri-dockerd caused by recent releases of golang rejecting invalid host headers sent by the docker client.

Updated kine to v0.10.2

K3s etcd-snapshot delete fail to delete local file when called with s3 flag (#8144)

Fix for cluster-reset backup from s3 when etcd snapshots are disabled (#8170)

The version of helm used by the bundled helm controller’s job image has been updated to v3.12.3

Bumped dynamiclistener to address an issue that could cause the apiserver/supervisor listener on 6443 to stop serving requests on etcd-only nodes.

The K3s external apiserver/supervisor listener on 6443 now sends a complete certificate chain in the TLS handshake.

Added a new --tls-san-security option. This flag defaults to false, but can be set to true to disable automatically adding SANs to the server’s TLS certificate to satisfy any hostname requested by a client.

Resolved an issue that caused agents joined with kubeadm-style bootstrap tokens to fail to rejoin the cluster when their node object is deleted.

The k3s certificate rotate-ca command now supports the data-dir flag.

The default helm-controller job image can now be overridden with the --helm-job-image CLI flag

Bump docker go.mod #7681

Shortcircuit commands with version or help flags #7683

Add Rotation certification Check, remove func to restart agents #7097

E2E: Sudo for RunCmdOnNode #7686

The bundled metrics-server has been bumped to v0.6.3, and now uses only secure TLS ciphers by default.

The coredns-custom ConfigMap now allows for *.override sections to be included in the .:53 default server block.

The K3s core controllers (supervisor, deploy, and helm) no longer use the admin kubeconfig. This makes it easier to determine from access and audit logs which actions are performed by the system, and which are performed by an administrative user.

Bumped klipper-lb image to v0.4.4 to resolve an issue that prevented access to ServiceLB ports from localhost when the Service ExternalTrafficPolicy was set to Local.

Make LB image configurable when compiling k3s

K3s now allows nodes to join the cluster even if the node password secret cannot be created at the time the node joins. The secret create will be retried in the background. This resolves a potential deadlock created by fail-closed validating webhooks that block secret creation, where the webhook is unavailable until new nodes join the cluster to run the webhook pod.

The bundled containerd’s aufs/devmapper/zfs snapshotter plugins have been restored. These were unintentionally omitted when moving containerd back into the k3s multicall binary in the previous release.

The embedded helm controller has been bumped to v0.15.0, and now supports creating the chart’s target namespace if it does not exist.

--Tls-sans now accepts multiple arguments: --tls-sans="foo,bar"

Prefer-bundled-bin: true now works properly when set in config.yaml.d files

The embedded containerd version has been bumped to v1.7.0-k3s1, and has been reintegrated into the main k3s binary for a significant savings in release artifact size.

K3s now retries the cluster join operation when receiving a "too many learners" error from etcd. This most frequently occurred when attempting to add multiple servers at the same time.

K3s once again supports aarch64 nodes with page size > 4k

The packaged Traefik version has been bumped to v2.9.10 / chart 21.2.0

K3s now prints a more meaningful error when attempting to run from a filesystem mounted noexec.

K3s now exits with a proper error message when the server token uses a bootstrap token id.secret format.

Fixed an issue where Addon, HelmChart, and HelmChartConfig CRDs were created without structural schema, allowing the creation of custom resources of these types with invalid content.

Servers started with the (experimental) --disable-agent flag no longer attempt to run the tunnel authorizer agent component.

Fixed an regression that prevented the pod and cluster egress-selector modes from working properly.

K3s now correctly passes through etcd-args to the temporary etcd that is used to extract cluster bootstrap data when restarting managed etcd nodes.

K3s now properly handles errors obtaining the current etcd cluster member list when a new server is joining the managed etcd cluster.

The embedded kine version has been bumped to v0.10.1. This replaces the legacy lib/pq postgres driver with pgx.

The bundled CNI plugins have been upgraded to v1.2.0-k3s1. The bandwidth and firewall plugins are now included in the bundle.

The embedded Helm controller now supports authenticating to chart repositories via credentials stored in a Secret, as well as passing repo CAs via ConfigMap.

The bundled containerd and runc versions have been bumped to v1.7.1-k3s1/v1.1.7

The bundled local-path-provisioner version has been bumped to v0.0.24

The embedded etcd version has been bumped to v3.5.7

The bundled runc version has been bumped to v1.1.5

Fixed a race condition during cluster reset that could cause the operation to hang and time out.

The bundled coredns version has been bumped to v1.10.1

The client load-balancer that maintains connections to active server nodes now closes connections to servers when they are removed from the cluster. This ensures that agent components immediately reconnect to a current cluster member.

When using an external datastore, K3s now locks the bootstrap key while creating initial cluster bootstrap data, preventing a race condition when multiple servers attempted to initialize the cluster simultaneously.

The embedded kube-router controller has been updated to fix a regression that caused traffic from pods to be blocked by any default drop/deny rules present on the host. Users should still confirm that any externally-managed firewall rules explicitly allow traffic to/from pod and service networks, but this returns the old behavior that was relied upon by some users.

The embedded kube-router controller has been updated to fix a regression that caused traffic from pods to be blocked by any default drop/deny rules present on the host. Users should still confirm that any externally-managed firewall rules explicitly allow traffic to/from pod and service networks, but this returns the old behavior that was relied upon by some users.

The agent tunnel authorizer now waits for the kubelet to be ready before reading the kubelet port from the node object.

The k3s certificate rotate-ca checks now support rotating self-signed certificates without the --force option.

MultiClusterCIDR feature

The embedded cri-dockerd has been updated to v0.3.1

ServiceLB now honors the Service’s ExternalTrafficPolicy. When set to Local, the LoadBalancer will only advertise addresses of Nodes with a Pod for the Service, and will not forward traffic to other cluster members.

K3s now functions properly when the cluster CA certificates are signed by an existing root or intermediate CA. You can find a sample script for generating such certificates before K3s starts in the github repo at contrib/util/certs.sh.

K3s now supports kubeadm style join tokens. k3s token create now creates join token secrets, optionally with a limited TTL.

K3s agents joined with an expired or deleted token stay in the cluster using existing client certificates via the NodeAuthorization admission plugin, unless their Node object is deleted from the cluster.

The apiserver advertised address and IP SAN entry are now set correctly on clusters that use IPv6 as the default IP family.

The embedded cloud controller manager will no longer attempt to unconditionally re-create its namespace and serviceaccount on startup. This resolves an issue that could cause a deadlocked cluster when fail-closed webhooks are in use.

Fixed an issue that would cause the apiserver egress proxy to attempt to use the agent tunnel to connect to service endpoints even in agent or disabled mode.

Fixed an issue where leader-elected controllers for managed etcd did not run on etcd-only nodes

Resolved an issue with ServiceLB that would cause it to advertise node IPv6 addresses, even if the cluster or service was not enabled for dual-stack operation.

The embedded kine version has been bumped to v0.9.9. Compaction log messages are now omitted at info level for increased visibility.

Scheduled etcd snapshots are now offset by a short random delay of up to several seconds. This should prevent multi-server clusters from executing pathological behavior when attempting to simultaneously update the snapshot list ConfigMap. The snapshot controller will also be more persistent in attempting to update the snapshot list.

The embedded containerd version has been bumped to v1.6.15-k3s1

The K3s default cipher suites are now explicitly passed in to kube-apiserver, ensuring that all listeners use these values.

The embedded containerd version has been bumped to v1.6.14-k3s1. This includes a backported fix for containerd/7843 which caused pods to lose their CNI info when containerd was restarted, which in turn caused the kubelet to recreate the pod.

Update kubernetes to v1.26.0-k3s1

Update cri-tools to v1.26.0-rc.0-k3s1

Update helm controller to v0.13.1

Update etcd to v3.5.5-k3s1

Update cri-dockerd to the latest 1.26.0

Update cadvisor

Update containerd to v1.6.12-k3s1 (#6370)