Rollbacks

This page outlines how to roll back Rancher to a previous version after an upgrade.

  • This is only possible by restoring to a previous version from a backup instance. Changing versions using kubectl or Helm is not supported.

  • When rolling back, ensure the version being restored is under active support. Rolling back to versions that are end-of-life is not supported. Refer to SUSE’s Product Support Lifecycle page to verify key lifecycle dates for your given version.

Follow the instructions from this page when:

  • The running Rancher instance has been upgraded to a newer version after the backup was made.

  • The upstream (local) cluster is the same as where the backup was made.

To roll back Rancher to a lower/previous version, use the Rancher Backups application and restore Rancher from backup.

Rancher has to be started with the lower/previous version after a rollback.

A restore is performed by creating a Restore custom resource.

Alternative Steps for Special Scenarios

Alternative steps need to be performed for rollbacks in the following scenarios:

  • Rolling back from v2.14.0 and later to an earlier version of v2.13.x.

In Rancher v2.13.0, Rancher Turtles became the default manager for CAPI resources, replacing the previously embedded cluster-api controllers, and in Rancher v2.14.0 the embedded cluster-api was removed entirely. As a result, if you roll back from Rancher v2.14.0 and later to an earlier version of Rancher v2.13.x and do not intend to continue using Rancher Turtles to manage CAPI resources, additional manual steps may be required to use the embedded cluster-api controllers. From Rancher v2.14.0 onward, Rancher Turtles is the only supported manager for CAPI resources.

In Rancher v2.14.0, the cluster-api module is upgraded from v1.10.6 to v1.12.2. The cluster-api v1.12.2, in turn, upgrades the apiVersions of its Custom Resource Definitions (CRDs) from cluster.x-k8s.io/v1beta1 to cluster.x-k8s.io/v1beta2. Rancher backup files include Cluster API CRDs. When restoring backup data from Rancher v2.13.x to a local cluster after upgrading to v2.14.0, the Rancher Backup application first restores the v1beta1 CRDs. This fails because the v1beta2 version cannot be removed from the CRDs while v1beta2 custom resources are present in the cluster.

Important:

  • Follow the instructions from this page for restoring Rancher on the same cluster where it was backed up from. In order to migrate Rancher to a new cluster, follow the steps to migrate Rancher.

  • While restoring Rancher on the same setup, the Rancher deployment is manually scaled down before the restore starts, then the operator will scale it back up once the restore completes. As a result, Rancher and its UI will be unavailable until the restore is complete. While the UI is unavailable, use the original cluster kubeconfig with the restore YAML file: kubectl create -f restore.yaml.

Step 1: Create the Restore Custom Resource

  1. Click ☰ > Cluster Management.

  2. Go to the local cluster and click Explore.

  3. In the left navigation bar, click Rancher Backups  Restore.

If the Rancher Backups app is not visible, you will need to install it from the Charts page in Apps. Refer here for more information.

  1. Click Create.

  2. Create the Restore with the form or with YAML. For help creating the Restore resource using the online form, refer to the configuration reference and to the examples.

  3. To use the YAML editor, you can click Create  Create from YAML. Enter the Restore YAML. The following is an example Restore custom resource:

     apiVersion: resources.cattle.io/v1
 kind: Restore
 metadata:
   name: restore-migration
 spec:
   backupFilename: backup-b0450532-cee1-4aa1-a881-f5f48a007b1c-2020-09-15T07-27-09Z.tar.gz
   encryptionConfigSecretName: encryptionconfig
   storageLocation:
     s3:
       credentialSecretName: s3-creds
       credentialSecretNamespace: default
       bucketName: rancher-backups
       folder: rancher
       region: us-west-2
       endpoint: s3.us-west-2.amazonaws.com

    For help configuring the Restore, refer to the configuration reference and to the examples.

  4. Click Create.

Result: The backup file is created and updated to the target storage location. The resources are restored in this order:

  1. Custom Resource Definitions (CRDs)

  2. Cluster-scoped resources

  3. Namespaced resources

To check how the restore is progressing, you can check the logs of the operator. Follow these steps to get the logs:

kubectl get pods -n cattle-resources-system
kubectl logs -n cattle-resources-system -f

Step 2: Roll Back to a Previous Rancher Version

Rancher can be rolled back using the Helm CLI. To roll back to the previous version:

helm rollback rancher -n cattle-system

If the previous revision is not the intended target, you can specify a revision to roll back to. To see the deployment history:

helm history rancher -n cattle-system

When the target revision is determined, perform the rollback. This example will roll back to revision 3:

helm rollback rancher 3 -n cattle-system