Upgrade Lifecycle

Upgrade Flow

Components

The SUSE® Rancher Prime OS Manager Operator supports several ways to configure the OS version on a Cluster level.
Whenever a new ManagedOSIMage is defined, a related Fleet Bundle is also generated by the SUSE® Rancher Prime OS Manager Operator.

In this context, Fleet is used to apply an upgrade.cattle.io/v1 Plan on each Cluster targeted by the ManagedOSImage.
The Plan is then executed by the System Upgrade Controller running on the downstream SUSE® Rancher Prime OS Manager Cluster.

The System Upgrade Controller is an upgrade tool used by k3s and rke2 clusters. It should always be automatically installed on each provisioned SUSE® Rancher Prime OS Manager cluster.
In this context the System Upgrade Controller orchestrates the application of upgrade.cattle.io/v1 Plan to the SUSE® Rancher Prime OS Manager nodes.

Versioning and components lifecycle

The Fleet and System Upgrade Controller versions are controlled by Rancher.

On the Rancher cluster, you can run:

kubectl get settings fleet-version system-upgrade-controller-chart-version

For more information about chart versions, you can visit the repository or read the documentation.

The charts version is determined by Rancher.
Depending on how Rancher is installed, the following environment variables can be set, for example when installing the Rancher Helm chart:

CATTLE_SYSTEM_UPGRADE_CONTROLLER_CHART_VERSION
CATTLE_FLEET_VERSION

Manually updating the versions is generally not recommended.
Also beware that a new CATTLE_SYSTEM_UPGRADE_CONTROLLER_CHART_VERSION will trigger a system-upgrade-controller chart update on all provisioned downstream clusters.

Verifying successful upgrades

Verifying the correct execution of the upgrade jobs is a fundamental part of ensuring the SUSE® Rancher Prime OS Manager upgrade process is complete.

To troubleshoot each step within the entire upgrade process, please consult the related document.
This section only describes the last step needed to verify the correct application of the upgrade Plan on one targeted cluster.

  • Versions higher or equal to v0.13.4 of the system-upgrade-controller:

    kubectl -n cattle-system describe plan os-upgrader-my-upgrade

    Once the Plan is applied to all nodes, it will have a Complete status condition.

  • Older versions:

    Each Plan has a latestHash status value that should match the Plan label on each node.
    A simple script can be used to list all nodes where the plan has not be applied yet:

    #!/bin/bash

# This script prints a list of all nodes where an upgrade plan was not applied.
# To further determine the cause of failures, you can analyze the Plan status and the related jobs.
# For ex: kubectl -n cattle-system get plans,jobs

# Edit this variable according to your ManagedOSImage name.
MANAGED_OS_IMAGE_NAME=my-upgrade

PLAN_NAME=os-upgrader-$MANAGED_OS_IMAGE_NAME
LATEST_HASH=$(kubectl -n cattle-system get plan $PLAN_NAME -o=jsonpath='{.status.latestHash}')

printf "Plan '$PLAN_NAME' latest hash is: '$LATEST_HASH'\n"

PLAN_LABEL=plan.upgrade.cattle.io/$PLAN_NAME
printf "Listing all nodes with mismatching or missing label '$PLAN_LABEL':\n"
kubectl get nodes -l $PLAN_LABEL!=$LATEST_HASH