28 Downstream clusters #

Note

For use-cases where you want to utilize Fleet (Chapter 6, Fleet) without Rancher, you can skip the Rancher component altogether.

Responsible for the management of your downstream clusters. Should be deployed on your management cluster.

For more information, see Chapter 4, Rancher.

Responsible for multi-cluster resource deployment.

Typically offered by the Rancher component. For use-cases where Rancher is not used, can be deployed as a standalone component.

For more information on installing Fleet as a standalone component, see Fleet’s Installation Details.

For more information regarding the Fleet component, see Chapter 6, Fleet.

Important

This documentation heavily relies on Fleet and more specifically on the GitRepo and Bundle resources (more on this in Section 28.1.2, “Determine your use-case”) for establishing a GitOps way of automating the deployment of resources related to Day 2 operations.

For use-cases, where a third party GitOps tool usage is desired, see:

1. For OS upgrades - Section 28.2.4.3, “SUC Plan deployment - third-party GitOps workflow”

2. For Kubernetes distribution upgrades - Section 28.3.4.3, “SUC Plan deployment - third-party GitOps workflow”

3. For EIB deployed Helm chart upgrades - Section 28.4.3.3.4, “Helm chart upgrade using a third-party GitOps tool”

4. For non-EIB deployed Helm chart upgrades - retrieve the chart version supported by the desired Edge release from the Section 36.1, “Abstract” page and populate the chart version and URL in your third party GitOps tool

System Upgrade Controller (SUC) is responsible for executing tasks on specified nodes based on configuration data provided through a custom resource, called a Plan.

Note

In order for SUC to be able to support different Day 2 operations, it is important that it is deployed on each downstream cluster that requires an upgrade.

For more information about the SUC component and how it fits in the Edge stack, see the System Upgrade Controller (Chapter 19, System Upgrade Controller) component documentation.

For information on how to deploy SUC on your downstream clusters, first determine your use-case (Section 28.1.2, “Determine your use-case”) and then refer to System Upgrade Controller installation - GitRepo (Section 19.2.1.1, “System Upgrade Controller installation - GitRepo”), or System Upgrade Controller installation - Bundle (Section 19.2.1.2, “System Upgrade Controller installation - Bundle”).

A GitRepo is a Fleet (Chapter 6, Fleet) resource that represents a Git repository from which Fleet can create Bundles. Each Bundle is created based on configuration paths defined inside of the GitRepo resource. For more information, see the GitRepo documentation.

In terms of Day 2 operations, GitRepo resources are normally used to deploy SUC or SUC Plans on non air-gapped environments that utilize a Fleet GitOps approach.

Alternatively, GitRepo resources can also be used to deploy SUC or SUC Plans on air-gapped environments, if you mirror your repository setup through a local git server.

Bundles hold raw Kubernetes resources that will be deployed on the targeted cluster. Usually they are created from a GitRepo resource, but there are use-cases where they can be deployed manually. For more information refer to the Bundle documentation.

In terms of Day 2 operations, Bundle resources are normally used to deploy SUC or SUC Plans on air-gapped environments that do not use some form of local GitOps procedure (e.g. a local git server).

Alternatively, if your use-case does not allow for a GitOps workflow (e.g. using a Git repository), Bundle resources could also be used to deploy SUC or SUC Plans on non air-gapped environments.

A different systemd.service is created depending on what upgrade your OS requires from one Edge version to another:

Shipped through a SUC plan, which should be located on each downstream cluster that is in need of an OS upgrade.

This section aims to describe the full workflow that the OS upgrade process goes through from start to finish.

Figure 28.1: OS upgrade workflow #

 

OS upgrade steps:

The OS upgrade procedure finishes with the system reboot. After the reboot, the OS package versions are upgraded and if the Edge release requires it, the OS might be migrated as well.

A GitRepo resource, that ships the needed OS upgrade SUC Plans, can be deployed in one of the following ways:

Once deployed, to monitor the OS upgrade process of the nodes of your targeted cluster, refer to the Section 19.3, “Monitoring System Upgrade Controller Plans” documentation.

28.2.4.1.1 GitRepo creation - Rancher UI #

 

To create a GitRepo resource through the Rancher UI, follow their official documentation.

The Edge team maintains a ready to use fleet that users can add as a path for their GitRepo resource.

Important

Always use this fleet from a valid Edge release tag.

For use-cases where no custom tolerations need to be included to the SUC plans that the fleet ships, users can directly refer the os-upgrade fleet from the suse-edge/fleet-examples repository.

In cases where custom tolerations are needed, users should refer the os-upgrade fleet from a separate repository, allowing them to add the tolerations to the SUC plans as required.

An example of how a GitRepo can be configured to use the fleet from the suse-edge/fleet-examples repository, can be viewed here.

A Bundle resource, that ships the needed OS upgrade SUC Plans, can be deployed in one of the following ways:

Once deployed, to monitor the OS upgrade process of the nodes of your targeted cluster, refer to the Section 19.3, “Monitoring System Upgrade Controller Plans” documentation.

28.2.4.2.1 Bundle creation - Rancher UI #

 

The Edge team maintains a ready to use bundle that can be used in the below steps.

Important

Always use this bundle from a valid Edge release tag.

To create a bundle through Rancher’s UI:

1. In the upper left corner, click ☰ → Continuous Delivery

2. Go to Advanced > Bundles

3. Select Create from YAML

4. From here you can create the Bundle in one of the following ways:

   

   Note

   There might be use-cases where you would need to include custom tolerations to the SUC plans that the bundle ships. Make sure to include those tolerations in the bundle that will be generated by the below steps.

   1. By manually copying the bundle content from suse-edge/fleet-examples to the Create from YAML page.

   2. By cloning the suse-edge/fleet-examples repository from the desired release tag and selecting the Read from File option in the Create from YAML page. From there, navigate to the bundle location (bundles/day2/system-upgrade-controller-plans/os-upgrade) and select the bundle file. This will auto-populate the Create from YAML page with the bundle content.

5. Change the target clusters for the Bundle:

   • To match all downstream clusters change the default Bundle .spec.targets to:

     spec:
       targets:
       - clusterSelector: {}

   • For a more granular downstream cluster mappings, see Mapping to Downstream Clusters.

6. Select Create

There might be use-cases where users would like to incorporate the OS upgrade SUC Plans to their own third-party GitOps workflow (e.g. Flux).

To get the OS upgrade resources that you need, first determine the Edge release tag of the suse-edge/fleet-examples repository that you would like to use.

After that, resources can be found at fleets/day2/system-upgrade-controller-plans/os-upgrade, where:

Important

These Plan resources are interpreted by the system-upgrade-controller and should be deployed on each downstream cluster that you wish to upgrade. For information on how to deploy the system-upgrade-controller, see Section 19.2, “Installing the System Upgrade Controller”.

To better understand how your GitOps workflow can be used to deploy the SUC Plans for OS upgrade, it can be beneficial to take a look at the overview (Section 28.2.3.1, “Overview”) of the update procedure using Fleet.

Image responsible for upgrading the RKE2 version of a specific node.

Shipped through a Pod created by SUC based on a SUC Plan. The Plan should be located on each downstream cluster that is in need of a RKE2 upgrade.

For more information regarding how the rke2-upgrade image performs the upgrade, see the upstream documentation.

Image responsible for upgrading the K3s version of a specific node.

Shipped through a Pod created by SUC based on a SUC Plan. The Plan should be located on each downstream cluster that is in need of a K3s upgrade.

For more information regarding how the k3s-upgrade image performs the upgrade, see the upstream documentation.

This section aims to describe the full workflow that the Kubernetes version upgrade process goes through from start to finish.

Figure 28.2: Kubernetes version upgrade workflow #

 

Kubernetes version upgrade steps:

With the above steps executed, the Kubernetes version of each cluster node should have been upgraded to the desired Edge compatible release.

A GitRepo resource, that ships the needed Kubernetes upgrade SUC Plans, can be deployed in one of the following ways:

Once deployed, to monitor the Kubernetes upgrade process of the nodes of your targeted cluster, refer to the Section 19.3, “Monitoring System Upgrade Controller Plans” documentation.

28.3.4.1.1 GitRepo creation - Rancher UI #

 

To create a GitRepo resource through the Rancher UI, follow their official documentation.

The Edge team maintains ready to use fleets for both rke2 and k3s Kubernetes distributions, that users can add as a path for their GitRepo resource.

Important

Always use this fleets from a valid Edge release tag.

For use-cases where no custom tolerations need to be included to the SUC plans that these fleets ship, users can directly refer the fleets from the suse-edge/fleet-examples repository.

In cases where custom tolerations are needed, users should refer the fleets from a separate repository, allowing them to add the tolerations to the SUC plans as required.

Configuration examples for a GitRepo resource using the fleets from suse-edge/fleet-examples repository:

• RKE2

• K3s

A Bundle resource, that ships the needed Kubernetes upgrade SUC Plans, can be deployed in one of the following ways:

Once deployed, to monitor the Kubernetes upgrade process of the nodes of your targeted cluster, refer to the Section 19.3, “Monitoring System Upgrade Controller Plans” documentation.

28.3.4.2.1 Bundle creation - Rancher UI #

 

The Edge team maintains ready to use bundles for both rke2 and k3s Kubernetes distributions that can be used in the below steps.

Important

Always use this bundle from a valid Edge release tag.

To create a bundle through Rancher’s UI:

1. In the upper left corner, click ☰ → Continuous Delivery

2. Go to Advanced > Bundles

3. Select Create from YAML

4. From here you can create the Bundle in one of the following ways:

   

   Note

   There might be use-cases where you would need to include custom tolerations to the SUC plans that the bundle ships. Make sure to include those tolerations in the bundle that will be generated by the below steps.

   1. By manually copying the bundle content for RKE2 or K3s from suse-edge/fleet-examples to the Create from YAML page.

   2. By cloning the suse-edge/fleet-examples repository from the desired release tag and selecting the Read from File option in the Create from YAML page. From there, navigate to the bundle that you need (bundles/day2/system-upgrade-controller-plans/rke2-upgrade/plan-bundle.yaml for RKE2 and bundles/day2/system-upgrade-controller-plans/k3s-upgrade/plan-bundle.yaml for K3s). This will auto-populate the Create from YAML page with the bundle content.

5. Change the target clusters for the Bundle:

   • To match all downstream clusters change the default Bundle .spec.targets to:

     spec:
       targets:
       - clusterSelector: {}

   • For a more granular downstream cluster mappings, see Mapping to Downstream Clusters.

6. Create

There might be use-cases where users would like to incorporate the Kubernetes upgrade resources to their own third-party GitOps workflow (e.g. Flux).

To get the upgrade resources that you need, first determine the Edge release tag of the suse-edge/fleet-examples repository that you would like to use.

After that, the resources can be found at:

Important

These Plan resources are interpreted by the system-upgrade-controller and should be deployed on each downstream cluster that you wish to upgrade. For information on how to deploy the system-upgrade-controller, see Section 19.2, “Installing the System Upgrade Controller”.

To better understand how your GitOps workflow can be used to deploy the SUC Plans for Kubernetes version upgrade, it can be beneficial to take a look at the overview (Section 28.3.3.1, “Overview”) of the update procedure using Fleet.

Depending on what your environment supports, you can take one of the following options:

On a machine with internet access:

On a machine with internet access:

On your air-gapped machine:

On your air-gapped machine:

For RKE2, see Containerd Registry Configuration

For K3s, see Embedded Registry Mirror

For users that want to manage their Helm chart lifecycle through Fleet.

This section covers how to:

28.4.3.1.2 Deploy your Fleet #

 

If the environment supports working with a GitOps workflow, you can deploy your Chart Fleet by either using a GitRepo (Section 28.4.3.1.2.1, “GitRepo”) or Bundle (Section 28.4.3.1.2.2, “Bundle”).

Note

While deploying your Fleet, if you get a Modified message, make sure to add a corresponding comparePatches entry to the Fleet’s diff section. For more information, see Generating Diffs to Ignore Modified GitRepos.

28.4.3.1.2.1 GitRepo #

 

Fleet’s GitRepo resource holds information on how to access your chart’s Fleet resources and to which clusters it needs to apply those resources.

The GitRepo resource can be deployed through the Rancher UI, or manually, by deploying the resource to the management cluster.

Example Longhorn GitRepo resource for manual deployment:

apiVersion: fleet.cattle.io/v1alpha1
kind: GitRepo
metadata:
  name: longhorn-git-repo
  namespace: fleet-default
spec:
  # If using a tag
  # revision: <user_repository_tag>
  #
  # If using a branch
  # branch: <user_repository_branch>
  paths:
  # As seen in the 'Prepare your Fleet resources' example
  - longhorn
  - longhorn-crd
  repo: <user_repository_url>
  targets:
  # Match all clusters
  - clusterSelector: {}

28.4.3.1.2.2 Bundle #

 

Bundle resources hold the raw Kubernetes resources that need to be deployed by Fleet. Normally it is encouraged to use the GitRepo approach, but for use-cases where the environment is air-gapped and cannot support a local Git server, Bundles can help you in propagating your Helm chart Fleet to your target clusters.

The Bundle can be deployed either through the Rancher UI (Continuous Delivery → Advanced → Bundles → Create from YAML) or by manually deploying the Bundle resource in the correct Fleet namespace. For information about Fleet namespaces, see the upstream documentation.

Example Longhorn Bundle resource deployment using a manual approach:

1. Navigate to the Longhorn Chart fleet located under fleets/day2/chart-templates/longhorn/longhorn:

   cd fleets/day2/chart-templates/longhorn/longhorn

2. Create a targets.yaml file that will instruct Fleet to which clusters it should deploy the Helm chart. In this case, we will deploy to a single downstream cluster. For information on how to map more complex targets, see Mapping to Downstream Clusters:

   cat > targets.yaml <<EOF
   targets:
   - clusterName: foo
   EOF

3. Convert the Longhorn Helm chart Fleet to a Bundle resource. For more information, see Convert a Helm Chart into a Bundle:

   fleet apply --compress --targets-file=targets.yaml -n fleet-default -o - longhorn-bundle > longhorn-bundle.yaml

4. Navigate to the Longhorn CRD Chart fleet located under fleets/day2/chart-templates/longhorn/longhorn-crd:

   cd fleets/day2/chart-templates/longhorn/longhorn-crd

5. Create a targets.yaml file that will instruct Fleet to which clusters it should deploy the Helm chart. In this case, we will deploy to a single downstream cluster. For information on how to map more complex targets, see Mapping to Downstream Clusters:

   cat > targets.yaml <<EOF
   targets:
   - clusterName: foo
   EOF

6. Convert the Longhorn CRD Helm chart Fleet to a Bundle resource. For more information, see Convert a Helm Chart into a Bundle:

   fleet apply --compress --targets-file=targets.yaml -n fleet-default -o - longhorn-crd-bundle > longhorn-crd-bundle.yaml

7. Deploy longhorn-bundle.yaml and longhorn-crd-bundle.yaml to your management cluster:

   kubectl apply -f longhorn-crd-bundle.yaml
   kubectl apply -f longhorn-bundle.yaml

Following these steps will ensure that Longhorn is deployed on all of the specified target clusters.

EIB deploys Helm charts by creating a HelmChart resource and utilizing the helm-controller introduced by the RKE2/K3s Helm integration feature.

To ensure that an EIB deployed Helm chart is successfully upgraded, users need to do an upgrade over the HelmChart resources created for the Helm chart by EIB.

Below you can find information on:

28.4.3.3.1 Overview #

 

This section is meant to give a high overview of the steps that need to be taken in order to upgrade one or multiple Helm charts that have been deployed by EIB. For a detailed explanation of the steps needed for a Helm chart upgrade, see Section 28.4.3.3.2, “Upgrade Steps”.

Figure 28.3: Helm chart upgrade workflow #

 

1. The workflow begins with the user pulling the new Helm chart archive(s) that he wishes to upgrade his chart(s) to.

2. The archive(s) should then be placed in a directory that will be processed by the generate-chart-upgrade-data.sh script.

3. The user then proceeds to execute the generate-chart-upgrade-data.sh script which will generate a Kubernetes Secret YAML file for each Helm chart archive in the provided archive directory. These secrets will be automatically placed under the Fleet that will be used to upgrade the Helm charts. This is further explained in the upgrade steps (Section 28.4.3.3.2, “Upgrade Steps”) section.

4. After the script finishes successfully, the user should continue to the configuration and deployment of either a Bundle or a GitRepo resource that will ship all the needed K8s resources to the target clusters.

   1. The resource is deployed on the management cluster under the fleet-default namespace.

5. Fleet (Chapter 6, Fleet) detects the deployed resource, parses its data and deploys its resources to the specified target clusters. The most notable resources that are deployed are:

   1. eib-charts-upgrader - a Job that deploys the Chart Upgrade Pod. The eib-charts-upgrader-script as well as all helm chart upgrade data secrets are mounted inside of the Chart Upgrade Pod.

   2. eib-charts-upgrader-script - a Secret shipping the script that will be used by the Chart Upgrade Pod to patch an existing HelmChart resource.

   3. Helm chart upgrade data secrets - Secret YAML files created by the generate-chart-upgrade-data.sh script based on the user provided data. Secret YAML files should not be edited.

6. Once the Chart Upgrade Pod has been deployed, the script from the eib-charts-upgrader-script secret is executed, which does the following:

   1. Process all the Helm chart upgrade data provided by the other secrets.

   2. Check if there is a HelmChart resource for each of the provided chart upgrade data.

   3. Proceed to patch the HelmChart resource with the data provided from the secret for the corresponding Helm chart.

7. RKE2/K3s helm-controller constantly monitors for edits over the existing HelmChart resource. It detects the patch of the HelmChart, reconciles the changes and then proceeds to upgrade the chart behind the HelmChart resource.

28.4.3.3.2 Upgrade Steps #

 

1. Clone the suse-edge/fleet-examples repository from the Edge release tag that you wish to use.

2. Create a directory in which you will store the pulled Helm chart archive(s).

   mkdir archives

3. Inside of the newly created archive directory, pull the Helm chart archive(s) that you wish to upgrade to:

   cd archives
   helm pull [chart URL | repo/chartname]
   
   # Alternatively if you want to pull a specific version:
   # helm pull [chart URL | repo/chartname] --version 0.0.0

4. From the desired release tag download the generate-chart-upgrade-data.sh script.

5. Execute the generate-chart-upgrade-data.sh script:

   

   Important

   Users should not make any changes over what the generate-chart-upgrade-data.sh script generates.

   chmod +x ./generate-chart-upgrade-data.sh
   
   ./generate-chart-upgrade-data.sh --archive-dir /foo/bar/archives/ --fleet-path /foo/bar/fleet-examples/fleets/day2/eib-charts-upgrader

   The script will go through the following logic:

   1. Validate that the user has provided --fleet-path points to a valid Fleet that can initiate a Helm chart upgrade.

   2. Process all Helm chart archives from the user-created archives dir (e.g. /foo/bar/archives/).

   3. For each Helm chart archive create a Kubernetes Secret YAML resource. This resource will hold:

      1. The name of the HelmChart resource that needs to be patched.

      2. The new version for the HelmChart resource.

      3. The base64 encoded Helm chart archive that will be used to replace the HelmChart’s currently running configuration.

   4. Each Kubernetes Secret YAML resource will be transferred to the base/secrets directory inside of the path to the eib-charts-upgrader Fleet that was given under --fleet-path.

   5. Furthermore the generate-chart-upgrade-data.sh script ensures that the secrets that it moved will be picked up and used in the Helm chart upgrade logic. It does that by:

      1. Editing the base/secrets/kustomization.yaml file to include the newly added resources.

      2. Edit the base/patches/job-patch.yaml file to include the newly added secrets to the mount configurations.

6. After a successful generate-chart-upgrade-data.sh run you should have the changes inside of the following directories of the suse-edge/fleet-examples repository:

   1. fleets/day2/eib-charts-upgrader/base/patches

   2. fleets/day2/eib-charts-upgrader/base/secrets

The steps below depend on the environment that you are running:

1. For an environment that supports GitOps (e.g. is non air-gapped, or is air-gapped, but allows for local Git server support):

   1. Copy the fleets/day2/eib-charts-upgrader Fleet to the repository that you will use for GitOps. Make sure that the Fleet includes the changes that have been made by the generate-chart-upgrade-data.sh script.

   2. Configure a GitRepo resource that will be used to ship all the resources of the eib-charts-upgrader Fleet.

      1. For GitRepo configuration and deployment through the Rancher UI, see Accessing Fleet in the Rancher UI.

      2. For GitRepo manual configuration and deployment, see Creating a Deployment.

2. For an environment that does not support GitOps (e.g. is air-gapped and does not allow local Git server usage):

   1. Download the fleet-cli binary from the rancher/fleet releases page. For Mac users, there is a Homebrew Formulae that can be used - fleet-cli.

   2. Navigate to the eib-charts-upgrader Fleet:

      cd /foo/bar/fleet-examples/fleets/day2/eib-charts-upgrader

   3. Create a targets.yaml file that will instruct Fleet where to deploy your resources:

      cat > targets.yaml <<EOF
      targets:
      - clusterSelector: {} # Change this with your target data
      EOF

      For information on how to map target clusters, see the upstream documentation.

   4. Use the fleet-cli to convert the Fleet to a Bundle resource:

      fleet apply --compress --targets-file=targets.yaml -n fleet-default -o - eib-charts-upgrade > bundle.yaml

      This will create a Bundle (bundle.yaml) that will hold all the templated resource from the eib-charts-upgrader Fleet.

      For more information regarding the fleet apply command, see fleet apply.

      For more information regarding converting Fleets to Bundles, see Convert a Helm Chart into a Bundle.

   5. Deploy the Bundle. This can be done in one of two ways:

      1. Through Rancher’s UI - Navigate to Continuous Delivery → Advanced → Bundles → Create from YAML and either paste the bundle.yaml contents, or click the Read from File option and pass the file itself.

      2. Manually - Deploy the bundle.yaml file manually inside of your management cluster.

Executing these steps will result in a successfully deployed GitRepo/Bundle resource. The resource will be picked up by Fleet and its contents will be deployed onto the target clusters that the user has specified in the previous steps. For an overview of the process, refer to the overview (Section 28.4.3.3.1, “Overview”) section.

For information on how to track the upgrade process, you can refer to the Example (Section 28.4.3.3.3, “Example”) section of this documentation.

Important

Once the chart upgrade has been successfully verified, remove the Bundle/GitRepo resource.

This will remove the no longer necessary upgrade resources from your downstream cluster, ensuring that no future version clashes might occur.

28.4.3.3.3 Example #

 

Note

The example below illustrates how to do an upgrade of an EIB deployed Helm chart from one version to another. The versions in the example should not be treated as version recommendations. Version recommendations for a specific Edge release, should be taken from the release notes (Section 36.1, “Abstract”).

Use-case:

• A cluster named doc-example is running Ranchers' Longhorn 103.3.0+up1.6.1 version.

• The cluster has been deployed through EIB, using the following image definition snippet:

  kubernetes:
    helm:
      charts:
      - name: longhorn-crd
        repositoryName: rancher-charts
        targetNamespace: longhorn-system
        createNamespace: true
        version: 103.3.0+up1.6.1
      - name: longhorn
        repositoryName: rancher-charts
        targetNamespace: longhorn-system
        createNamespace: true
        version: 103.3.0+up1.6.1
      repositories:
      - name: rancher-charts
        url: https://charts.rancher.io/
  ...

  

  Figure 28.4: doc-example installed Longhorn version #

   

• Longhorn needs to be upgraded to a version that is compatible with the Edge 3.1 release. Meaning it needs to be upgraded to 104.2.0+up1.7.1.

• It is assumed that the management cluster in charge of managing the doc-example cluster is air-gapped, without support for a local Git server and has a working Rancher setup.

Follow the Upgrade Steps (Section 28.4.3.3.2, “Upgrade Steps”):

1. Clone the suse-edge/fleet-example repository from the release-3.1.1 tag.

   git clone -b release-3.1.1 https://github.com/suse-edge/fleet-examples.git

2. Create a directory where the Longhorn upgrade archive will be stored.

   mkdir archives

3. Pull the desired Longhorn chart archive version:

   # First add the Rancher Helm chart repository
   helm repo add rancher-charts https://charts.rancher.io/
   
   # Pull the Longhorn 1.7.1 CRD archive
   helm pull rancher-charts/longhorn-crd --version 104.2.0+up1.7.1
   
   # Pull the Longhorn 1.7.1 chart archive
   helm pull rancher-charts/longhorn --version 104.2.0+up1.7.1

4. Outside of the archives directory, download the generate-chart-upgrade-data.sh script from the release-3.1.1 release tag.

5. Directory setup should look similar to:

   .
   ├── archives
   |   ├── longhorn-104.2.0+up1.7.1.tgz
   │   └── longhorn-crd-104.2.0+up1.7.1.tgz
   ├── fleet-examples
   ...
   │   ├── fleets
   │   │   ├── day2
   |   |   |   ├── ...
   │   │   │   ├── eib-charts-upgrader
   │   │   │   │   ├── base
   │   │   │   │   │   ├── job.yaml
   │   │   │   │   │   ├── kustomization.yaml
   │   │   │   │   │   ├── patches
   │   │   │   │   │   │   └── job-patch.yaml
   │   │   │   │   │   ├── rbac
   │   │   │   │   │   │   ├── cluster-role-binding.yaml
   │   │   │   │   │   │   ├── cluster-role.yaml
   │   │   │   │   │   │   ├── kustomization.yaml
   │   │   │   │   │   │   └── sa.yaml
   │   │   │   │   │   └── secrets
   │   │   │   │   │       ├── eib-charts-upgrader-script.yaml
   │   │   │   │   │       └── kustomization.yaml
   │   │   │   │   ├── fleet.yaml
   │   │   │   │   └── kustomization.yaml
   │   │   │   └── ...
   │   └── ...
   └── generate-chart-upgrade-data.sh

6. Execute the generate-chart-upgrade-data.sh script:

   # First make the script executable
   chmod +x ./generate-chart-upgrade-data.sh
   
   # Then execute the script
   ./generate-chart-upgrade-data.sh --archive-dir ./archives --fleet-path ./fleet-examples/fleets/day2/eib-charts-upgrader

   The directory structure after the script execution should look similar to:

   .
   ├── archives
   |   ├── longhorn-104.2.0+up1.7.1.tgz
   │   └── longhorn-crd-104.2.0+up1.7.1.tgz
   ├── fleet-examples
   ...
   │   ├── fleets
   │   │   ├── day2
   │   │   │   ├── ...
   │   │   │   ├── eib-charts-upgrader
   │   │   │   │   ├── base
   │   │   │   │   │   ├── job.yaml
   │   │   │   │   │   ├── kustomization.yaml
   │   │   │   │   │   ├── patches
   │   │   │   │   │   │   └── job-patch.yaml
   │   │   │   │   │   ├── rbac
   │   │   │   │   │   │   ├── cluster-role-binding.yaml
   │   │   │   │   │   │   ├── cluster-role.yaml
   │   │   │   │   │   │   ├── kustomization.yaml
   │   │   │   │   │   │   └── sa.yaml
   │   │   │   │   │   └── secrets
   │   │   │   │   │       ├── eib-charts-upgrader-script.yaml
   │   │   │   │   │       ├── kustomization.yaml
   │   │   │   │   │       ├── longhorn-104-2-0-up1-7-1.yaml <- secret created by the generate-chart-upgrade-data.sh script
   │   │   │   │   │       └── longhorn-crd-104-2-0-up1-7-1.yaml <- secret created by the generate-chart-upgrade-data.sh script
   │   │   │   │   ├── fleet.yaml
   │   │   │   │   └── kustomization.yaml
   │   │   │   └── ...
   │   └── ...
   └── generate-chart-upgrade-data.sh

   The files changed in git should look like this:

   

   Figure 28.5: Changes over fleet-examples made by generate-chart-upgrade-data.sh #

    

7. Since the management cluster does not support for a GitOps workflow, a Bundle needs to be created for the eib-charts-upgrader Fleet:

   1. First, navigate to the Fleet itself:

      cd ./fleet-examples/fleets/day2/eib-charts-upgrader

   2. Then create a targets.yaml file targeting the doc-example cluster:

      cat > targets.yaml <<EOF
      targets:
      - clusterName: doc-example
      EOF

   3. Then use the fleet-cli binary to convert the Fleet to a Bundle:

      fleet apply --compress --targets-file=targets.yaml -n fleet-default -o - eib-charts-upgrade > bundle.yaml

   4. Now, transfer the bundle.yaml on your management cluster machine.

8. Since the management cluster is running Rancher, deploy the Bundle through the Rancher UI:

   

   Figure 28.6: Deploy Bundle through Rancher UI #

    

   From here, select Read from File and find the bundle.yaml file on your system.

   This will auto-populate the Bundle inside of Rancher’s UI:

   

   Figure 28.7: Auto-populated Bundle snippet #

    

   Select Create.

9. After a successful deployment, your Bundle would look similar to:

   

   Figure 28.8: Successfully deployed Bundle #

    

After the successful deployment of the Bundle, to monitor the upgrade process:

1. First, verify the logs of the Upgrade Pod:

   

   Figure 28.9: View the upgrade pod logs #

    

2. Now verify the logs of the Pod created for the upgrade by the helm-controller:

   1. The Pod name will be with the following template - helm-install-longhorn-<random-suffix>

   2. The Pod will be in the namespace where the HelmChart resource was deployed. In our case this is default.

      

      Figure 28.10: Logs for successfully upgraded Longhorn chart #

       

3. Check that the HelmChart version has been bumped:

   

   Figure 28.11: Bumped Longhorn version #

    

4. Finally check that the Longhorn Pods are running:

   

   Figure 28.12: Example for validating the instance-manager pod #

    

After making the above validations, it is safe to assume that the Longhorn Helm chart has been upgraded from 103.3.0+up1.6.1 to 104.2.0+up1.7.1.

cd /foo/bar/fleets/day2/eib-charts-upgrader

kustomize build .