|Index|SUSE Telco Cloud 文档|全自动定向网络配置|隔离环境中的下游集群置备
适用范围 SUSE Telco Cloud 3.7

57 隔离环境中的下游集群置备

定向网络置备工作流允许自动执行隔离环境中的下游集群置备。

57.1 隔离环境的要求

  1. 使用 EIB 生成的 raw 镜像必须包含在隔离环境中运行下游集群所需的特定容器镜像(helm-chart OCI 和容器镜像)。有关详细信息,请参考 此部分 (第 51 章 “为隔离场景准备下游集群映像”)。

  2. 如果使用 SR-IOV 或任何其他自定义工作负载,则必须按照 预加载专用注册表部分 (第 51.2.7 节 “准备隔离工件”) 将运行工作负载所需的镜像预加载到您的专用注册表中。

57.2 在隔离环境中注册裸机主机

将裸机主机注册到管理集群的过程与上一节 (第 52 章 “使用定向网络配置(单节点)进行下游集群配置”)中描述的过程相同。

57.3 在隔离场景中置备下游集群

在隔离的场景中置备下游集群需要进行一些重要更改:

  1. `capi-provisioning-example.yaml`文件中的`RKE2ControlPlane`块必须包含`spec.agentConfig.airGapped: true`指令。

  2. 私有镜像仓库配置必须包含在`capi-provisioning-airgap-example.yaml`文件中的`RKE2ControlPlane`块内,并位于私有镜像仓库部分 (第 56 章 “私有镜像仓库”)之后。

  3. 如果您正在使用SR-IOV或任何其他需要安装helm-chart的`AdditionalUserData`配置(燃烧脚本),则必须修改内容以引用私有镜像仓库,而不是使用公共镜像仓库。

以下示例显示了`capi-provisioning-airgap-example.yaml`文件中`AdditionalUserData`块内的SR-IOV配置,以及引用私有镜像仓库所需的修改

  • 私有镜像仓库密钥引用

  • 使用私有镜像仓库而非公共OCI镜像的Helm-Chart定义。

# secret to include the private registry certificates
apiVersion: v1
kind: Secret
metadata:
  name: private-registry-cert
  namespace: default
data:
  tls.crt: ${TLS_BASE64_CERT}
  tls.key: ${TLS_BASE64_KEY}
  ca.crt: ${CA_BASE64_CERT}
type: kubernetes.io/tls
---
# secret to include the private registry auth credentials
apiVersion: v1
kind: Secret
metadata:
  name: private-registry-auth
  namespace: default
data:
  username: ${REGISTRY_USERNAME}
  password: ${REGISTRY_PASSWORD}
---
apiVersion: controlplane.cluster.x-k8s.io/v1beta2
kind: RKE2ControlPlane
metadata:
  name: single-node-cluster
  namespace: default
spec:
  infrastructureRef:
    apiGroup: infrastructure.cluster.x-k8s.io
    kind: Metal3MachineTemplate
    name: single-node-cluster-controlplane
  replicas: 1
  version: ${RKE2_VERSION}
  rolloutStrategy:
    type: "RollingUpdate"
    rollingUpdate:
      maxSurge: 0
  privateRegistriesConfig:       # Private registry configuration to add your own mirror and credentials
    mirrors:
      docker.io:
        endpoint:
          - "${PRIVATE_REGISTRY_URL}"
        rewrite:
          "^(.*)$": "mirror/$1"
      registry.suse.com:
        endpoint:
          - "${PRIVATE_REGISTRY_URL}"
        rewrite:
          "^(.*)$": "mirror/$1"
      registry.suse.de:
        endpoint:
          - "${PRIVATE_REGISTRY_URL}"
        rewrite:
          "^(.*)$": "mirror/$1"
      registry.opensuse.org:
        endpoint:
          - "${PRIVATE_REGISTRY_URL}"
        rewrite:
          "^(.*)$": "mirror/$1"
      registry.rancher.com:
        endpoint:
          - "${PRIVATE_REGISTRY_URL}"
        rewrite:
          "^(.*)$": "mirror/$1"
    configs:
      "192.168.100.22:5000":
        authSecret:
          apiVersion: v1
          kind: Secret
          namespace: default
          name: private-registry-auth
        tls:
          tlsConfigSecret:
            apiVersion: v1
            kind: Secret
            namespace: default
            name: private-registry-cert
          insecureSkipVerify: false
  serverConfig:
    cni: calico
    cniMultusEnable: true
  agentConfig:
    airGapped: true       # Airgap true to enable airgap mode
    format: ignition
    additionalUserData:
      config: |
        variant: fcos
        version: 1.4.0
        storage:
          files:
          - path: /var/lib/rancher/rke2/server/manifests/configmap-sriov-custom-auto.yaml
            overwrite: true
            contents:
              inline: |
                apiVersion: v1
                kind: ConfigMap
                metadata:
                  name: sriov-custom-auto-config
                  namespace: sriov-network-operator
                data:
                  config.json: |
                    [
                        {
                          "resourceName": "${RESOURCE_NAME1}",
                          "interface": "${SRIOV-NIC-NAME1}",
                          "pfname": "${PF_NAME1}",
                          "driver": "${DRIVER_NAME1}",
                          "numVFsToCreate": ${NUM_VFS1}
                        },
                        {
                          "resourceName": "${RESOURCE_NAME2}",
                          "interface": "${SRIOV-NIC-NAME2}",
                          "pfname": "${PF_NAME2}",
                          "driver": "${DRIVER_NAME2}",
                          "numVFsToCreate": ${NUM_VFS2}
                        }
                    ]
            mode: 0644
            user:
              name: root
            group:
              name: root
          - path: /var/lib/rancher/rke2/server/manifests/sriov.yaml
            overwrite: true
            contents:
              inline: |
                apiVersion: v1
                data:
                  .dockerconfigjson: ${REGISTRY_AUTH_DOCKERCONFIGJSON}
                kind: Secret
                metadata:
                  name: privregauth
                  namespace: kube-system
                type: kubernetes.io/dockerconfigjson
                ---
                apiVersion: v1
                kind: ConfigMap
                metadata:
                  namespace: kube-system
                  name: example-repo-ca
                data:
                  ca.crt: |-
                    -----BEGIN CERTIFICATE-----
                    ${CA_BASE64_CERT}
                    -----END CERTIFICATE-----
                ---
                apiVersion: helm.cattle.io/v1
                kind: HelmChart
                metadata:
                  name: sriov-crd
                  namespace: kube-system
                spec:
                  chart: oci://${PRIVATE_REGISTRY_URL}/mirror/sriov-crd
                  dockerRegistrySecret:
                    name: privregauth
                  repoCAConfigMap:
                    name: example-repo-ca
                  createNamespace: true
                  set:
                    global.clusterCIDR: 192.168.0.0/18
                    global.clusterCIDRv4: 192.168.0.0/18
                    global.clusterDNS: 10.96.0.10
                    global.clusterDomain: cluster.local
                    global.rke2DataDir: /var/lib/rancher/rke2
                    global.serviceCIDR: 10.96.0.0/12
                  targetNamespace: sriov-network-operator
                  version: 307.1.0+up1.6.0
                ---
                apiVersion: helm.cattle.io/v1
                kind: HelmChart
                metadata:
                  name: sriov-network-operator
                  namespace: kube-system
                spec:
                  chart: oci://${PRIVATE_REGISTRY_URL}/mirror/sriov-network-operator
                  dockerRegistrySecret:
                    name: privregauth
                  repoCAConfigMap:
                    name: example-repo-ca
                  createNamespace: true
                  set:
                    global.clusterCIDR: 192.168.0.0/18
                    global.clusterCIDRv4: 192.168.0.0/18
                    global.clusterDNS: 10.96.0.10
                    global.clusterDomain: cluster.local
                    global.rke2DataDir: /var/lib/rancher/rke2
                    global.serviceCIDR: 10.96.0.0/12
                  targetNamespace: sriov-network-operator
                  version: 307.1.0+up1.6.0
            mode: 0644
            user:
              name: root
            group:
              name: root
        kernel_arguments:
          should_exist:
            - intel_iommu=on
            - iommu=pt
            - idle=poll
            - mce=off
            - hugepagesz=1G hugepages=40
            - hugepagesz=2M hugepages=0
            - default_hugepagesz=1G
            - irqaffinity=${NON-ISOLATED_CPU_CORES}
            - isolcpus=domain,nohz,managed_irq,${ISOLATED_CPU_CORES}
            - nohz_full=${ISOLATED_CPU_CORES}
            - rcu_nocbs=${ISOLATED_CPU_CORES}
            - rcu_nocb_poll
            - nosoftlockup
            - nowatchdog
            - nohz=on
            - nmi_watchdog=0
            - skew_tick=1
            - quiet
        systemd:
          units:
          - name: rke2-preinstall.service
            enabled: true
            contents: |
              [Unit]
              Description=rke2-preinstall
              Wants=network-online.target
              Before=rke2-install.service
              ConditionPathExists=!/run/cluster-api/bootstrap-success.complete
              [Service]
              Type=oneshot
              User=root
              ExecStartPre=/bin/sh -c "mount -L config-2 /mnt"
              # Note we use _ as a delimiter for sed here because the providerid includes / characters
              ExecStart=/bin/sh -c "sed -i \"s_PROVIDERID_$(jq -r .providerid /mnt/openstack/latest/meta_data.json)_\" /etc/rancher/rke2/config.yaml"
              ExecStart=/bin/sh -c "echo \"node-name: $(jq -r .name /mnt/openstack/latest/meta_data.json)\" >> /etc/rancher/rke2/config.yaml"
              ExecStart=/bin/sh -c "echo \"node-label:\" >> /etc/rancher/rke2/config.yaml"
              ExecStart=/bin/sh -c "echo \"  - metal3.io/uuid=$(jq -r .uuid /mnt/openstack/latest/meta_data.json)\" >> /etc/rancher/rke2/config.yaml"
              ExecStartPost=/bin/sh -c "umount /mnt"
              [Install]
              WantedBy=multi-user.target
          - name: cpu-partitioning.service
            enabled: true
            contents: |
              [Unit]
              Description=cpu-partitioning
              Wants=network-online.target
              After=network.target network-online.target
              [Service]
              Type=oneshot
              User=root
              ExecStart=/bin/sh -c "echo isolated_cores=${ISOLATED_CPU_CORES} > /etc/tuned/cpu-partitioning-variables.conf"
              ExecStartPost=/bin/sh -c "tuned-adm profile cpu-partitioning"
              ExecStartPost=/bin/sh -c "systemctl enable tuned.service"
              [Install]
              WantedBy=multi-user.target
          - name: performance-settings.service
            enabled: true
            contents: |
              [Unit]
              Description=performance-settings
              Wants=network-online.target
              After=network.target network-online.target cpu-partitioning.service
              [Service]
              Type=oneshot
              User=root
              ExecStart=/bin/sh -c "/opt/performance-settings/performance-settings.sh"
              [Install]
              WantedBy=multi-user.target
          - name: sriov-custom-auto-vfs.service
            enabled: true
            contents: |
              [Unit]
              Description=SRIOV Custom Auto VF Creation
              Wants=network-online.target  rke2-server.target
              After=network.target network-online.target rke2-server.target
              [Service]
              User=root
              Type=forking
              TimeoutStartSec=1800
              ExecStart=/bin/sh -c "while ! /var/lib/rancher/rke2/bin/kubectl --kubeconfig=/etc/rancher/rke2/rke2.yaml wait --for condition=ready nodes --timeout=30m --all ; do sleep 10 ; done"
              ExecStartPost=/bin/sh -c "/opt/sriov/sriov-auto-filler.sh"
              RemainAfterExit=yes
              KillMode=process
              [Install]
              WantedBy=multi-user.target
    kubelet:
      extraArgs:
      - provider-id=metal3://PROVIDERID
    nodeName: "localhost.localdomain"