1 BMC automated deployments with Metal3 #
Metal3 is a CNCF project which provides bare-metal infrastructure management capabilities for Kubernetes.
Metal3 provides Kubernetes-native resources to manage the lifecycle of bare-metal servers which support management via out-of-band protocols such as Redfish.
It also has mature support for Cluster API (CAPI) which enables management of infrastructure resources across multiple infrastructure providers via broadly adopted vendor-neutral APIs.
1.1 Why use this method #
This method is useful for scenarios where the target hardware supports out-of-band management, and a fully automated infrastructure management flow is desired.
A management cluster is configured to provide declarative APIs that enable inventory and state management of downstream cluster bare-metal servers, including automated inspection, cleaning and provisioning/deprovisioning.
1.2 High-level architecture #
1.3 Prerequisites #
There are some specific constraints related to the downstream cluster server hardware and networking:
Management cluster
Must have network connectivity to the target server management/BMC API
Must have network connectivity to the target server control plane network
For multi-node management clusters, an additional reserved IP address is required
Hosts to be controlled
Must support out-of-band management via Redfish, iDRAC or iLO interfaces
Must support deployment via virtual media (PXE is not currently supported)
Must have network connectivity to the management cluster for access to the Metal3 provisioning APIs
Some tools are required, these can be installed either on the management cluster, or on a host which can access it.
Kubectl, Helm and Clusterctl
A container runtime such as Podman or Rancher Desktop
The SLE-Micro.x86_64-5.5.0-Default-GM.raw.xz
OS image file must be downloaded from the SUSE Customer Center or the SUSE Download page.
1.3.1 Setup Management Cluster #
The basic steps to install a management cluster and use Metal3 are:
Install an RKE2 management cluster
Install Rancher
Install a storage provider
Install the Metal3 dependencies
Install CAPI dependencies
Build a SLEMicro OS image for downstream cluster hosts
Register BareMetalHost CRs to define the bare-metal inventory
Create a downstream cluster by defining CAPI resources
This guide assumes an existing RKE2 cluster and Rancher (including cert-manager) has been installed, for example by using Edge Image Builder (Chapter 9, Edge Image Builder).
The steps here can also be fully automated as described in the ATIP management cluster documentation (Chapter 29, Setting up the management cluster).
1.3.2 Installing Metal3 dependencies #
If not already installed as part of the Rancher installation, cert-manager must be installed and running.
A persistent storage provider must be installed. Longhorn is recommended but local-path can also be used for dev/PoC environments. The instructions below assume a StorageClass has been marked as default, otherwise additional configuration for the Metal3 chart is required.
An additional IP is required, which is managed by MetalLB to provide a consistent endpoint for the Metal3 management services. This IP must be part of the control plane subnet and reserved for static configuration (not part of any DHCP pool).
If the management cluster is a single node, the requirement for an additional floating IP managed via MetalLB can be avoided, see Single-node configuration (Section 1.6.1, “Single-node configuration”)
First, we install MetalLB:
helm install \ metallb oci://registry.suse.com/edge/metallb-chart \ --namespace metallb-system \ --create-namespace
Then we define an
IPAddressPool
andL2Advertisment
using the reserved IP, defined asSTATIC_IRONIC_IP
below:export STATIC_IRONIC_IP=<STATIC_IRONIC_IP> cat <<-EOF | kubectl apply -f - apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: ironic-ip-pool namespace: metallb-system spec: addresses: - ${STATIC_IRONIC_IP}/32 serviceAllocation: priority: 100 serviceSelectors: - matchExpressions: - {key: app.kubernetes.io/name, operator: In, values: [metal3-ironic]} EOF
cat <<-EOF | kubectl apply -f - apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: ironic-ip-pool-l2-adv namespace: metallb-system spec: ipAddressPools: - ironic-ip-pool EOF
Now Metal3 can be installed:
helm install \ metal3 oci://registry.suse.com/edge/metal3-chart \ --namespace metal3-system \ --create-namespace \ --set global.ironicIP="${STATIC_IRONIC_IP}"
It can take around two minutes for the initContainer to run on this deployment, so ensure the pods are all running before proceeding:
kubectl get pods -n metal3-system NAME READY STATUS RESTARTS AGE baremetal-operator-controller-manager-85756794b-fz98d 2/2 Running 0 15m metal3-metal3-ironic-677bc5c8cc-55shd 4/4 Running 0 15m metal3-metal3-mariadb-7c7d6fdbd8-64c7l 1/1 Running 0 15m
Do not proceed to the following steps until all pods in the metal3-system
namespace are running
1.3.3 Installing cluster API dependencies #
First, we need to disable the Rancher-embedded CAPI controller:
cat <<-EOF | kubectl apply -f -
apiVersion: management.cattle.io/v3
kind: Feature
metadata:
name: embedded-cluster-api
spec:
value: false
EOF
kubectl delete mutatingwebhookconfiguration.admissionregistration.k8s.io mutating-webhook-configuration
kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io validating-webhook-configuration
kubectl wait --for=delete namespace/cattle-provisioning-capi-system --timeout=300s
Then, to use the SUSE images, a configuration file is needed:
mkdir ~/.cluster-api
cat > ~/.cluster-api/clusterctl.yaml <<EOF
images:
all:
repository: registry.suse.com/edge
EOF
Install clusterctl 1.6.x, after which we will install the core, infrastructure, bootstrap and control plane providers as follows:
clusterctl init --core "cluster-api:v1.6.2" --infrastructure "metal3:v1.6.0" --bootstrap "rke2:v0.4.1" --control-plane "rke2:v0.4.1"
After some time, the controller pods should be running in the capi-system
, capm3-system
, rke2-bootstrap-system
and rke2-control-plane-system
namespaces.
1.3.4 Prepare downstream cluster image #
Edge Image Builder (Chapter 9, Edge Image Builder) is used to prepare a modified SLEMicro base image which is provisioned on downstream cluster hosts.
In this guide, we cover the minimal configuration necessary to deploy the downstream cluster.
1.3.4.1 Image configuration #
When running Edge Image Builder, a directory is mounted from the host, so it is necessary to create a directory structure to store the configuration files used to define the target image.
downstream-cluster-config.yaml
is the image definition file, see Chapter 3, Standalone clusters with Edge Image Builder for more details.The base image when downloaded is
xz
compressed, which must be uncompressed withunxz
and copied/moved under thebase-images
folder.The
network
folder is optional, see Section 1.3.5.1.1, “Additional script for static network configuration” for more details.The custom/scripts directory contains scripts to be run on first-boot; currently a
growfs.sh
script is required to resize the OS root partition on deployment
├── downstream-cluster-config.yaml ├── base-images/ │ └ SLE-Micro.x86_64-5.5.0-Default-GM.raw ├── network/ | └ configure-network.sh └── custom/ └ scripts/ └ growfs.sh
1.3.4.1.1 Downstream cluster image definition file #
The downstream-cluster-config.yaml
file is the main configuration file for the downstream cluster image. The following is a minimal example for deployment via Metal3:
apiVersion: 1.0
image:
imageType: RAW
arch: x86_64
baseImage: SLE-Micro.x86_64-5.5.0-Default-GM.raw
outputImageName: SLE-Micro-eib-output.raw
operatingSystem:
kernelArgs:
- ignition.platform.id=openstack
- net.ifnames=1
systemd:
disable:
- rebootmgr
users:
- username: root
encryptedPassword: ${ROOT_PASSWORD}
sshKeys:
- ${USERKEY1}
${ROOT_PASSWORD}
is the encrypted password for the root user, which can be useful for test/debugging. It can be generated with the openssl passwd -6 PASSWORD
command
For the production environments, it is recommended to use the SSH keys that can be added to the users block replacing the ${USERKEY1}
with the real SSH keys.
net.ifnames=1
enables Predictable Network Interface Naming
This matches the default configuration for the metal3 chart, but the setting must match the configured chart predictableNicNames
value.
Also note ignition.platform.id=openstack
is mandatory, without this argument SLEMicro configuration via ignition will fail in the Metal3 automated flow.
1.3.4.1.2 Growfs script #
Currently is a custom script (custom/scripts/growfs.sh
) which is required to grow the file system to the match the disk size on first-boot after provisioning. The growfs.sh
script contains the following information:
#!/bin/bash growfs() { mnt="$1" dev="$(findmnt --fstab --target ${mnt} --evaluate --real --output SOURCE --noheadings)" # /dev/sda3 -> /dev/sda, /dev/nvme0n1p3 -> /dev/nvme0n1 parent_dev="/dev/$(lsblk --nodeps -rno PKNAME "${dev}")" # Last number in the device name: /dev/nvme0n1p42 -> 42 partnum="$(echo "${dev}" | sed 's/^.*[^0-9]\([0-9]\+\)$/\1/')" ret=0 growpart "$parent_dev" "$partnum" || ret=$? [ $ret -eq 0 ] || [ $ret -eq 1 ] || exit 1 /usr/lib/systemd/systemd-growfs "$mnt" } growfs /
Add your own custom scripts to be executed during the provisioning process using the same approach. For more information, see Chapter 3, Standalone clusters with Edge Image Builder.
1.3.4.2 Image creation #
Once the directory structure is prepared following the previous sections, run the following command to build the image:
podman run --rm --privileged -it -v $PWD:/eib \ registry.suse.com/edge/edge-image-builder:1.0.2 \ build --definition-file downstream-cluster-config.yaml
This creates the output image file named SLE-Micro-eib-output.raw
, based on the definition described above.
The output image must then be made available via a webserver, either the media-server container enabled via the Metal3 chart (Note)
or some other locally accessible server. In the examples below, we refer to this server as imagecache.local:8080
1.3.5 Adding BareMetalHost inventory #
Registering bare-metal servers for automated deployment requires creating two resources: a Secret storing BMC access credentials and a Metal3 BareMetalHost resource defining the BMC connection and other details:
apiVersion: v1
kind: Secret
metadata:
name: controlplane-0-credentials
type: Opaque
data:
username: YWRtaW4=
password: cGFzc3dvcmQ=
---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
name: controlplane-0
labels:
cluster-role: control-plane
spec:
online: true
bootMACAddress: "00:f3:65:8a:a3:b0"
bmc:
address: redfish-virtualmedia://192.168.125.1:8000/redfish/v1/Systems/68bd0fb6-d124-4d17-a904-cdf33efe83ab
disableCertificateVerification: true
credentialsName: controlplane-0-credentials
Note the following:
The Secret username/password must be base64 encoded. Note this should not include any trailing newlines (for example, use
echo -n
, not justecho
!)The
cluster-role
label may be set now or later on cluster creation. In the example below, we expectcontrol-plane
orworker
bootMACAddress
must be a valid MAC that matches the control plane NIC of the hostThe
bmc
address is the connection to the BMC management API, the following are supported:redfish-virtualmedia://<IP ADDRESS>/redfish/v1/Systems/<SYSTEM ID>
: Redfish virtual media, for example, SuperMicroidrac-virtualmedia://<IP ADDRESS>/redfish/v1/Systems/System.Embedded.1
: Dell iDRAC
See the Upstream API docs for more details on the BareMetalHost API
1.3.5.1 Configuring Static IPs #
The BareMetalHost example above assumes DHCP provides the controlplane network configuration, but for scenarios where manual configuration is needed such as static IPs it is possible to provide additional configuration, as described below.
1.3.5.1.1 Additional script for static network configuration #
When creating the base image with Edge Image Builder, in the network
folder, create the following configure-network.sh
file.
This consumes configuration drive data on first-boot, and configures the host networking using the NM Configurator tool.
#!/bin/bash set -eux # Attempt to statically configure a NIC in the case where we find a network_data.json # In a configuration drive CONFIG_DRIVE=$(blkid --label config-2 || true) if [ -z "${CONFIG_DRIVE}" ]; then echo "No config-2 device found, skipping network configuration" exit 0 fi mount -o ro $CONFIG_DRIVE /mnt NETWORK_DATA_FILE="/mnt/openstack/latest/network_data.json" if [ ! -f "${NETWORK_DATA_FILE}" ]; then umount /mnt echo "No network_data.json found, skipping network configuration" exit 0 fi DESIRED_HOSTNAME=$(cat /mnt/openstack/latest/meta_data.json | tr ',{}' '\n' | grep '\"metal3-name\"' | sed 's/.*\"metal3-name\": \"\(.*\)\"/\1/') echo "${DESIRED_HOSTNAME}" > /etc/hostname mkdir -p /tmp/nmc/{desired,generated} cp ${NETWORK_DATA_FILE} /tmp/nmc/desired/_all.yaml umount /mnt ./nmc generate --config-dir /tmp/nmc/desired --output-dir /tmp/nmc/generated ./nmc apply --config-dir /tmp/nmc/generated
1.3.5.1.2 Additional secret with host network configuration #
An additional secret containing data in the nmstate format supported by NM Configurator (Chapter 10, Edge Networking) can be defined for each host.
The secret is then referenced in the BareMetalHost
resource via the preprovisioningNetworkDataName
spec field.
apiVersion: v1
kind: Secret
metadata:
name: controlplane-0-networkdata
type: Opaque
stringData:
networkData: |
interfaces:
- name: enp1s0
type: ethernet
state: up
mac-address: "00:f3:65:8a:a3:b0"
ipv4:
address:
- ip: 192.168.125.200
prefix-length: 24
enabled: true
dhcp: false
dns-resolver:
config:
server:
- 192.168.125.1
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.125.1
next-hop-interface: enp1s0
---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
name: controlplane-0
labels:
cluster-role: control-plane
spec:
preprovisioningNetworkDataName: controlplane-0-networkdata
# Remaining content as in previous example
In some circumstances the mac-address may be omitted but the configure-network.sh
script must use the _all.yaml
filename described above to enable
Unified node configuration (Section 10.5.8, “Unified node configurations”) in nm-configurator.
1.3.5.2 BareMetalHost preparation #
After creating the BareMetalHost resource and associated secrets as described above, a host preparation workflow is triggered:
A ramdisk image is booted by virtualmedia attachment to the target host BMC
The ramdisk inspects hardware details, and prepares the host for provisioning (for example by cleaning disks of previous data)
On completion of this process, hardware details in the BareMetalHost
status.hardware
field are updated and can be verified
This process can take several minutes, but when completed you should see the BareMetalHost state become available
:
% kubectl get baremetalhost
NAME STATE CONSUMER ONLINE ERROR AGE
controlplane-0 available true 9m44s
worker-0 available true 9m44s
1.3.6 Creating downstream clusters #
We now create Cluster API resources which define the downstream cluster, and Machine resources which will cause the BareMetalHost resources to be provisioned, then bootstrapped to form an RKE2 cluster.
1.3.7 Control plane deployment #
To deploy the controlplane we define a yaml manifest similar to the one below, which contains the following resources:
Cluster resource defines the cluster name, networks, and type of controlplane/infrastructure provider (in this case RKE2/Metal3)
Metal3Cluster defines the controlplane endpoint (host IP for single-node, LoadBalancer endpoint for multi-node, this example assumes single-node)
RKE2ControlPlane defines the RKE2 version and any additional configuration needed during cluster bootstrapping
Metal3MachineTemplate defines the OS Image to be applied to the BareMetalHost resources, and the hostSelector defines which BareMetalHosts to consume
Metal3DataTemplate defines additional metaData to be passed to the BareMetalHost (note networkData is not currently supported in the Edge solution)
Note for simplicity this example assumes a single-node controlplane, where the BareMetalHost is configured with an IP of 192.168.125.200
- for more
advanced multi-node examples please see the ATIP documentation (Chapter 31, Fully automated directed network provisioning)
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: sample-cluster
namespace: default
spec:
clusterNetwork:
pods:
cidrBlocks:
- 192.168.0.0/18
services:
cidrBlocks:
- 10.96.0.0/12
controlPlaneRef:
apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
kind: RKE2ControlPlane
name: sample-cluster
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3Cluster
name: sample-cluster
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3Cluster
metadata:
name: sample-cluster
namespace: default
spec:
controlPlaneEndpoint:
host: 192.168.125.200
port: 6443
noCloudProvider: true
---
apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
kind: RKE2ControlPlane
metadata:
name: sample-cluster
namespace: default
spec:
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3MachineTemplate
name: sample-cluster-controlplane
replicas: 1
agentConfig:
format: ignition
kubelet:
extraArgs:
- provider-id=metal3://BAREMETALHOST_UUID
additionalUserData:
config: |
variant: fcos
version: 1.4.0
systemd:
units:
- name: rke2-preinstall.service
enabled: true
contents: |
[Unit]
Description=rke2-preinstall
Wants=network-online.target
Before=rke2-install.service
ConditionPathExists=!/run/cluster-api/bootstrap-success.complete
[Service]
Type=oneshot
User=root
ExecStartPre=/bin/sh -c "mount -L config-2 /mnt"
ExecStart=/bin/sh -c "sed -i \"s/BAREMETALHOST_UUID/$(jq -r .uuid /mnt/openstack/latest/meta_data.json)/\" /etc/rancher/rke2/config.yaml"
ExecStart=/bin/sh -c "echo \"node-name: $(jq -r .name /mnt/openstack/latest/meta_data.json)\" >> /etc/rancher/rke2/config.yaml"
ExecStartPost=/bin/sh -c "umount /mnt"
[Install]
WantedBy=multi-user.target
version: v1.28.10+rke2r1
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3MachineTemplate
metadata:
name: sample-cluster-controlplane
namespace: default
spec:
template:
spec:
dataTemplate:
name: sample-cluster-controlplane-template
hostSelector:
matchLabels:
cluster-role: control-plane
image:
checksum: http://imagecache.local:8080/SLE-Micro-eib-output.raw.sha256
checksumType: sha256
format: raw
url: http://imagecache.local:8080/SLE-Micro-eib-output.raw
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3DataTemplate
metadata:
name: sample-cluster-controlplane-template
namespace: default
spec:
clusterName: sample-cluster
metaData:
objectNames:
- key: name
object: machine
- key: local-hostname
object: machine
- key: local_hostname
object: machine
When the example above has been copied and adapted to suit your environment, it can be applied via kubectl
then the cluster status can be monitored with clusterctl
% kubectl apply -f rke2-control-plane.yaml
# Wait for the cluster to be provisioned - status can be checked via clusterctl
% clusterctl describe cluster sample-cluster
NAME READY SEVERITY REASON SINCE MESSAGE
Cluster/sample-cluster True 22m
├─ClusterInfrastructure - Metal3Cluster/sample-cluster True 27m
├─ControlPlane - RKE2ControlPlane/sample-cluster True 22m
│ └─Machine/sample-cluster-chflc True 23m
1.3.8 Worker/Compute deployment #
Similar to the controlplane we define a yaml manifest, which contains the following resources:
MachineDeployment defines the number of replicas (hosts) and the bootstrap/infrastructure provider (in this case RKE2/Metal3)
RKE2ConfigTemplate describes the RKE2 version and first-boot configuration for agent host bootstrapping
Metal3MachineTemplate defines the OS Image to be applied to the BareMetalHost resources, and the hostSelector defines which BareMetalHosts to consume
Metal3DataTemplate defines additional metaData to be passed to the BareMetalHost (note networkData is not currently supported in the Edge solution)
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
labels:
cluster.x-k8s.io/cluster-name: sample-cluster
name: sample-cluster
namespace: default
spec:
clusterName: sample-cluster
replicas: 1
selector:
matchLabels:
cluster.x-k8s.io/cluster-name: sample-cluster
template:
metadata:
labels:
cluster.x-k8s.io/cluster-name: sample-cluster
spec:
bootstrap:
configRef:
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha1
kind: RKE2ConfigTemplate
name: sample-cluster-workers
clusterName: sample-cluster
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3MachineTemplate
name: sample-cluster-workers
nodeDrainTimeout: 0s
version: v1.28.10+rke2r1
---
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha1
kind: RKE2ConfigTemplate
metadata:
name: sample-cluster-workers
namespace: default
spec:
template:
spec:
agentConfig:
format: ignition
version: v1.28.10+rke2r1
kubelet:
extraArgs:
- provider-id=metal3://BAREMETALHOST_UUID
additionalUserData:
config: |
variant: fcos
version: 1.4.0
systemd:
units:
- name: rke2-preinstall.service
enabled: true
contents: |
[Unit]
Description=rke2-preinstall
Wants=network-online.target
Before=rke2-install.service
ConditionPathExists=!/run/cluster-api/bootstrap-success.complete
[Service]
Type=oneshot
User=root
ExecStartPre=/bin/sh -c "mount -L config-2 /mnt"
ExecStart=/bin/sh -c "sed -i \"s/BAREMETALHOST_UUID/$(jq -r .uuid /mnt/openstack/latest/meta_data.json)/\" /etc/rancher/rke2/config.yaml"
ExecStart=/bin/sh -c "echo \"node-name: $(jq -r .name /mnt/openstack/latest/meta_data.json)\" >> /etc/rancher/rke2/config.yaml"
ExecStartPost=/bin/sh -c "umount /mnt"
[Install]
WantedBy=multi-user.target
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3MachineTemplate
metadata:
name: sample-cluster-workers
namespace: default
spec:
template:
spec:
dataTemplate:
name: sample-cluster-workers-template
hostSelector:
matchLabels:
cluster-role: worker
image:
checksum: http://imagecache.local:8080/SLE-Micro-eib-output.raw.sha256
checksumType: sha256
format: raw
url: http://imagecache.local:8080/SLE-Micro-eib-output.raw
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: Metal3DataTemplate
metadata:
name: sample-cluster-workers-template
namespace: default
spec:
clusterName: sample-cluster
metaData:
objectNames:
- key: name
object: machine
- key: local-hostname
object: machine
- key: local_hostname
object: machine
When the example above has been copied and adapted to suit your environment, it can be applied via kubectl
then the cluster status can be monitored with clusterctl
% kubectl apply -f rke2-agent.yaml
# Wait some time for the compute/agent hosts to be provisioned
% clusterctl describe cluster sample-cluster
NAME READY SEVERITY REASON SINCE MESSAGE
Cluster/sample-cluster True 25m
├─ClusterInfrastructure - Metal3Cluster/sample-cluster True 30m
├─ControlPlane - RKE2ControlPlane/sample-cluster True 25m
│ └─Machine/sample-cluster-chflc True 27m
└─Workers
└─MachineDeployment/sample-cluster True 22m
└─Machine/sample-cluster-56df5b4499-zfljj True 23m
1.3.9 Cluster deprovisioning #
The downstream cluster may be deprovisioned by deleting the resources applied in the creation steps above:
% kubectl delete -f rke2-agent.yaml
% kubectl delete -f rke2-control-plane.yaml
This triggers deprovisioning of the BareMetalHost resources, which may take several minutes, after which they should be in available state again:
% kubectl get bmh
NAME STATE CONSUMER ONLINE ERROR AGE
controlplane-0 deprovisioning sample-cluster-controlplane-vlrt6 false 10m
worker-0 deprovisioning sample-cluster-workers-785x5 false 10m
...
% kubectl get bmh
NAME STATE CONSUMER ONLINE ERROR AGE
controlplane-0 available false 15m
worker-0 available false 15m
1.4 Known issues #
The upstream IP Address Management controller is currently not supported, because it’s not yet compatible with our choice of network configuration tooling and first-boot toolchain in SLEMicro.
Relatedly, the IPAM resources and Metal3DataTemplate networkData fields are not currently supported.
Only deployment via redfish-virtualmedia is currently supported.
Deployed clusters are not currently imported into Rancher
Due to disabling the Rancher embedded CAPI controller, a management cluster configured for Metal3 as described above cannot also be used for other cluster provisioning methods such as Elemental (Chapter 11, Elemental)
1.5 Planned changes #
Deployed clusters imported into Rancher, this is planned via Rancher Turtles in future
Aligning with Rancher Turtles is also expected to remove the requirement to disable the Rancher embedded CAPI, so other cluster methods should be possible via the management cluster.
Enable support of the IPAM resources and configuration via networkData fields
1.6 Additional resources #
The ATIP Documentation (Chapter 26, SUSE Adaptive Telco Infrastructure Platform (ATIP)) has examples of more advanced usage of Metal3 for telco use-cases.
1.6.1 Single-node configuration #
For test/PoC environments where the management cluster is a single node, it is possible to avoid the requirement for an additional floating IP managed via MetalLB.
In this mode, the endpoint for the management cluster APIs is the IP of the management cluster, therefore it should be reserved when using DHCP
or statically configured to ensure the management cluster IP does not change - referred to as <MANAGEMENT_CLUSTER_IP>
below.
To enable this scenario the metal3 chart values required are as follows:
global:
ironicIP: <MANAGEMENT_CLUSTER_IP>
metal3-ironic:
service:
type: NodePort
1.6.2 Disabling TLS for virtualmedia ISO attachment #
Some server vendors verify the SSL connection when attaching virtual-media ISO images to the BMC, which can cause a problem because the generated certificates for the Metal3 deployment are self-signed, to work around this issue it’s possible to disable TLS only for the virtualmedia disk attachment with metal3 chart values as follows:
global:
enable_vmedia_tls: false
An alternative solution is to configure the BMCs with the CA cert - in this case you can read the certificates from the cluster using kubectl
:
kubectl get secret -n metal3-system ironic-vmedia-cert -o yaml
The certificate can then be configured on the server BMC console, although the process for that is vendor specific (and not possible for all
vendors, in which case the enable_vmedia_tls
flag may be required).