Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / Managing network bonding devices

Managing network bonding devices

Publication Date: 27 Sep 2024
WHAT?

Combine two or more network interfaces into a single bonded device.

WHY?

Network bonding can increase bandwidth and/or provide redundancy.

EFFORT

Approximately 15 minutes to read and understand the content.

REQUIREMENTS
  • Network connection

  • Basic understanding of networking and IP addresses

  • Existing network interfaces to include in the bonded device

  • Switch support, depending on the bonding mode

This article is about network bonding. To configure network teaming, see Managing network teaming devices.

1 Configuring network bonding

Network bonding combines two or more network interfaces into a single bonded device to increase bandwidth and/or provide redundancy. You can configure network bonding with YaST or by manually creating ifcfg files. The behavior of the bonded device is configured using bonding modes.

1.1 Requirements

  • Network connection

  • Basic understanding of networking and IP addresses

  • Existing network interfaces to include in the bonded device

  • Switch support, depending on the bonding mode

1.2 Restrictions

Do not split bonds over multiple switches

In most hardware setups, all network interfaces in a bonded device must be connected to the same switch. For more information, consult your switch vendor documentation.

IBM POWER: Bonding modes 5 and 6 (balance-tlb and balance-alb) unsupported by ibmveth

The bonding drivers in tlb mode and alb mode send Ethernet Loopback packets with both the source and destination MAC addresses listed as the Virtual Ethernet MAC address. These packets are not supported by Power firmware. Therefore, bonding modes 5 and 6 are unsupported by ibmveth.

Bonding and virtualization

Bonded devices are made up of multiple network interfaces. In most configurations you should only configure bonding in the host. Virtual interfaces to guests are then created as a bridge with the bonded devices, simplifying guest creation and deployment.

It is possible, but not recommended, to configure bonding in a guest. When configuring bonding in a guest, you must assign multiple interfaces to the guest and configure the host without bonding. You must also be careful to configure the host and its network bridges so that you do not mix bonding in the host and guests.

1.3 Bonding modes

The following bonding modes are available:

(0) balance-rr

Packets are transmitted in round-robin fashion from the first to the last available interface. Provides fault tolerance and load balancing. Requires switch support. Certain switches might fail with this mode.

(1) active-backup

Only one network interface is active. If it fails, a different interface becomes active. Provides fault tolerance. This is the default mode. No specific switch support is required.

(2) balance-xor

Traffic is split between all available interfaces based on the number of interfaces included in the bonded device. Provides fault tolerance and load balancing. Requires switch support. Certain switches might fail with this mode.

(3) broadcast

All traffic is broadcast on all interfaces. Provides fault tolerance. Requires switch support. Certain switches might fail with this mode. If possible, use mode 1 instead, or use this mode to provide sniffing capability by connecting each member of the bond to a different switch or device.

(4) 802.3ad

Also called LACP. All interfaces in the LACP group must share the same speed and duplex settings, and must be connected to the same switch. Provides fault tolerance and load balancing. Requires ethtool support in the interface drivers, and a switch that supports and is configured for IEEE 802.3ad Dynamic link aggregation. If your switch supports it, this is the preferred mode.

(5) balance-tlb

Adaptive transmit load balancing. Provides fault tolerance and load balancing. Requires ethtool support in the interface drivers. No specific switch support is required, but certain switches might fail with this mode.

(6) balance-alb

Adaptive load balancing. Provides fault tolerance and load balancing. Requires ethtool support in the interface drivers. No specific switch support is required, but certain switches might fail with this mode.

Consult your hardware manual to check which modes your switch supports.

For a more detailed description of the modes, see https://www.kernel.org/doc/Documentation/networking/bonding.txt.

1.4 Configuring network bonding with YaST

  1. Start the graphical version of YaST, or run the command yast2 to start YaST in text mode.

  2. Select System › Network Settings.

  3. Select Add and change the Device Type to Bonding.

  4. Select Next to open the Network Card Setup menu.

  5. In the Address tab, select how to assign an IP address to the bonded device:

    • Dynamic Address: Automatically assign a dynamic IP address. The default setup is to use DHCP with both IPv4 and IPv6. You can change these settings if required.

    • Statically Assigned IP Address: Manually assign an IP Address, Subnet Mask, and Hostname.

    • Do not use No Link and IP Setup. This option is only used for individual network interfaces that will be added to a bonded device.

  6. In the Bond Ports tab, select the network interfaces to include in the bonded device by activating the check boxes.

  7. From the Bond Driver Options drop-down box, choose a bonding mode. The default mode is active-backup.

    Important

    Do not remove miimon=100. Without this parameter, data integrity is not checked regularly.

  8. Select Next.

  9. If you are prompted to adapt the existing configuration for bonding, select Continue.

  10. Select OK to create the bonded device.

  11. Select Quit or press F9 to close YaST.

1.5 Configuring network bonding with ifcfg

  1. Create a configuration file named /etc/sysconfig/network/ifcfg-bond0. If you need more than one bonded device, give them ascending numbers. For more information, see man ifcfg, man ifcfg-bonding, and /etc/sysconfig/network/ifcfg.template.

    In the configuration file, define the following parameters:

    STARTMODE=MODE1
    BOOTPROTO=IP_ASSIGNMENT2
    IPADDR=IPv4_ADDRESS3
    IPADDR6=IPv6_ADDRESS3
    
    BONDING_MASTER="yes"4
    
    BONDING_SLAVE0="FIRST_DEVICE_NAME"5
    BONDING_SLAVE1="SECOND_DEVICE_NAME"5
    
    BONDING_MODULE_OPTS="mode=BONDING_MODE6 miimon=100"7

    1

    Defines how the bonded device starts. Use auto to start the interface automatically on every reboot, or use manual to prevent the interface from starting automatically.

    2

    Defines how IP addresses are assigned to the bonded device: Use static to manually assign static IP addresses, or use dhcp to automatically assign dynamic IP addresses.

    3

    Specifies the IPv4 and IPv6 addresses for the device, if you are manually assigning static IP addresses.

    4

    Identifies this interface as a bonded device.

    5

    Specifies two or more network interfaces to include in the bonded device.

    6

    Defines the bonding mode for this interface.

    7

    Specifies how often, in milliseconds, each bond member is inspected for link failures.

    Example 1: Bond configuration
    STARTMODE=auto
    BOOTPROTO=static
    IPADDR="192.168.1.2/24"
    IPADDR6="fd00:deca:fbad:50::2/64"
    
    BONDING_MASTER="yes"
    
    BONDING_SLAVE0="eth0"
    BONDING_SLAVE1="eth1"
    
    BONDING_MODULE_OPTS="mode=802.3ad miimon=100"
  2. Back up the existing ifcfg files for the bond member interfaces, in case you need to restore them later.

  3. Adjust the ifcfg files for both bond member interfaces so that they contain only the following content:

    BOOTPROTO='none' 1
    STARTMODE='hotplug' 2

    1

    BOOTPROTO=none uses the ethtool options (when provided), but does not set up the link on startup because the bond member interface is controlled by the bond device.

    2

    STARTMODE=hotplug allows the bond member interface to join the bond automatically when it is available.

  4. Check if everything is included in Wicked's configuration file:

    > sudo wicked show-config
  5. Start the bonded device:

    > sudo wicked ifup all bond0

    If you need additional debug information, use the option --debug all after the all subcommand.

    Important

    Do not use systemctl to start or stop the bonded device. Always use the wicked command.

  6. Check the status of the bonded device:

    • Get the state of the bonded device from Wicked:

      > sudo wicked ifstatus --verbose bond0
    • Get the state of the bonded device and the bond members:

      > ip addr

    Each command shows a slightly different view depending on your needs.

If you make changes to the ifcfg-bond0 file after starting the device, reload its configuration with the command wicked ifreload bond0.

2 Enabling hotplugging for network bond members

In some network environments (such as High Availability), you might need to replace a bond member interface with a new one, for example, if a network interface has a fault. However, before you can replace the interface, you must configure the networking setup to recognize when a new device is added.

You can enable hotplugging either manually or with YaST.

2.1 Requirements

  • A bonded device is already configured.

2.2 Enabling hotplugging for network bond members with YaST

  1. Start the graphical version of YaST, or run the command yast2 to start YaST in text mode.

  2. Select System › Network Settings. The Overview tab shows the existing devices. If devices are already configured in a bond, this is stated in the Note column.

  3. Select one of the bond members, then select Edit. The Network Card Setup menu opens.

  4. Switch to the General tab.

  5. Under Udev rules, select Change. A dialog box opens.

  6. In the dialog box, choose the BusID option, then select OK.

  7. Make sure that Activate device is set to On Hotplug.

  8. Select Next to return to the Network Settings menu. When you select the bond member now, the bottom pane shows the device's details, including the bus ID.

  9. Repeat Step 3 to Step 8 for all bond members.

  10. Select OK to confirm your changes.

  11. Select Quit or press F9 to close YaST.

2.3 Enabling hotplugging for network bond members manually

  1. Check that the bond members are configured with BOOTPROTO=none and STARTMODE=hotplug. If you configured the bonded device with YaST, this is the default setting.

  2. List the hardware information for the network interfaces. You can use grep to show only the information required for this procedure:

    > sudo hwinfo --netcard | grep -E "SysFS BusID|Device File"
      SysFS BusID: 0000:00:17.0
      Device File: eth0
      SysFS BusID: 0000:00:19.0
      Device File: eth1

    For each bond member, make a note of the associated SysFS BusID.

  3. Open the file /etc/udev/rules.d/70-persistent-net.rules. For each bond member, change the value of KERNELS to match the SysFS BusID from the previous step. For example:

    SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", KERNELS=="0000:00:17.0",
    ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
    
    SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", KERNELS=="0000:00:19.0",
    ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

    This configures the udev rules to match the device by bus ID instead of by MAC address, allowing the bond to detect a new device in the same slot but with a different MAC address.

2.4 Summary

When one of the bond member interfaces is removed from the system, the kernel removes it from the bond automatically. When a new card is added to the system, udev uses the bus-based persistent name rule to rename the interface to the name of the bond member, and calls ifup for it. The ifup call automatically joins the new interface into the bond.

At boot time, network.service does not wait for the hotplugged bond members, but for the bond to become ready, which only requires one available bond member.

3 Removing a bonded device

Use this procedure to remove the configuration for a bonded or teamed device and restore the member interfaces to their original, separate configuration.

The following examples use the device name team0. Replace this with the actual name of your device.

  1. Stop the device:

    > sudo wicked ifdown team0
  2. Hide the configuration file by renaming it from /etc/sysconfig/network/ifcfg-team0 to /etc/sysconfig/network/.ifcfg-team0.

    Alternatively, if you definitely do not need the configuration anymore, remove the file instead of renaming it.

  3. To re-enable the network interfaces that were included in the device, restore their original configuration from the backup files.

  4. Reload the network configuration:

    > sudo wicked ifreload all
  5. Check the status of the remaining network interfaces:

    > ip addr

4 Network bonding or network teaming: feature comparison

Network bonding and network teaming are different methods for combining network connections to provide a single combined interface. Bonding is handled exclusively in the kernel. Teaming includes a small set of kernel modules that provide an interface for teamd instances, but everything else is handled in user space.

Table 1: Bonding and teaming feature comparison
FeatureBondingTeaming
broadcast, round-robin TX policyyesyes
active-backup TX policyyesyes
LACP (802.3ad) supportyesyes
hash-based TX policyyesyes
user can set hash functionnoyes
TX load-balancing support (TLB)yesyes
TX load-balancing support for LACPnoyes
Ethtool link monitoringyesyes
ARP link monitoringyesyes
NS/NA (IPV6) link monitoringnoyes
RCU locking on TX/RX pathsnoyes
port prio and stickinessnoyes
separate per-port link monitoring setupnoyes
multiple link monitoring setuplimitedyes
VLAN supportyesyes
multiple device stackingyesyes