Connecting Virtual Machines to a VXLAN

Publication Date: 12 Dec 2024

WHAT?

The VXLAN protocol can address more isolated virtual networks than the traditional VLAN. Within these networks, groups of virtual machines (VMs) can safely communicate even across regions.

WHY?

You want to connect virtual machines to VXLAN networks in a large data center and configure them to communicate with other VMs in a dedicated network space.

EFFORT

30 minutes is enough to learn and configure a VXLAN to enable communication between VMs.

REQUIREMENTS

A SLES libvirt-based VM Host Server with an Ethernet network connection configured by NetworkManager.

Revision History: Connecting Virtual Machines to a VXLAN

1 Introduction #

1.1 What is a VLAN? #

A virtual local area network (VLAN) is a domain isolated within a local area network (LAN) at the data link layer (layer-2). VLANs help keep network applications separate despite being connected to the same physical network. Also, there is no need for additional sets of cabling and networking devices to be deployed.

1.2 What is a VXLAN? #

A virtual extensible local area network (VXLAN) is a network virtualization technology that addresses scalability limitations with large cloud computing deployments. While the VLAN protocol can address approximately four thousand virtual networks, VXLAN can address over 16 million virtual networks.

1.3 How does a VXLAN work? #

A VXLAN creates network tunnels for VM communication. It encapsulates layer-2 network traffic within the UDP datagrams, using the default destination port number 4789. VMs that are the endpoints of the tunnel can communicate across different hosts or even data centers around the world.

Figure 1: Simple VXLAN scheme #

1.4 Benefits of using VXLAN #

VXLAN offers the following important benefits:

With VXLAN, you can create over 16 million separated virtual networks. As a comparison, the traditional VLAN offers approximately four thousand networks.
Compared to VLANs which operate in data link layer (layer-2), VXLAN encapsulates layer-2 frames within IP packets to make use of existing network layer (layer-3) networks. This results in easier routing and administration, enabling VMs to connect to logical networks across different hosts or locations.
While the VLAN protocol requires a router or a layer-3 switch to enable communication between hosts in different VLANs, VXLAN does not require reconfiguring the existing network equipment as it spans across layer-3 boundaries and uses logical addressing, routing and packet forwarding.

1.5 Procedure outline #

To connect a VM to a VXLAN network, you need to complete the following steps on the VM Host Server:

Create a network bridge with a VXLAN attached.
Create a virtual network attached to the existing bridge.
Configure the VM to use the VXLAN network via the virtual network attached to the network bridge.

2 Creating a network bridge with a VXLAN attached #

To make the communication with VXLAN more transparent, virtual machines use a network bridge to which the VXLAN is attached. Run the following procedure on the VM Host Server.

Create the network bridge br0 with no IPv4 and IPv6 addresses assigned.
```
> sudo nmcli connection add type bridge con-name br0 ifname br0 \
ipv4.method disabled1 ipv6.method disabled2
```
1
Prevents assigning an IPv4 address on the bridge.
2
Prevents assigning an IPv6 address on the bridge.
Tip: Custom destination port
NetworkManager uses 8472 as the default destination port. To specify a different destination port, pass the option destination-port PORT_NUMBER to the command.

Create a VXLAN network interface attached to the bridge br0.

> sudo nmcli connection add type vxlan slave-type bridge con-name vxlan1-br0 \
ifname vxlan11 id 12 local 192.10.3.13 \
remote 10.5.0.24 master br05

1	Specifies the interface name for the VXLAN connection. This is the name that will be assigned to the VXLAN interface.
2	Specifies a unique numeric VXLAN identifier to differentiate between different VXLAN networks.
3	Specifies the local IP address to be used for the VXLAN interface. This is the IP address that NetworkManager will use for the local VXLAN endpoint. This address must be reachable by the VMs that will be using the VXLAN network.
4	Specifies the remote IP address of the VXLAN endpoint with which the local VXLAN interface will communicate. This address must be reachable by the VM Host Server that hosts the VMs that will be using the VXLAN network.
5	Specifies the name of the bridge device to which the VXLAN interface will be attached. This is typically the bridge device that acts as the VXLAN endpoint.

Tip: Multiple VXLANs attached to a single bridge

You can attach multiple VXLAN interfaces to a single bridge device. In this case, VMs on a single VM Host Server can belong to different VXLANs.

Activate the bridge br0.
```
> sudo nmcli connection up br0
```
If firewalld is active, modify its rules to allow incoming UDP connections via port 8472.
```
# firewall-cmd --permanent --add-port=8472/udp && firewall-cmd --reload
```

3 Creating a virtual network #

Virtual machines need a virtual network that uses the network bridge br0 to be able to communicate via a VXLAN attached to that bridge. Run the following procedure on the VM Host Server to add such a virtual network to the libvirtd service.

Requirements #

You installed libvirt virtualization tools and the libvirtd service is enabled and started.
You configured the network bridge br0 with the VXLAN attached on SLES.

Create a temporary XML file (/tmp/vxlan1-br0.xml) that defines a new virtual network. The file should be similar to the following one:
```
<network>
<name>vxlan1-br0</name>
<forward mode="bridge" />
<bridge name="br0" />
</network>
```
Use the XML file to create a new libvirt-based virtual network.
```
> sudo virsh net-define /tmp/vxlan1-br0.xml
```
(Optional) Remove the XML definition file from disk. It is no longer needed.
```
> rm /tmp/vxlan1-br0.xml
```
Start the new vxlan1-br0 virtual network and configure it to start automatically when the libvirtd service starts.
```
> sudo virsh net-start vxlan1-br0
> sudo virsh net-autostart vxlan1-br0
```

Verify the status of the newly created network. If the newly created virtual network is listed as active, the configuration was successful.

> sudo virsh net-list
Name              State    Autostart   Persistent
----------------------------------------------------
vxlan1-br0        active   yes         yes

4 Configuring virtual machines #

You need to configure virtual machines to use the virtual network vxlan1-br0 to communicate via a network bridge with an attached VXLAN network.

Tip: Configuring new VMs to use VXLAN

To attach a new VM to a VXLAN network, configure it to use the vxlan1-br0 network when creating the VM. If you use the virt-install tool to create VMs, for example, pass the --network network:vxlan1-br0 option to it.

The following procedure describes how to adjust an existing VM. Our example virtual machine VM1 is running on host SLES-HOST-A.

Requirements #

You created a VM using libvirt.
You configured the virtual network vxlan1-br0 using libvirt.

Connect the network interface of the VM to the virtual network vxlan1-br0.
```
> sudo virt-xml VM1 --edit --network network=vxlan1-br0
```

Restart the VM, for example:

> sudo virsh shutdown VM1
> sudo virsh start VM1

Verify the virtual network interfaces on the host.

> sudo virsh domiflist VM1
Interface   Type     Source           Model    MAC
-------------------------------------------------------------------
vnet11    bridge   vxlan1-br02    virtio   52:54:12:a7:89:1f
[...]

1	A virtual network automatically created by `libvirt`. It is used by the virtual machine VM1.
2	A network bridge with the attached VXLAN network. The `vnet1` network is connected to that bridge.

Verify the interface attached to the vxlan1-br0 network bridge on the host.

> sudo ip link show master vxlan1-br0
[...]
14: vxlan1:1 <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 2a:53:bd:d5:b3:0a brd ff:ff:ff:ff:ff:ff
15: vnet1:2 <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:12:a7:89:1f brd ff:ff:ff:ff:ff:ff

1	The configured VXLAN network attached to the `vxlan1-br0` bridge.
2	A virtual network automatically created by `libvirt`. It is used by the virtual machine VM1.

5 Legal Notice #

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”.

For SUSE trademarks, see https://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.

All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors, nor the translators shall be held liable for possible errors or the consequences thereof.

1	Prevents assigning an IPv4 address on the bridge.
2	Prevents assigning an IPv6 address on the bridge.