14 Configuring Virtual Machines #
Virtual Machine Manager's
view offers in-depth information about the VM Guest's complete configuration and hardware equipment. Using this view, you can also change the guest configuration or add and modify virtual hardware. To access this view, open the guest's console in Virtual Machine Manager and either choose › from the menu, or click in the toolbar.
The left panel of the window lists VM Guest overview and already installed
hardware. After clicking an item in the list, you can access its detailed
settings in the details view. You can change the hardware parameters to match
your needs, then click virt-manager
warns you about that fact.
To remove installed hardware from a VM Guest, select the appropriate list entry in the left panel and then click
in the bottom right of the window.To add new hardware, click
below the left panel, then select the type of the hardware you want to add in the window. Modify its parameters and confirm with .The following sections describe configuration options for the specific hardware type being added. They do not focus on modifying an existing piece of hardware as the options are identical.
14.1 Machine Setup #
This section describes the setup of the virtualized processor and memory hardware. These components are vital to a VM Guest, therefore you cannot remove them. It also shows how to view the overview and performance information, and how to change boot options.
14.1.1 Overview #
shows basic details about VM Guest and the hypervisor.
, , and are editable and help you identify VM Guest in the list of machines.
shows the universally unique identifier of the virtual machine, while shows its current status— , , or .
The
section shows the hypervisor type, CPU architecture, used emulator, and chipset type. None of the hypervisor parameters can be changed.14.1.2 Performance #
shows regularly updated charts of CPU and memory usage, and disk and network I/O.
Not all the charts in the
view are enabled by default. To enable these charts, go to › , then select › › , and check the charts that you want to see regularly updated.14.1.3 Processor #
includes detailed information about VM Guest processor configuration.
In the
section, you can configure several parameters related to the number of allocated CPUs.The real number of CPUs installed on VM Host Server.
The number of currently allocated CPUs. You can hotplug more CPUs by increasing this value up to the
value.Maximum number of allocatable CPUs for the current session. Any change to this value will take effect after the next VM Guest reboot.
The
section lets you configure the CPU model and topology.When activated, the
option uses the host CPU model for VM Guest. Otherwise you need to specify the CPU model from the drop-down box.After you activate
, you can specify a custom number of sockets, cores and threads for the CPU.14.1.4 Memory #
contains information about the memory that is available to VM Guest.
Total amount of memory installed on VM Host Server.
The amount of memory currently available to VM Guest. You can hotplug more memory by increasing this value up to the value of
.The maximum value to which you can hotplug the currently available memory. Any change to this value will take effect after the next VM Guest reboot.
14.1.5 Boot Options #
introduces options affecting the VM Guest boot process.
In the
section, you can specify whether the virtual machine should automatically start during the VM Host Server boot phase.In the
, activate the devices that will be used for booting VM Guest. You can change their order with the up and down arrow buttons on the right side of the list. To choose from a list of bootable devices on VM Guest start, activate .To boot a different kernel than the one on the boot device, activate
and specify the paths to the alternative kernel and initrd placed on the VM Host Server file system. You can also specify kernel arguments that will be passed to the loaded kernel.14.2 Storage #
This section gives you a detailed description of configuration options for storage devices. It includes both hard disks and removable media, such as USB or CD-ROM drives.
Click
below the left panel, then select from the window.Figure 14.9: Add a New Storage #To create a
qcow2
disk image in the default location, activate and specify its size in gigabytes.To gain more control over the disk image creation, activate Section 12.1, “Managing Storage with Virtual Machine Manager”.
and click to manage storage pools and images. The window opens which has almost identical functionality as the tab described inTip: Supported Storage FormatsSUSE only supports the following storage formats:
raw
,qcow2
, andqed
.After you manage to create and specify the disk image file, specify the
. It can be one of the following options:Select the
for your device. The list of available options depends on the device type you selected in the previous step. The types based on use paravirtualized drivers.In the Chapter 15, Disk Cache Modes.
section, select the preferred . For more information on cache modes, seeConfirm your settings with
. A new storage device appears in the left panel.
14.3 Controllers #
This section focuses on adding and configuring new controllers.
Click
below the left panel, then select from the window.Figure 14.10: Add a New Controller #Select the type of the controller. You can choose from
, , , , (paravirtualized), , or (smart card devices).Optionally, in the case of a USB or SCSI controller, select a controller model.
Confirm your settings with
. A new controller appears in the left panel.
14.4 Networking #
This section describes how to add and configure new network devices.
Click
below the left panel, then select from the window.Figure 14.11: Add a New Controller #From the Chapter 13, Managing Networks for more information on setting up virtual networks with Virtual Machine Manager.
list, select the source for the network connection. The list includes VM Host Server's available physical network interfaces, network bridges, or network bonds. You can also assign the VM Guest to an already defined virtual network. SeeSpecify a
for the network device. While Virtual Machine Manager pre-fills a random value for your convenience, it is recommended to supply a MAC address appropriate for your network environment to avoid network conflicts.Select a device model from the list. You can either leave the virtio uses paravirtualized drivers.
, or specify one of , , or models. Note thatConfirm your settings with
. A new network device appears in the left panel.
14.5 Enabling Seamless and Synchronized Mouse Pointer Movement #
When you click within a VM Guest's console with the mouse, the pointer is captured by the console window and cannot be used outside the console unless it is explicitly released (by pressing Alt–Ctrl). To prevent the console from grabbing the key and to enable seamless pointer movement between host and guest instead, add a tablet to the VM Guest.
Adding a tablet has the additional advantage of synchronizing the mouse pointer movement between VM Host Server and VM Guest when using a graphical environment on the guest. With no tablet configured on the guest, you will often see two pointers with one dragging behind the other.
Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the
view with › .Click
and choose and then in the pop-up window. Proceed with .If the guest is running, you will be asked whether to enable the tablet after the next reboot. Confirm with
.When you start or restart the VM Guest, the tablet becomes available in the VM Guest.
14.6 Adding a CD/DVD-ROM Device with Virtual Machine Manager #
KVM supports CD or DVD-ROMs in VM Guest either by directly accessing a
physical drive on the VM Host Server or by accessing ISO images. To create an ISO
image from an existing CD or DVD, use dd
:
dd if=/dev/CD_DVD_DEVICE of=my_distro.iso bs=2048
To add a CD/DVD-ROM device to your VM Guest, proceed as follows:
Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the
view with › .Click
and choose in the pop-up window.Change the
to .Select
.To assign the device to a physical medium, enter the path to the VM Host Server's CD/DVD-ROM device (for example,
/dev/cdrom
) next to . Alternatively, use to open a file browser and then click to select the device. Assigning the device to a physical medium is only possible when the Virtual Machine Manager was started on the VM Host Server.To assign the device to an existing image, click
to choose an image from a storage pool. If the Virtual Machine Manager was started on the VM Host Server, alternatively choose an image from another location on the file system by clicking . Select an image and close the file browser with .
Save the new virtualized device with
.Reboot the VM Guest to make the new device available. For more information, see Section 14.8, “Ejecting and Changing Floppy or CD/DVD-ROM Media with Virtual Machine Manager”.
14.7 Adding a Floppy Device with Virtual Machine Manager #
Currently KVM only supports the use of floppy disk images—using a
physical floppy drive is not supported. Create a floppy disk image from an
existing floppy using dd
:
dd if=/dev/fd0 of=/var/lib/libvirt/images/floppy.img
To create an empty floppy disk image use one of the following commands:
- Raw Image
dd if=/dev/zero of=/var/lib/libvirt/images/floppy.img bs=512 count=2880
- FAT Formatted Image
mkfs.msdos -C /var/lib/libvirt/images/floppy.img 1440
To add a floppy device to your VM Guest, proceed as follows:
Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the
view with › .Click
and choose in the pop-up window.Change the
to .Choose
and click to choose an existing image from a storage pool. If Virtual Machine Manager was started on the VM Host Server, alternatively choose an image from another location on the file system by clicking . Select an image and close the file browser with .Save the new virtualized device with
.Reboot the VM Guest to make the new device available. For more information, see Section 14.8, “Ejecting and Changing Floppy or CD/DVD-ROM Media with Virtual Machine Manager”.
14.8 Ejecting and Changing Floppy or CD/DVD-ROM Media with Virtual Machine Manager #
Whether you are using the VM Host Server's physical CD/DVD-ROM device or an
ISO/floppy image: Before you can change the media or image of an existing
device in the VM Guest, you first need to disconnect
the
media from the guest.
Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the
view with › .Choose the Floppy or CD/DVD-ROM device and “eject” the medium by clicking .
To “insert” a new medium, click .
If using the VM Host Server's physical CD/DVD-ROM device, first change the media in the device (this may require unmounting it on the VM Host Server before it can be ejected). Then choose
and select the device from the drop-down box.If you are using an ISO image, choose
and select an image by clicking . When connecting from a remote host, you may only choose images from existing storage pools.
Click
to finish. The new media can now be accessed in the VM Guest.
14.9 Editing VM Configuration with virsh
#
The configuration of a VM is stored in an XML file in
/etc/libvirtd/qemu/
and looks like this:
<domain type='kvm'> <name>sles15</name> <uuid>ab953e2f-9d16-4955-bb43-1178230ee625</uuid> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>2</vcpu> <os> <type arch='x86_64' machine='pc-i440fx-2.11'>hvm</type> </os> <features>...</features> <cpu mode='custom' match='exact' check='partial'> <model fallback='allow'>Skylake-Client-IBRS</model> </cpu> <clock>...</clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <pm> <suspend-to-mem enabled='no'/> <suspend-to-disk enabled='no'/> </pm> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'>...</disk> </devices> ... </domain>
If you want to edit the configuration of a VM Guest, check if it is offline:
tux >
sudo
virsh list --inactive
If your VM Guest is in this list, you can safely edit its configuration:
tux >
sudo
virsh edit NAME_OF_VM_GUEST
Before saving the changes, virsh
validates your input against a RelaxNG
schema.
14.10 Changing the Machine Type with virsh
#
By default, when installing with the virt-install
tool,
the machine type for VM Guest is pc-i440fx. The
machine type is stored in the VM Guest's xml configuration file in
/etc/libvirt/qemu/
in the tag type
:
<type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
As an example, the following procedure shows how to change this value to the
machine type q35
. q35
is an Intel*
chipset. It includes PCIe, supports up to
12 USB ports, and has support for SATA
and IOMMU. IRQ routing has also been
improved.
Check whether your VM Guest is inactive:
virsh list --inactive Id Name State ---------------------------------------------------- - sles11 shut off
Edit the configuration for this VM Guest:
virsh edit sles11
Change the value of the
machine
attribute:<type arch='x86_64' machine='pc-q35-2.0'>hvm</type>
Restart the VM Guest.
root #
virsh start sles11
Check that the machine type has changed. Log in to the VM Guest as root and run the following command:
root #
dmidecode | grep Product
Product Name: Standard PC (Q35 + ICH9, 2009)
Whenever the QEMU version on the host system is upgraded (for example,
when upgrading the VM Host Server to a new service pack), upgrade the machine
type of the VM Guests to the latest available version. To check, use the
command qemu-system-x86_64 -M help
on the VM Host Server.
The default machine type pc-i440fx
, for example, is
regularly updated. If your VM Guest still runs with a machine type of
pc-i440fx-1.X
, an update to
pc-i440fx-2.X
is strongly
recommended. This allows taking advantage of the most recent updates and
corrections in machine definitions, and ensures better future
compatibility.
14.11 Assigning a Host PCI Device to a VM Guest #
You can directly assign host-PCI devices to guests (PCI pass-through). When the PCI device is assigned to one VM Guest, it cannot be used on the host or by another VM Guest unless it is re-assigned. A prerequisite for this feature is a VM Host Server configuration as described in Important: Requirements for VFIO and SR-IOV.
14.11.1 Adding a PCI Device with Virtual Machine Manager #
The following procedure describes how to add a PCI device to a VM Guest using Virtual Machine Manager:
Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the
view with › .Click
and choose the category in the left panel. A list of available PCI devices appears in the right part of the window.Figure 14.12: Adding a PCI Device #From the list of available PCI devices, choose the one you want to pass to the guest. Confirm with
.
Although it is possible to assign a PCI device to a running VM Guest as described above, the device will not become available until you shut down the VM Guest and reboot it afterward.
14.11.2 Adding a PCI Device with virsh
#
To assign a PCI device to VM Guest with virsh
, follow
these steps:
Identify the host PCI device to assign to the guest. In the following example, we are assigning a DEC network card to the guest:
tux >
sudo lspci -nn
[...] 03:07.0 Ethernet controller [0200]: Digital Equipment Corporation DECchip \ 21140 [FasterNet] [1011:0009] (rev 22) [...]Note down the device ID (
03:07.0
in this case).Gather detailed information about the device using
virsh nodedev-dumpxml ID
. To get the ID, you need to replace colon and period in the device ID (03:07.0
) with underscore and prefix the result with “pci_0000_” (pci_0000_03_07_0
).tux >
virsh nodedev-dumpxml pci_0000_03_07_0 <device> <name>pci_0000_03_07_0</name> <path>/sys/devices/pci0000:00/0000:00:14.4/0000:03:07.0</path> <parent>pci_0000_00_14_4</parent> <driver> <name>tulip</name> </driver> <capability type='pci'> <domain>0</domain> <bus>3</bus> <slot>7</slot> <function>0</function> <product id='0x0009'>DECchip 21140 [FasterNet]</product> <vendor id='0x1011'>Digital Equipment Corporation</vendor> <numa node='0'/> </capability> </device>Note down the values for domain, bus, and function.
Detach the device from the host system prior to attaching it to VM Guest.
tux >
virsh nodedev-detach pci_0000_03_07_0 Device pci_0000_03_07_0 detachedTip: Multi-Function PCI DevicesWhen using a multi-function PCI device that does not support FLR (function level reset) or PM (power management) reset, you need to detach all its functions from the VM Host Server. The whole device must be reset for security reasons.
libvirt
will refuse to assign the device if one of its functions is still in use by the VM Host Server or another VM Guest.Convert the domain, bus, slot, and function value from decimal to hexadecimal, and prefix with
0x
to tell the system that the value is hexadecimal. In our example, domain = 0, bus = 3, slot = 7, and function = 0. Their hexadecimal values are:tux >
printf %x 0 0tux >
printf %x 3 3tux >
printf %x 7 7This results in domain = 0x0000, bus = 0x03, slot = 0x07 and function = 0x00.
Run
virsh edit
on your domain, and add the following device entry in the<devices>
section using the values from the previous step:<hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x03' slot='0x07' function='0x00'/> </source> </hostdev>
Tip:managed
Compared tounmanaged
libvirt
recognizes two modes for handling PCI devices: they can be eithermanaged
orunmanaged
. In the managed case,libvirt
handles all details of unbinding the device from the existing driver if needed, resetting the device, binding it tovfio-pci
before starting the domain, etc. When the domain is terminated or the device is removed from the domain,libvirt
will unbind fromvfio-pci
and rebind to the original driver in the case of a managed device. If the device is unmanaged, the user must ensure all of these management aspects of the device are done before assigning it to a domain, and after the device is no longer used by the domain.In the example above, the
managed='yes'
option means that the device is managed. To switch the device mode to unmanaged, setmanaged='no'
in the listing above. If you do so, you need to take care of the related driver with thevirsh nodedev-detach
andvirsh nodedev-reattach
commands. That means you need to runvirsh nodedev-detach pci_0000_03_07_0
prior to starting the VM Guest to detach the device from the host. In case the VM Guest is not running, you can make the device available for the host by runningvirsh nodedev-reattach pci_0000_03_07_0
.Shut down the VM Guest and restart it to make the assigned PCI device available.
Tip: SELinuxIf you are running SELinux on your VM Host Server, you need to disable it prior to starting the VM Guest with
setsebool -P virt_use_sysfs 1
14.12 Assigning a Host USB Device to a VM Guest #
Analogous to assigning host PCI devices (see Section 14.11, “Assigning a Host PCI Device to a VM Guest”), you can directly assign host USB devices to guests. When the USB device is assigned to one VM Guest, it cannot be used on the host or by another VM Guest unless it is re-assigned.
14.12.1 Adding a USB Device with Virtual Machine Manager #
To assign a host USB device to VM Guest using Virtual Machine Manager, follow these steps:
Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the
view with › .Click
and choose the category in the left panel. A list of available USB devices appears in the right part of the window.Figure 14.13: Adding a USB Device #From the list of available USB devices, choose the one you want to pass to the guest. Confirm with
. The new USB device appears in the left pane of the view.Tip: USB Device RemovalTo remove the host USB device assignment, click it in the left pane of the
view and confirm with .
14.12.2 Adding a USB Device with virsh
#
To assign a USB device to VM Guest using virsh
, follow
these steps:
Identify the host USB device to assign to the guest:
tux >
sudo lsusb
[...] Bus 001 Device 003: ID 0557:2221 ATEN International Co., Ltd Winbond Hermon [...]Note down the vendor and product IDs. In our example, the vendor ID is
0557
and the product ID is2221
.Run
virsh edit
on your domain, and add the following device entry in the<devices>
section using the values from the previous step:<hostdev mode='subsystem' type='usb'> <source startupPolicy='optional'> <vendor id='0557'/> <product id='2221'/> </source> </hostdev>
Tip: Vendor/Product or Device's AddressInstead of defining the host device with <vendor/> and <product/> IDs, you can use the <address/> element as described for host PCI devices in Section 14.11.2, “Adding a PCI Device with
virsh
”.Shut down the VM Guest and restart it to make the assigned USB device available.
Tip: SELinuxIf you are running SELinux on your VM Host Server, you need to disable it prior to starting the VM Guest with
setsebool -P virt_use_sysfs 1
14.13 Adding SR-IOV Devices #
Single Root I/O Virtualization (SR-IOV) capable PCIe devices can replicate their resources, so they appear to be multiple devices. Each of these "pseudo-devices" can be assigned to a VM Guest.
SR-IOV is an industry specification that was created by the Peripheral Component Interconnect Special Interest Group (PCI-SIG) consortium. It introduces physical functions (PF) and virtual functions (VF). PFs are full PCIe functions used to manage and configure the device. PFs also can move data. VFs lack the configuration and management part—they only can move data and a reduced set of configuration functions. Since VFs do not have all PCIe functions, the host operating system or the Hypervisor must support SR-IOV to be able to access and initialize VFs. The theoretical maximum for VFs is 256 per device (consequently the maximum for a dual-port Ethernet card would be 512). In practice this maximum is much lower, since each VF consumes resources.
14.13.1 Requirements #
The following requirements must be met to be able to use SR-IOV:
An SR-IOV-capable network card (as of SUSE Linux Enterprise Server 12 SP5, only network cards support SR-IOV)
An AMD64/Intel 64 host supporting hardware virtualization (AMD-V or Intel VT-x), see Section 7.3, “KVM Hardware Requirements” for more information
A chipset that supports device assignment (AMD-Vi or Intel VT-d)
libvirt-0.9.10 or better
SR-IOV drivers must be loaded and configured on the host system
A host configuration that meets the requirements listed at Important: Requirements for VFIO and SR-IOV
A list of the PCI addresses of the VF(s) that will be assigned to VM Guests
The information whether a device is SR-IOV-capable can be obtained from
its PCI descriptor by running lspci
. A device that
supports SR-IOV reports a capability similar to the
following:
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
Before adding an SR-IOV device to a VM Guest when initially setting it up, the VM Host Server already needs to be configured as described in Section 14.13.2, “Loading and Configuring the SR-IOV Host Drivers”.
14.13.2 Loading and Configuring the SR-IOV Host Drivers #
To be able to access and initialize VFs, an SR-IOV-capable driver needs to be loaded on the host system.
Before loading the driver, make sure the card is properly detected by running
lspci
. The following example shows thelspci
output for the dual-port Intel 82576NS network card:tux >
sudo /sbin/lspci | grep 82576 01:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01) 04:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01) 04:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)In case the card is not detected, it is likely that the hardware virtualization support in the BIOS/EFI has not been enabled.
Check whether the SR-IOV driver is already loaded by running
lsmod
. In the following example a check for the igb driver (for the Intel 82576NS network card) returns a result. That means the driver is already loaded. If the command returns nothing, the driver is not loaded.tux >
sudo /sbin/lsmod | egrep "^igb " igb 185649 0Skip this step if the driver is already loaded.
If the SR-IOV driver is not yet loaded, the non-SR-IOV driver needs to be removed first, before loading the new driver. Use
rmmod
to unload a driver. The following example unloads the non-SR-IOV driver for the Intel 82576NS network card:sudo /sbin/rmmod igbvf
Load the SR-IOV driver subsequently using the
modprobe
command—the VF parameter (max_vfs
) is mandatory:sudo /sbin/modprobe igb max_vfs=8
Or load the driver via SYSFS:
Find the PCI ID of the physical NIC by listing Ethernet devices:
tux >
sudo lspci | grep Eth 06:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10) 06:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)To enable VFs, echo the number of desired VFs to load to the
sriov_numvfs
parameter:tux >
sudo echo 1 > /sys/bus/pci/devices/0000:06:00.1/sriov_numvfsVerify that the VF NIC was loaded:
tux >
sudo lspci | grep Eth 06:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10) 06:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10) 06:08.0 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)Obtain the maximum number of VFs available:
tux >
sudo lspci -vvv -s 06:00.1 | grep 'Initial VFs' Initial VFs: 32, Total VFs: 32, Number of VFs: 0, Function Dependency Link: 01Create a
before.service
file which loads VF via SYSFS on boot:[Unit] Before= After=network-online.target [Service] Type=oneshot RemainAfterExit=true ExecStart=/bin/bash -c "echo 1 > /sys/bus/pci/devices/0000:06:00.1/sriov_numvfs" # beware, executable is run directly, not through a shell, check the man pages # systemd.service and systemd.unit for full syntax [Install] # target in which to start the service WantedBy=multi-user.target #WantedBy=graphical.target
And copy it to
/etc/systemd/system
.Additionally, it is required to create another service file (
after-local.service
) pointing to/etc/init.d/after.local
script that detaches the NIC prior to starting the VM, otherwise the VM would fail to start:[Unit] Description=/etc/init.d/after.local Compatibility After=libvirtd.service Requires=libvirtd.service [Service] Type=oneshot ExecStart=/etc/init.d/after.local RemainAfterExit=true [Install] WantedBy=multi-user.target
And copy it to
/etc/systemd/system
.#! /bin/sh # # Copyright (c) 2010 SuSE LINUX Products GmbH, Germany. All rights reserved. # ... virsh nodedev-detach pci_0000_06_08_0
Then save it as
/etc/init.d/after.local
.Reboot the machine and check if the SR-IOV driver is loaded by re-running the
lspci
command from the first step of this procedure. If the SR-IOV driver was loaded successfully you should see additional lines for the VFs:01:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01) 01:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 01:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 01:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) [...] 04:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01) 04:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01) 04:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 04:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 04:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) [...]
14.13.3 Adding a VF Network Device to an Existing VM Guest #
When the SR-IOV hardware is properly set up on the VM Host Server, you can add VFs to VM Guests. To do so, you need to collect some data first.
Note: The following procedure is using example data. Make sure to replace it by appropriate data from your setup.
Use the
virsh nodedev-list
command to get the PCI address of the VF you want to assign and its corresponding PF. Numerical values from thelspci
output shown in Section 14.13.2, “Loading and Configuring the SR-IOV Host Drivers” (for example01:00.0
or04:00.1
) are transformed by adding the prefix "pci_0000_" and by replacing colons and dots with underscores. So a PCI ID listed as "04:00.0" bylspci
is listed as "pci_0000_04_00_0" by virsh. The following example lists the PCI IDs for the second port of the Intel 82576NS network card:tux >
sudo virsh nodedev-list | grep 0000_04_ pci_0000_04_00_0 pci_0000_04_00_1 pci_0000_04_10_0 pci_0000_04_10_1 pci_0000_04_10_2 pci_0000_04_10_3 pci_0000_04_10_4 pci_0000_04_10_5 pci_0000_04_10_6 pci_0000_04_10_7 pci_0000_04_11_0 pci_0000_04_11_1 pci_0000_04_11_2 pci_0000_04_11_3 pci_0000_04_11_4 pci_0000_04_11_5The first two entries represent the PFs, whereas the other entries represent the VFs.
Get more data that will be needed by running the command
virsh nodedev-dumpxml
on the PCI ID of the VF you want to add:tux >
sudo virsh nodedev-dumpxml pci_0000_04_10_0 <device> <name>pci_0000_04_10_0</name> <parent>pci_0000_00_02_0</parent> <capability type='pci'> <domain>0</domain> <bus>4</bus> <slot>16</slot> <function>0</function> <product id='0x10ca'>82576 Virtual Function</product> <vendor id='0x8086'>Intel Corporation</vendor> <capability type='phys_function'> <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </capability> </capability> </device>The following data is needed for the next step:
<domain>0</domain>
<bus>4</bus>
<slot>16</slot>
<function>0</function>
Create a temporary XML file (for example
/tmp/vf-interface.xml
containing the data necessary to add a VF network device to an existing VM Guest. The minimal content of the file needs to look like the following:<interface type='hostdev'>1 <source> <address type='pci' domain='0' bus='11' slot='16' function='0'2/>2 </source> </interface>
VFs do not get a fixed MAC address; it changes every time the host reboots. When adding network devices the “traditional” way with <hostdev>, it would require to reconfigure the VM Guest's network device after each reboot of the host, because of the MAC address change. To avoid this kind of problem, libvirt introduced the “interface type='hostdev'” directive, which sets up network-specific data before assigning the device.
Specify the data you acquired in the previous step here.
In case a device is already attached to the host, it cannot be attached to a guest. To make it available for guests, detach it from the host first:
virsh nodedev-detach pci_0000_04_10_0
Last, add the VF interface to an existing VM Guest:
virsh attach-device GUEST /tmp/vf-interface.xml --OPTION
GUEST needs to be replaced by the domain name, ID or UUID of the VM Guest and --OPTION can be one of the following:
--persistent
This option will always add the device to the domain's persistent XML. In addition, if the domain is running, it will be hotplugged.
--config
This option will only affect the persistent XML, even if the domain is running. The device will only show up in the guest on next boot.
--live
This option will only affect a running domain. If the domain is inactive, the operation will fail. The device is not persisted in the XML and will not be available in the guest on next boot.
- --current
This option affects the current state of the domain. If the domain is inactive, the device is added to the persistent XML and will be available on next boot. If the domain is active, the device is hotplugged but not added to the persistent XML.
To detach a VF interface, use the
virsh detach-device
command, which also takes the options listed above.
14.13.4 Dynamic Allocation of VFs from a Pool #
If you define the PCI address of a VF into a guest's configuration statically as described in Section 14.13.3, “Adding a VF Network Device to an Existing VM Guest”, it is hard to migrate such guest to another host. The host must have identical hardware in the same location on the PCI bus, or the guest configuration must be modified prior to each start.
Another approach is to create a libvirt
network with a device pool that
contains all the VFs of an SR-IOV device. The guest
then references this network, and each time it is started, a single VF is
dynamically allocated to it. When the guest is stopped, the VF is returned
to the pool, available for another guest.
14.13.4.1 Defining Network with Pool of VFs on VM Host Server #
The following example of network definition creates a pool of all VFs for the SR-IOV device with its physical function (PF) at the network interface eth0 on the host:
<network> <name>passthrough</name> <forward mode='hostdev' managed='yes'> <pf dev='eth0'/> </forward> </network>
To use this network on the host, save the above code to a file, for
example /tmp/passthrough.xml
, and execute the
following commands. Remember to replace eth0 with the real network
interface name of your SR-IOV device's PF:
virsh net-define /tmp/passthrough.xml virsh net-autostart passthrough virsh net-start passthrough
14.13.4.2 Configuring VM Guest to Use VF from the Pool #
The following example of guest device interface definition uses a VF of
the SR-IOV device from the pool created in
Section 14.13.4.1, “Defining Network with Pool of VFs on VM Host Server”. libvirt
automatically
derives the list of all VFs associated with that PF the first time the
guest is started.
<interface type='network'> <source network='passthrough'> </interface>
To verify the list of associated VFs, run virsh net-dumpxml
passthrough
on the host after the first guest that uses the
network with the pool of VFs starts.
<network connections='1'> <name>passthrough</name> <uuid>a6a26429-d483-d4ed-3465-4436ac786437</uuid> <forward mode='hostdev' managed='yes'> <pf dev='eth0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x1'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x3'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x5'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x7'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x11' function='0x1'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x11' function='0x3'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x11' function='0x5'/> </forward> </network>
14.14 Using Macvtap to Share VM Host Server Network Interfaces #
Macvtap provides direct attachment of a VM Guest virtual interface to a host network interface. The macvtap-based interface extends the VM Host Server network interface and has its own MAC address on the same Ethernet segment. Typically, this is used to make both the VM Guest and the VM Host Server show up directly on the switch that the VM Host Server is connected to.
Macvtap cannot be used with network interfaces already connected to a Linux bridge. Before attempting to create the macvtap interface, remove the interface from the bridge.
When using macvtap, a VM Guest can communicate with other VM Guests, and with other external hosts on the network. But it cannot communicate with the VM Host Server on which the VM Guest runs. This is the defined behavior of macvtap, because of the way the VM Host Server's physical Ethernet is attached to the macvtap bridge. Traffic from the VM Guest into that bridge that is forwarded to the physical interface cannot be bounced back up to the VM Host Server's IP stack. Similarly, traffic from the VM Host Server's IP stack that is sent to the physical interface cannot be bounced back up to the macvtap bridge for forwarding to the VM Guest.
Virtual network interfaces based on macvtap are supported by libvirt by
specifying an interface type of direct
. For example:
<interface type='direct'> <mac address='aa:bb:cc:dd:ee:ff'/> <source dev='eth0' mode='bridge'/> <model type='virtio'/> </interface>
The operation mode of the macvtap device can be controlled with the
mode
attribute. The following list shows its possible
values and a description for each:
vepa
: All VM Guest packets are sent to an external bridge. Packets whose destination is a VM Guest on the same VM Host Server as where the packet originates from are sent back to the VM Host Server by the VEPA capable bridge (today's bridges are typically not VEPA capable).bridge
: Packets whose destination is on the same VM Host Server as where they originate from are directly delivered to the target macvtap device. Both origin and destination devices need to be inbridge
mode for direct delivery. If either one of them is invepa
mode, a VEPA capable bridge is required.private
: All packets are sent to the external bridge and will only be delivered to a target VM Guest on the same VM Host Server if they are sent through an external router or gateway and that device sends them back to the VM Host Server. This procedure is followed if either the source or destination device is in private mode.passthrough
: A special mode that gives more power to the network interface. All packets will be forwarded to the interface, allowing virtio VM Guests to change the MAC address or set promiscuous mode to bridge the interface or create VLAN interfaces on top of it. Note that a network interface is not shareable inpassthrough
mode. Assigning an interface to a VM Guest will disconnect it from the VM Host Server. For this reason SR-IOV virtual functions are often assigned to the VM Guest inpassthrough
mode.
14.15 Managing Guest Memory Allocation (Xen only) #
libvirt
now includes support for adjusting memory
allocation per guest with virsh
. Xen paravirtual
devices connect to the xenbus
controller, which is
analogous to a physical device bus such as a PCI controller. Xen's
max_grant_frames
attribute sets how many frames, which
are analogous to memory, are allocated to the xenbus
controller for each guest.
The default is 32, and this can be increased as needed, up to the amount set
for dom0, or decreased. How much is enough? This depends on the number of
devices and workload demand, such as a saturated network interface or heavy
I/O. Use xen-diag
to see your current used and maximum
max_grant_frames
values for dom0 and your guests. The
guests must be running:
tux >
sudo
virsh list Id Name State -------------------------------- 0 Domain-0 running 3 sle15sp1 runningtux >
sudo
xen-diag gnttab_query_size 0 domid=0: nr_frames=1, max_nr_frames=256tux >
sudo
xen-diag gnttab_query_size 3 domid=3: nr_frames=3, max_nr_frames=32
The sle15sp1 guest is using only 3 frames out of 32. If you are seeing
performance issues, and log entries that point to insufficient frames,
increase the value with virsh
. Look for the
<controller type='xenbus'
line in the guest's
configuration file and add the maxGrantFrames
control
element:
tux >
sudo
virsh edit sle15sp1 <controller type='xenbus' index='0' maxGrantFrames='40'/>
Save your changes and restart the guest. Now it should show your change:
tux >
sudo
xen-diag gnttab_query_size 3 domid=3: nr_frames=3, max_nr_frames=40
See the Controllers section of the libvirt Domain XML format manual at https://libvirt.org/formatdomain.html#elementsControllers for more information.
14.16 Disabling a Memory Balloon Device #
Memory Balloon has become a default option for KVM. The device will be added
to the VM Guest explicitly, so you do not need to add this element in the
VM Guest's XML configuration. However, if you want to disable Memory
Balloon in the VM Guest for any reason, you need to set
model='none'
as shown below:
<devices> <memballoon model='none'/> </device>
14.17 Configuring Multiple Monitors (Dual Head) #
libvirt
supports a dual head configuration to display the video output of
the VM Guest on multiple monitors.
The Xen hypervisor does not support dual head configuration.
While the virtual machine is running, verify that the xf86-video-qxl package is installed in the VM Guest:
tux >
rpm -q xf86-video-qxlShut down the VM Guest and start editing its configuration XML as described in Section 14.9, “Editing VM Configuration with
virsh
”.Verify that the model of the virtual graphics card is 'qxl':
<video> <model type='qxl' ... />
Increase the
heads
parameter in the graphics card model specification from the default1
to2
, for example:<video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='2' primary='yes'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video>
Configure the virtual machine to use the Spice display instead of VNC:
<graphics type='spice' port='5916' autoport='yes' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics>
Start the virtual machine and connect to its display with
virt-viewer
, for example:tux >
virt-viewer --connect qemu+ssh://USER@VM_HOST/systemFrom the list of VMs, select the one whose configuration you have modified and confirm with
.After the graphical subsystem (Xorg) loads in the VM Guest, select
› › to open a new window with the second monitor's output.