25 Setting up a virtual machine host #
This section documents how to set up and use SUSE Linux Enterprise Server 15 SP6 as a virtual machine host.
The hardware requirements for the Dom0 are often the same as those for the SUSE Linux Enterprise Server operating system. Additional CPU, disk, memory and network resources should be added to accommodate the resource demands of all planned VM Guest systems.
Remember that VM Guest systems, like physical machines, perform better when they run on faster processors and have access to more system memory.
The virtual machine host requires several software packages and their dependencies to be installed. To install all necessary packages, run YaST
, select › and choose for installation. The installation can also be performed with YaST using the module › .After the Xen software is installed, restart the computer and, on the boot screen, choose the newly added option with the Xen kernel.
Updates are available through your update channel. To be sure to have the latest updates installed, run YaST
after the installation has finished.25.1 Best practices and suggestions #
When installing and configuring the SUSE Linux Enterprise Server operating system on the host, be aware of the following best practices and suggestions:
If the host should always run as Xen host, run YaST
› and activate the Xen boot entry as default boot section.In YaST, click
.Change the default boot to the
label, then click .Click
.
For best performance, only the applications and processes required for virtualization should be installed on the virtual machine host.
If you intend to use a watchdog device attached to the Xen host, use only one at a time. It is recommended to use a driver with actual hardware integration over a generic software one.
The Dom0 kernel is running virtualized, so tools like
irqbalance
or lscpu
do not
reflect the real hardware characteristics.
Trusted boot (Tboot) is not supported by Xen. To ensure that the Xen host boots correctly, verify that the
option is deactivated in the GRUB 2 configuration dialog.25.2 Managing Dom0 memory #
In previous versions of SUSE Linux Enterprise Server, the default memory allocation scheme of a Xen host was to allocate all host physical memory to Dom0 and enable auto-ballooning. Memory was automatically ballooned from Dom0 when additional domains were started. This behavior has always been error prone and disabling it was strongly encouraged. Starting with SUSE Linux Enterprise Server 15 SP1, auto-ballooning has been disabled by default and Dom0 is given 10% of host physical memory + 1 GB. For example, on a host with 32 GB of physical memory, 4.2 GB of memory is allocated for Dom0.
The use of the dom0_mem
Xen command line option in
/etc/default/grub
is still supported and encouraged.
You can restore the old behavior by setting dom0_mem
to
the host physical memory size and enabling the
autoballoon
setting in
/etc/xen/xl.conf
.
The amount of memory reserved for Dom0 is a function of the number of VM Guests running on the host since Dom0 provides back-end network and disk I/O services for each VM Guest. Other workloads running in Dom0 should also be considered when calculating Dom0 memory allocation. Memory sizing of Dom0 should be determined like any other virtual machine.
25.2.1 Setting Dom0 memory allocation #
Determine memory allocation required for Dom0.
At Dom0, type
xl info
to view the amount of memory that is available on the machine. Dom0's current memory allocation can be determined with thexl list
command.Edit
/etc/default/grub
and adjust theGRUB_CMDLINE_XEN
option so that it includesdom0_mem=MEM_AMOUNT
. Replace MEM_AMOUNT with the maximum amount of memory to allocate to Dom0. AddK
,M
, orG
, to specify the size unit. For example:GRUB_CMDLINE_XEN="dom0_mem=2G"
Restart the computer to apply the changes.
Refer to Abschnitt 18.2.2, „Die Datei /etc/default/grub
“ for more
details about Xen-related boot configuration options.
When using the XL tool stack and the dom0_mem=
option for the Xen hypervisor in GRUB 2 you need to disable xl
autoballoon in
etc/xen/xl.conf
. Otherwise launching VMs fails
with errors about not being able to balloon down Dom0. So add
autoballoon=0 to xl.conf
if
you have the dom0_mem=
option specified for Xen.
Also see
Xen
dom0 memory
25.3 Network card in fully virtualized guests #
In a fully virtualized guest, the default network card is an emulated Realtek network card. However, it also possible to use the split network driver to run the communication between Dom0 and a VM Guest. By default, both interfaces are presented to the VM Guest, because the drivers of certain operating systems require both to be present.
When using SUSE Linux Enterprise Server, only the paravirtualized network cards are available for the VM Guest by default. The following network options are available:
- emulated
To use an emulated network interface like an emulated Realtek card, specify
type=ioemu
in thevif
device section of the domain xl configuration. An example configuration would look like:vif = [ 'type=ioemu,mac=00:16:3e:5f:48:e4,bridge=br0' ]
Find more details about the xl configuration in the
xl.conf
man pageman 5 xl.conf
.- paravirtualized
When you specify
type=vif
and do not specify a model or type, the paravirtualized network interface is used:vif = [ 'type=vif,mac=00:16:3e:5f:48:e4,bridge=br0,backen=0' ]
- emulated and paravirtualized
If the administrator should be offered both options, simply specify both type and model. The xl configuration would look like:
vif = [ 'type=ioemu,mac=00:16:3e:5f:48:e4,model=rtl8139,bridge=br0' ]
In this case, one of the network interfaces should be disabled on the VM Guest.
25.4 Starting the virtual machine host #
If virtualization software is correctly installed, the computer boots to display the GRUB 2 boot loader with a
option on the menu. Select this option to start the virtual machine host.
When booting a Xen system, you may observe error messages in the
/var/log/messages log file or systemd
journal of dom0 similar to
following:
isst_if_mbox_pci: probe of 0000:ff:1e.1 failed with error -5 isst_if_pci: probe of 0000:fe:00.1 failed with error -5
Ignore them as they are harmless and are caused by the fact that the ISST driver does not provide any power or frequency scaling feature for virtual machines.
In Xen, the hypervisor manages the memory resource. If you need to
reserve system memory for a recovery kernel in Dom0, this memory needs
to be reserved by the hypervisor. Thus, it is necessary to add
crashkernel=size
to the
GRUB_CMDLINE_XEN_DEFAULT
variable in the
/etc/dfault/grub
file, save it and run the
following command:
>
sudo
grub2-mkconfig -o /boot/grub2/grub.cfg
For more information on the crashkernel parameter, see
Section 20.4, “Calculating crashkernel
allocation size”.
If the
option is not on the GRUB 2 menu, review the steps for installation and verify that the GRUB 2 boot loader has been updated. If the installation has been done without selecting the Xen pattern, run the YaST , select the filter and choose for installation.After booting the hypervisor, the Dom0 virtual machine starts and displays its graphical desktop environment. If you did not install a graphical desktop, the command line environment appears.
Sometimes it may happen that the graphics system does not work
properly. In this case, add vga=ask
to the boot
parameters. To activate permanent settings, use
vga=mode-0x???
where ???
is
calculated as 0x100
+ VESA mode from
https://en.wikipedia.org/wiki/VESA_BIOS_Extensions,
for example, vga=mode-0x361
.
Before starting to install virtual guests, make sure that the system time is correct. To do this, configure NTP (Network Time Protocol) on the controlling domain:
In YaST select
› .Select the option to automatically start the NTP daemon during boot. Provide the IP address of an existing NTP time server, then click
.
Hardware clocks are not precise. All modern operating systems try to
correct the system time compared to the hardware time bvia an
additional time source. To get the correct time on all VM Guest
systems, also activate the network time services on each respective
guest or make sure that the guest uses the system time of the host. For
more about Independent Wallclocks
in SUSE Linux Enterprise Server
see Section 20.2, “Xen virtual machine clock settings”.
For more information about managing virtual machines, see Chapter 27, Managing a virtualization environment.
25.5 PCI Pass-Through #
To take full advantage of VM Guest systems, it is sometimes necessary to assign specific PCI devices to a dedicated domain. When using fully virtualized guests, this functionality is only available if the chipset of the system supports this feature, and if it is activated from the BIOS.
This feature is available from both AMD* and Intel*. For AMD machines, the feature is called IOMMU. In Intel speak, this is VT-d. Be aware that Intel-VT technology is not sufficient to use this feature for fully virtualized guests. To make sure that your computer supports this feature, ask your supplier specifically to deliver a system that supports PCI Pass-Through.
Certain graphics drivers use highly optimized ways to access DMA. This is not supported, and thus using graphics cards may be difficult.
When accessing PCI devices behind a PCIe bridge, all the PCI devices must be assigned to a single guest. This limitation does not apply to PCIe devices.
Guests with dedicated PCI devices cannot be migrated live to a different host.
The configuration of PCI Pass-Through is twofold. First, the hypervisor must be informed at boot time that a PCI device should be available for reassigning. Second, the PCI device must be assigned to the VM Guest.
25.5.1 Configuring the hypervisor for PCI Pass-Through #
Select a device to reassign to a VM Guest. To do this, run
lspci -k
, and read the device number and the name of the original module that is assigned to the device:06:01.0 Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 05) Subsystem: Dell Device 0617 Kernel driver in use: e1000e Kernel modules: e1000e
In this case, the PCI number is
(06:01.0)
and the dependent kernel module ise1000e
.Specify a module dependency to ensure that
xen_pciback
is the first module to control the device. Add the file named/etc/modprobe.d/50-e1000e.conf
with the following content:install e1000e /sbin/modprobe xen_pciback ; /sbin/modprobe \ --first-time --ignore-install e1000e
Instruct the
xen_pciback
module to control the device using thehide
option. Edit or create/etc/modprobe.d/50-xen-pciback.conf
with the following content:options xen_pciback hide=(06:01.0)
Reboot the system.
Check if the device is in the list of assignable devices with the command
xl pci-assignable-list
25.5.1.1 Dynamic assignment with xl #
To avoid restarting the host system, you can use dynamic assignment with xl to use PCI Pass-Through.
Begin by making sure that dom0 has the pciback module loaded:
>
sudo
modprobe pciback
Then make a device assignable by using xl
pci-assignable-add
. For example, to make the device
06:01.0 available for guests, run the command:
>
sudo
xl pci-assignable-add 06:01.0
25.5.2 Assigning PCI devices to VM Guest systems #
There are several possibilities to dedicate a PCI device to a VM Guest:
- Adding the device while installing:
During installation, add the
pci
line to the configuration file:pci=['06:01.0']
- Hotplugging PCI devices to VM Guest systems
The command
xl
can be used to add or remove PCI devices on the fly. To add the device with number06:01.0
to a guest with namesles12
use:xl pci-attach sles12 06:01.0
- Adding the PCI device to Xend
To add the device to the guest permanently, add the following snippet to the guest configuration file:
pci = [ '06:01.0,power_mgmt=1,permissive=1' ]
After assigning the PCI device to the VM Guest, the guest system must care for the configuration and device drivers for this device.
25.5.3 VGA Pass-Through #
Xen 4.0 and newer supports VGA graphics adapter pass-through on fully virtualized VM Guests. The guest can take full control of the graphics adapter with high-performance full 3D and video acceleration.
VGA Pass-Through functionality is similar to PCI Pass-Through and as such also requires IOMMU (or Intel VT-d) support from the mainboard chipset and BIOS.
Only the primary graphics adapter (the one that is used when you power on the computer) can be used with VGA Pass-Through.
VGA Pass-Through is supported only for fully virtualized guests. Paravirtual guests (PV) are not supported.
The graphics card cannot be shared between multiple VM Guests using VGA Pass-Through—you can dedicate it to one guest only.
To enable VGA Pass-Through, add the following settings to your fully virtualized guest configuration file:
gfx_passthru=1 pci=['yy:zz.n']
where yy:zz.n
is the PCI controller ID of the VGA
graphics adapter as found with lspci -v
on Dom0.
25.5.4 Troubleshooting #
In certain circumstances, problems may occur during the installation of the VM Guest. This section describes several known problems and their solutions.
- During boot, the system hangs
The software I/O translation buffer allocates a large chunk of low memory early in the bootstrap process. If the requests for memory exceed the size of the buffer, it may result in a hung boot process. To check if this is the case, switch to console 10 and check the output there for a message similar to
kernel: PCI-DMA: Out of SW-IOMMU space for 32768 bytes at device 000:01:02.0
In this case, you need to increase the size of the
swiotlb
. Addswiotlb=VALUE
(where VALUE is specified as the number of slab entries) on the command line of Dom0. The number can be adjusted up or down to find the optimal size for the machine.
The swiotlb=force
kernel parameter is required for
DMA access to work for PCI devices on a PV guest. For more
information about IOMMU and the swiotlb
option see
the file boot-options.txt
from the package
kernel-source
.
25.5.5 More information #
There are several resources on the Internet that provide interesting information about PCI Pass-Through:
25.6 USB pass-through #
There are two methods for passing through individual host USB devices to a guest. The first is via an emulated USB device controller, the second is using PVUSB.
25.6.1 Identify the USB device #
Before you can pass through a USB device to the VM Guest, you need to
identify it on the VM Host Server. Use the lsusb
command
to list the USB devices on the host system:
#
lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 003: ID 0461:4d15 Primax Electronics, Ltd Dell Optical Mouse
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
To pass through the Dell mouse, for example, specify either the device
tag in the form vendor_id:device_id
(0461:4d15) or
the bus address in the form bus.device
(2.3).
Remember to remove leading zeros, otherwise xl
would
interpret the numbers as octal values.
25.6.2 Emulated USB device #
In emulated USB, the device model (QEMU) presents an emulated USB controller to the guest. The USB device is then controlled from Dom0 while USB commands are translated between the VM Guest and the host USB device. This method is only available to fully virtualized domains (HVM).
Enable the emulated USB hub with the usb=1
option.
Then specify devices among the list of devices in the configuration
file along with other emulated devices by using
host:USBID
. For example:
usb=1 usbdevice=['tablet','host:2.3','host:0424:460']
25.6.3 Paravirtualized PVUSB #
PVUSB is a new high performance method for USB Pass-Through from dom0 to the virtualized guests. With PVUSB, there are two ways to add USB devices to a guest:
via the configuration file at domain creation time
via hotplug while the VM is running
PVUSB uses paravirtualized front- and back-end interfaces. PVUSB supports USB 1.1 and USB 2.0, and it works for both PV and HVM guests. To use PVUSB, you need usbfront in your guest OS, and usbback in dom0 or usb back-end in qemu. On SUSE Linux Enterprise Server, the USB back-end comes with qemu.
As of Xen 4.7, xl
PVUSB support and hotplug support
is introduced.
In the configuration file, specify USB controllers and USB host devices
with usbctrl
and usbdev
. For
example, in case of HVM guests:
usbctrl=['type=qusb,version=2,ports=4', 'type=qusb,version=1,ports=4', ] usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ]
It is important to specify type=qusb
for the
controller of HVM guests.
To manage hotplugging PVUSB devices, use the
usbctrl-attach
, usbctrl-detach
,
usb-list
, usbdev-attach
and
usb-detach
subcommands. For example:
Create a USB controller which is version USB 1.1 and has 8 ports:
#
xl usbctrl-attach test_vm version=1 ports=8 type=qusb
Find the first available controller:port in the domain, and attach USB
device whose busnum:devnum is 2:3 to it; you can also specify
controller
and port
:
#
xl usbdev-attach test_vm hostbus=2 hostaddr=3
Show all USB controllers and USB devices in the domain:
#
xl usb-list test_vm
Devid Type BE state usb-ver ports
0 qusb 0 1 1 8
Port 1: Bus 002 Device 003
Port 2:
Port 3:
Port 4:
Port 5:
Port 6:
Port 7:
Port 8:
Detach the USB device under controller 0 port 1:
#
xl usbdev-detach test_vm 0 1
Remove the USB controller with the indicated dev_id
,
and all USB devices under it:
#
xl usbctrl-detach test_vm dev_id
For more information, see https://wiki.xenproject.org/wiki/Xen_USB_Passthrough.