18 Setting Up a Virtual Machine Host #
This section documents how to set up and use SUSE Linux Enterprise Server 12 SP5 as a virtual machine host.
Usually, the hardware requirements for the Dom0 are the same as those for the SUSE Linux Enterprise Server operating system. Additional CPU, disk, memory, and network resources should be added to accommodate the resource demands of all planned VM Guest systems.
Remember that VM Guest systems, like physical machines, perform better when they run on faster processors and have access to more system memory.
The virtual machine host requires several software packages and their dependencies to be installed. To install all necessary packages, run YaST
, select › and choose for installation. The installation can also be performed with YaST using the module › .After the Xen software is installed, restart the computer and, on the boot screen, choose the newly added option with the Xen kernel.
Updates are available through your update channel. To be sure to have the latest updates installed, run YaST
after the installation has finished.18.1 Best Practices and Suggestions #
When installing and configuring the SUSE Linux Enterprise Server operating system on the host, be aware of the following best practices and suggestions:
If the host should always run as Xen host, run YaST
› and activate the Xen boot entry as default boot section.In YaST, click
.Change the default boot to the
label, then click .Click
.
For best performance, only the applications and processes required for virtualization should be installed on the virtual machine host.
When using both iSCSI and OCFS2 to host Xen images, the latency required for OCFS2 default timeouts in SUSE Linux Enterprise Server may not be met. To reconfigure this timeout, run
systemctl configure o2cb
or editO2CB_HEARTBEAT_THRESHOLD
in the system configuration.If you intend to use a watchdog device attached to the Xen host, use only one at a time. It is recommended to use a driver with actual hardware integration over a generic software one.
The Dom0 kernel is running virtualized, so tools like
irqbalance
or lscpu
do not
reflect the real hardware characteristics.
Trusted boot (Tboot) is not supported by Xen. To ensure that the Xen host boots correctly, verify that the
option is deactivated in the GRUB 2 configuration dialog.18.2 Managing Dom0 memory #
When the host is set up, a percentage of system memory is reserved for the hypervisor, and all remaining memory is automatically allocated to Dom0.
A better solution is to set a default amount of memory for Dom0, so the memory can be allocated appropriately to the hypervisor. An adequate amount would be 20 percent of the total system memory up to 4 GiB. A recommended minimum amount would be 512 MiB
The minimum amount of memory heavily depends on how many VM Guest(s) the host should handle. So be sure you have enough memory to support all your VM Guests. If the value is too low, the host system may hang when multiple VM Guests use most of the memory.
18.2.1 Setting a Maximum Amount of Memory #
Determine the amount of memory to set for Dom0.
At Dom0, type
xl info
to view the amount of memory that is available on the machine. The memory that is currently allocated by Dom0 can be determined with the commandxl list
.Run
› .Select the Xen section.
In
, adddom0_mem=MEM_AMOUNT
where MEM_AMOUNT is the maximum amount of memory to allocate to Dom0. AddK
,M
, orG
, to specify the size, for example,dom0_mem=768M
.Restart the computer to apply the changes.
When using the XL tool stack and the dom0_mem=
option
for the Xen hypervisor in GRUB 2 you need to disable xl
autoballoon in etc/xen/xl.conf
.
Otherwise launching VMs will fail with errors about not being able to
balloon down Dom0. So add autoballoon=0 to
xl.conf
if you have the dom0_mem=
option specified for Xen. Also see
Xen
dom0 memory
18.3 Network Card in Fully Virtualized Guests #
In a fully virtualized guest, the default network card is an emulated Realtek network card. However, it also possible to use the split network driver to run the communication between Dom0 and a VM Guest. By default, both interfaces are presented to the VM Guest, because the drivers of some operating systems require both to be present.
When using SUSE Linux Enterprise Server, only the paravirtualized network cards are available for the VM Guest by default. The following network options are available:
- emulated
To use an emulated network interface like an emulated Realtek card, specify
type=ioemu
in thevif
device section of the domain xl configuration. An example configuration would look like:vif = [ 'type=ioemu,mac=00:16:3e:5f:48:e4,bridge=br0' ]
Find more details about the xl configuration in the
xl.conf
man pageman 5 xl.conf
.- paravirtualized
When you specify
type=vif
and do not specify a model or type, the paravirtualized network interface is used:vif = [ 'type=vif,mac=00:16:3e:5f:48:e4,bridge=br0,backen=0' ]
- emulated and paravirtualized
If the administrator should be offered both options, simply specify both type and model. The xl configuration would look like:
vif = [ 'type=ioemu,mac=00:16:3e:5f:48:e4,model=rtl8139,bridge=br0' ]
In this case, one of the network interfaces should be disabled on the VM Guest.
18.4 Starting the Virtual Machine Host #
If virtualization software is correctly installed, the computer boots to display the GRUB 2 boot loader with a
option on the menu. Select this option to start the virtual machine host.
When booting a Xen system, you might observe error messages in the /var/log/messages log file or systemd
journal of dom0 similar to following:
isst_if_mbox_pci: probe of 0000:ff:1e.1 failed with error -5 isst_if_pci: probe of 0000:fe:00.1 failed with error -5
Ignore them as they are harmless and are caused by the fact that the ISST driver does not provide any power or frequency scaling feature for virtual machines.
In Xen, the hypervisor manages the memory resource. If you need to
reserve system memory for a recovery kernel in Dom0, this memory needs
to be reserved by the hypervisor. Thus, it is necessary to add
crashkernel=size
to the
GRUB_CMDLINE_XEN_DEFAULT
variable in the
/etc/dfault/grub
file, save it and run the
following command:
tux >
sudo
grub2-mkconfig -o /boot/grub2/grub.cfg
For more information on the crashkernel parameter, see
Section 17.4, “Calculating crashkernel
Allocation Size”.
If the
option is not on the GRUB 2 menu, review the steps for installation and verify that the GRUB 2 boot loader has been updated. If the installation has been done without selecting the Xen pattern, run the YaST , select the filter and choose for installation.After booting the hypervisor, the Dom0 virtual machine starts and displays its graphical desktop environment. If you did not install a graphical desktop, the command line environment appears.
Sometimes it may happen that the graphics system does not work
properly. In this case, add vga=ask
to the boot
parameters. To activate permanent settings, use
vga=mode-0x???
where ???
is
calculated as 0x100
+ VESA mode from
https://en.wikipedia.org/wiki/VESA_BIOS_Extensions,
for example, vga=mode-0x361
.
Before starting to install virtual guests, make sure that the system time is correct. To do this, configure NTP (Network Time Protocol) on the controlling domain:
In YaST select
› .Select the option to automatically start the NTP daemon during boot. Provide the IP address of an existing NTP time server, then click
.
Hardware clocks commonly are not very precise. All modern operating systems
try to correct the system time compared to the hardware time by means of an
additional time source. To get the correct time on all VM Guest systems,
also activate the network time services on each respective guest or make
sure that the guest uses the system time of the host. For more about
Independent Wallclocks
in SUSE Linux Enterprise Server see
Section 16.2, “Xen Virtual Machine Clock Settings”.
For more information about managing virtual machines, see Chapter 20, Managing a Virtualization Environment.
18.5 PCI Pass-Through #
To take full advantage of VM Guest systems, it is sometimes necessary to assign specific PCI devices to a dedicated domain. When using fully virtualized guests, this functionality is only available if the chipset of the system supports this feature, and if it is activated from the BIOS.
This feature is available from both AMD* and Intel*. For AMD machines, the feature is called IOMMU; in Intel speak, this is VT-d. Note that Intel-VT technology is not sufficient to use this feature for fully virtualized guests. To make sure that your computer supports this feature, ask your supplier specifically to deliver a system that supports PCI Pass-Through.
Certain graphics drivers use highly optimized ways to access DMA. This is not supported, and thus using graphics cards may be difficult.
When accessing PCI devices behind a PCIe bridge, all the PCI devices must be assigned to a single guest. This limitation does not apply to PCIe devices.
Guests with dedicated PCI devices cannot be migrated live to a different host.
The configuration of PCI Pass-Through is twofold. First, the hypervisor must be informed at boot time that a PCI device should be available for reassigning. Second, the PCI device must be assigned to the VM Guest.
18.5.1 Configuring the Hypervisor for PCI Pass-Through #
Select a device to reassign to a VM Guest. To do this, run
lspci
and read the device number. For example, iflspci
contains the following line:06:01.0 Ethernet controller: Digital Equipment Corporation DECchip 21142/43 (rev 41)
In this case, the PCI number is
(06:01.0)
.Specify a module dependency to ensure that
xen_pciback
is the first module to control the device. Add the file named/etc/modprobe.d/50-e1000e.conf
with the following content:install e1000e /sbin/modprobe xen_pciback ; /sbin/modprobe \ --first-time --ignore-install e1000e
Instruct the
xen_pciback
module to control the device using the 'hide' option. Edit or create/etc/modprobe.d/50-xen-pciback.conf
with the following content:options xen_pciback hide=(06:01.0)
Reboot the system.
Check if the device is in the list of assignable devices with the command
xl pci-assignable-list
18.5.1.1 Dynamic Assignment with xl #
To avoid restarting the host system, you can use dynamic assignment with xl to use PCI Pass-Through.
Begin by making sure that dom0 has the pciback module loaded:
modprobe pciback
Then make a device assignable by using xl
pci-assignable-add
. For example, to make the device
06:01.0 available for guests, run the command:
xl pci-assignable-add 06:01.0
18.5.2 Assigning PCI Devices to VM Guest Systems #
There are several possibilities to dedicate a PCI device to a VM Guest:
- Adding the device while installing:
During installation, add the
pci
line to the configuration file:pci=['06:01.0']
- Hotplugging PCI devices to VM Guest systems
The command
xl
can be used to add or remove PCI devices on the fly. To add the device with number06:01.0
to a guest with namesles12
use:xl pci-attach sles12 06:01.0
- Adding the PCI device to Xend
To add the device to the guest permanently, add the following snippet to the guest configuration file:
pci = [ '06:01.0,power_mgmt=1,permissive=1' ]
After assigning the PCI device to the VM Guest, the guest system must care for the configuration and device drivers for this device.
18.5.3 VGA Pass-Through #
Xen 4.0 and newer supports VGA graphics adapter pass-through on fully virtualized VM Guests. The guest can take full control of the graphics adapter with high-performance full 3D and video acceleration.
VGA Pass-Through functionality is similar to PCI Pass-Through and as such also requires IOMMU (or Intel VT-d) support from the mainboard chipset and BIOS.
Only the primary graphics adapter (the one that is used when you power on the computer) can be used with VGA Pass-Through.
VGA Pass-Through is supported only for fully virtualized guests. Paravirtual guests (PV) are not supported.
The graphics card cannot be shared between multiple VM Guests using VGA Pass-Through—you can dedicate it to one guest only.
To enable VGA Pass-Through, add the following settings to your fully virtualized guest configuration file:
gfx_passthru=1 pci=['yy:zz.n']
where yy:zz.n
is the PCI controller ID of the VGA
graphics adapter as found with lspci -v
on Dom0.
18.5.4 Troubleshooting #
In some circumstances, problems may occur during the installation of the VM Guest. This section describes some known problems and their solutions.
- During boot, the system hangs
The software I/O translation buffer allocates a large chunk of low memory early in the bootstrap process. If the requests for memory exceed the size of the buffer it usually results in a hung boot process. To check if this is the case, switch to console 10 and check the output there for a message similar to
kernel: PCI-DMA: Out of SW-IOMMU space for 32768 bytes at device 000:01:02.0
In this case, you need to increase the size of the
swiotlb
. Addswiotlb=VALUE
(where VALUE is specified as the number of slab entries) on the command line of Dom0. The number can be adjusted up or down to find the optimal size for the machine.
The swiotlb=force
kernel parameter is required for
DMA access to work for PCI devices on a PV guest. For more
information about IOMMU and the swiotlb
option see
the file boot-options.txt
from the package
kernel-source
.
18.5.5 For More Information #
There are several resources on the Internet that provide interesting information about PCI Pass-Through:
18.6 USB Pass-Through #
There are two methods for passing through individual host USB devices to a guest. The first is via an emulated USB device controller, the second is using PVUSB.
18.6.1 Identify the USB Device #
Before you can pass through a USB device to the VM Guest, you need to
identify it on the VM Host Server. Use the lsusb
command to
list the USB devices on the host system:
root #
lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 003: ID 0461:4d15 Primax Electronics, Ltd Dell Optical Mouse
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
To pass through the Dell mouse, for example, specify either the device tag
in the form vendor_id:device_id
(0461:4d15) or the bus
address in the form bus.device
(2.3). Remember to remove
leading zeros, otherwise xl
would interpret the numbers
as octal values.
18.6.2 Emulated USB Device #
In emulated USB, the device model (QEMU) presents an emulated USB controller to the guest. The USB device is then controlled from Dom0 while USB commands are translated between the VM Guest and the host USB device. This method is only available to fully virtualized domains (HVM).
Enable the emulated USB hub with the usb=1
option. Then
specify devices among the list of devices in the config file along with
other emulated devices by using host:USBID
. For example:
usb=1 usbdevice=['tablet','host:2.3','host:0424:460']
18.6.3 Paravirtualized PVUSB #
PVUSB is a new high performance method for USB Pass-Through from dom0 to the virtualized guests. PVUSB supports both USB 2.0 and USB 1.1 devices. PVUSB uses paravirtualized front-end and back-end interfaces. With PVUSB, there are two ways to add USB devices to a guest:
via the configuration file at domain creation time
via hotplug while the VM is running
PVUSB uses a paravirtualized front- and back-end interface. PVUSB supports USB 1.1 and USB 2.0, and it works for both PV and HVM guests. To use PVUSB, you need usbfront in your guest OS, and usbback in dom0 or usb back-end in qemu. On SUSE Linux Enterprise Server, the USB back-end comes with qemu.
As of Xen 4.7, xl
PVUSB support and hotplug support is
introduced.
In the configuration file, specify USB controllers and USB host devices
with usbctrl
and usbdev
. For example,
in case of HVM guests:
usbctrl=['type=qusb,version=2,ports=4', 'type=qusb,version=1,ports=4', ] usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ]
It is important to specify type=qusb
for the controller
of HVM guests.
To manage hotplugging PVUSB devices, use the
usbctrl-attach
, usbctrl-detach
,
usb-list
, usbdev-attach
and
usb-detach
subcommands. For example:
Create a USB controller which is version USB 1.1 and has 8 ports:
root #
xl usbctrl-attach test_vm version=1 ports=8 type=qusb
Find the first available controller:port in the domain, and attach USB
device whose busnum:devnum is 2:3 to it; you can also specify
controller
and port
:
root #
xl usbdev-attach test_vm hostbus=2 hostaddr=3
Show all USB controllers and USB devices in the domain:
root #
xl usb-list test_vm
Devid Type BE state usb-ver ports
0 qusb 0 1 1 8
Port 1: Bus 002 Device 003
Port 2:
Port 3:
Port 4:
Port 5:
Port 6:
Port 7:
Port 8:
Detach the USB device under controller 0 port 1:
root #
xl usbdev-detach test_vm 0 1
Remove the USB controller with the indicated dev_id
, and
all USB devices under it:
root #
xl usbctrl-detach test_vm dev_id
For more information, see https://wiki.xenproject.org/wiki/Xen_USB_Passthrough.