23 Setting up a virtual machine host #

When installing and configuring the SUSE Linux Enterprise Server operating system on the host, be aware of the following best practices and suggestions:

Note: Hardware monitoring

The Dom0 kernel is running virtualized, so tools like irqbalance or lscpu do not reflect the real hardware characteristics.

Important: Trusted boot not supported by Xen

Trusted boot (Tboot) is not supported by Xen. To ensure that the Xen host boots correctly, verify that the Enable Trusted Boot Support option is deactivated in the GRUB 2 configuration dialog.

In previous versions of SUSE Linux Enterprise Server, the default memory allocation scheme of a Xen host was to allocate all host physical memory to Dom0 and enable auto-ballooning. Memory was automatically ballooned from Dom0 when additional domains were started. This behavior has always been error prone and disabling it was strongly encouraged. Starting with SUSE Linux Enterprise Server 15 SP1, auto-ballooning has been disabled by default and Dom0 is given 10% of host physical memory + 1 GB. For example, on a host with 32 GB of physical memory, 4.2 GB of memory is allocated for Dom0.

The use of the dom0_mem Xen command line option in /etc/default/grub is still supported and encouraged. You can restore the old behavior by setting dom0_mem to the host physical memory size and enabling the autoballoon setting in /etc/xen/xl.conf.

Warning: Insufficient memory for Dom0

The amount of memory reserved for Dom0 is a function of the number of VM Guests running on the host since Dom0 provides back-end network and disk I/O services for each VM Guest. Other workloads running in Dom0 should also be considered when calculating Dom0 memory allocation. Memory sizing of Dom0 should be determined like any other virtual machine.

23.2.1 Setting Dom0 memory allocation #

 

1. Determine memory allocation required for Dom0.

2. At Dom0, type xl info to view the amount of memory that is available on the machine. Dom0's current memory allocation can be determined with the xl list command.

3. Edit /etc/default/grub and adjust the GRUB_CMDLINE_XEN option so that it includes dom0_mem=MEM_AMOUNT. Replace MEM_AMOUNT with the maximum amount of memory to allocate to Dom0. Add K, M, or G, to specify the size unit. For example:

   GRUB_CMDLINE_XEN="dom0_mem=2G"

4. Restart the computer to apply the changes.

Tip

Refer to Section 18.2.2, “The file /etc/default/grub” for more details about Xen-related boot configuration options.

Warning: Xen Dom0 memory

When using the XL tool stack and the dom0_mem= option for the Xen hypervisor in GRUB 2 you need to disable xl autoballoon in etc/xen/xl.conf. Otherwise launching VMs will fail with errors about not being able to balloon down Dom0. So add autoballoon=0 to xl.conf if you have the dom0_mem= option specified for Xen. Also see Xen dom0 memory

In a fully virtualized guest, the default network card is an emulated Realtek network card. However, it also possible to use the split network driver to run the communication between Dom0 and a VM Guest. By default, both interfaces are presented to the VM Guest, because the drivers of some operating systems require both to be present.

When using SUSE Linux Enterprise Server, only the paravirtualized network cards are available for the VM Guest by default. The following network options are available:

If virtualization software is correctly installed, the computer boots to display the GRUB 2 boot loader with a Xen option on the menu. Select this option to start the virtual machine host.

Warning

When booting a Xen system, you might observe error messages in the /var/log/messages log file or systemd journal of dom0 similar to following:

isst_if_mbox_pci: probe of 0000:ff:1e.1 failed with error -5
isst_if_pci: probe of 0000:fe:00.1 failed with error -5

Ignore them as they are harmless and are caused by the fact that the ISST driver does not provide any power or frequency scaling feature for virtual machines.

Note: Xen and Kdump

In Xen, the hypervisor manages the memory resource. If you need to reserve system memory for a recovery kernel in Dom0, this memory needs to be reserved by the hypervisor. Thus, it is necessary to add crashkernel=size to the GRUB_CMDLINE_XEN_DEFAULT variable in the /etc/dfault/grub file, save it and run the following command:

> sudo grub2-mkconfig -o /boot/grub2/grub.cfg

For more information on the crashkernel parameter, see Section 19.4, “Calculating crashkernel allocation size”.

If the Xen option is not on the GRUB 2 menu, review the steps for installation and verify that the GRUB 2 boot loader has been updated. If the installation has been done without selecting the Xen pattern, run the YaST Software Management, select the filter Patterns and choose Xen Virtual Machine Host Server for installation.

After booting the hypervisor, the Dom0 virtual machine starts and displays its graphical desktop environment. If you did not install a graphical desktop, the command line environment appears.

Tip: Graphics problems

Sometimes it may happen that the graphics system does not work properly. In this case, add vga=ask to the boot parameters. To activate permanent settings, use vga=mode-0x??? where ??? is calculated as 0x100 + VESA mode from https://en.wikipedia.org/wiki/VESA_BIOS_Extensions, for example, vga=mode-0x361.

Before starting to install virtual guests, make sure that the system time is correct. To do this, configure NTP (Network Time Protocol) on the controlling domain:

Note: Time services on virtual guests

Hardware clocks commonly are not very precise. All modern operating systems try to correct the system time compared to the hardware time by means of an additional time source. To get the correct time on all VM Guest systems, also activate the network time services on each respective guest or make sure that the guest uses the system time of the host. For more about Independent Wallclocks in SUSE Linux Enterprise Server see Section 18.2, “Xen virtual machine clock settings”.

For more information about managing virtual machines, see Chapter 25, Managing a virtualization environment.

To take full advantage of VM Guest systems, it is sometimes necessary to assign specific PCI devices to a dedicated domain. When using fully virtualized guests, this functionality is only available if the chipset of the system supports this feature, and if it is activated from the BIOS.

This feature is available from both AMD* and Intel*. For AMD machines, the feature is called IOMMU; in Intel speak, this is VT-d. Note that Intel-VT technology is not sufficient to use this feature for fully virtualized guests. To make sure that your computer supports this feature, ask your supplier specifically to deliver a system that supports PCI Pass-Through.

The configuration of PCI Pass-Through is twofold. First, the hypervisor must be informed at boot time that a PCI device should be available for reassigning. Second, the PCI device must be assigned to the VM Guest.

> sudo modprobe pciback

> sudo xl pci-assignable-add 06:01.0

gfx_passthru=1
pci=['yy:zz.n']

23.5.4 Troubleshooting #

 

In some circumstances, problems may occur during the installation of the VM Guest. This section describes some known problems and their solutions.

During boot, the system hangs

The software I/O translation buffer allocates a large chunk of low memory early in the bootstrap process. If the requests for memory exceed the size of the buffer it usually results in a hung boot process. To check if this is the case, switch to console 10 and check the output there for a message similar to

kernel: PCI-DMA: Out of SW-IOMMU space for 32768 bytes at device 000:01:02.0

In this case, you need to increase the size of the swiotlb. Add swiotlb=VALUE (where VALUE is specified as the number of slab entries) on the command line of Dom0. The number can be adjusted up or down to find the optimal size for the machine.

Note: swiotlb a PV guest

The swiotlb=force kernel parameter is required for DMA access to work for PCI devices on a PV guest. For more information about IOMMU and the swiotlb option see the file boot-options.txt from the package kernel-source.

There are two methods for passing through individual host USB devices to a guest. The first is via an emulated USB device controller, the second is using PVUSB.

# lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 003: ID 0461:4d15 Primax Electronics, Ltd Dell Optical Mouse
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

usb=1
usbdevice=['tablet','host:2.3','host:0424:460']

23.6.3 Paravirtualized PVUSB #

 

PVUSB is a new high performance method for USB Pass-Through from dom0 to the virtualized guests. With PVUSB, there are two ways to add USB devices to a guest:

• via the configuration file at domain creation time

• via hotplug while the VM is running

PVUSB uses paravirtualized front- and back-end interfaces. PVUSB supports USB 1.1 and USB 2.0, and it works for both PV and HVM guests. To use PVUSB, you need usbfront in your guest OS, and usbback in dom0 or usb back-end in qemu. On SUSE Linux Enterprise Server, the USB back-end comes with qemu.

As of Xen 4.7, xl PVUSB support and hotplug support is introduced.

In the configuration file, specify USB controllers and USB host devices with usbctrl and usbdev. For example, in case of HVM guests:

usbctrl=['type=qusb,version=2,ports=4', 'type=qusb,version=1,ports=4', ]
usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ]

Note

It is important to specify type=qusb for the controller of HVM guests.

To manage hotpluggin PVUSB devices, use the usbctrl-attach, usbctrl-detach, usb-list, usbdev-attach and usb-detach subcommands. For example:

Create a USB controller which is version USB 1.1 and has 8 ports:

# xl usbctrl-attach test_vm version=1 ports=8 type=qusb

Find the first available controller:port in the domain, and attach USB device whose busnum:devnum is 2:3 to it; you can also specify controller and port:

# xl usbdev-attach test_vm hostbus=2 hostaddr=3

Show all USB controllers and USB devices in the domain:

# xl usb-list test_vm
Devid  Type   BE  state usb-ver ports
0      qusb   0   1     1       8
  Port 1: Bus 002 Device 003
  Port 2:
  Port 3:
  Port 4:
  Port 5:
  Port 6:
  Port 7:
  Port 8:

Detach the USB device under controller 0 port 1:

# xl usbdev-detach test_vm 0 1

Remove the USB controller with the indicated dev_id, and all USB devices under it:

# xl usbctrl-detach test_vm dev_id

For more information, see https://wiki.xenproject.org/wiki/Xen_USB_Passthrough.