Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / Assigning Host Devices to Virtual Machines

Assigning Host Devices to Virtual Machines

Publication Date: 18 Apr 2024
WHAT?

Virtual machines can directly access host PCI or USB devices, such as graphics cards or network adapters.

WHY?

A virtual machine that is assigned direct access to a host device can use all its capabilities, bypassing the virtualization layer. Such an approach offers near-native performance or hardware acceleration.

GOAL

After completing the included procedures, you will be able to assign a host PCI or USB device to a virtual machine using both the graphical application Virtual Machine Manager and the command-line tool virsh.

EFFORT

It takes less than 30 minutes to understand and manage assigning a host device to a virtual machine.

REQUIREMENTS

  • A libvirt-based VM Host Server running at least one virtual machine

  • libvirt-client and virt-manager packages installed on the host

  • root privileges

1 libvirt and device assignment

 

libvirt is a unified interface to multiple virtualization solutions. libvirt virtual machines are typically managed by either virsh, a command-line utility, or Virtual Machine Manager, an application with a graphical user interface. Assigning VM Host Server's PCI or USB devices to a VM Guest dedicates the device's resources to the VM Guest. The reasons for assigning a host device to a VM are near-native performance of the device, using its hardware acceleration capabilities, or isolating the device from other VMs to increase security.

This article describes how to assign a VM Host Server's device to a VM Guest using both the virsh command-line utility and the Virtual Machine Manager GUI application.

2 Assigning a device using Virtual Machine Manager

 

The following procedure describes how to assign a host device to a VM Guest using the Virtual Machine Manager GUI application.

  1. If SELinux is running on the VM Host Server, you need to allow managing the PCI or USB device configuration: 

    > sudo setsebool -P virt_use_sysfs 1

  2. Double-click a VM Guest entry in the Virtual Machine Manager to open its console, then switch to the Details view with View › Details.

  3. Click Add Hardware and choose the PCI Host Device or USB Host Device from the left list. A list of available PCI devices appears in the right part of the window.

    Adding a PCI device
    Figure 1: Adding a PCI device
     

  4. From the list of available devices, choose the one you want to pass to the guest. Confirm with Finish.

  5. Restart the domain if it is running for the changes to take effect.

3 Assigning a device using virsh

 

The following procedure describes how to assign a host device to a VM Guest using the virsh command-line tool.

Important
Important: KVM guests using QEMU Q35 machine type

KVM guests using the QEMU Q35 machine type have a PCI topology that includes a pcie-root controller and seven pcie-root-port controllers. The pcie-root controller does not support hotplugging. Each pcie-root-port controller supports hotplugging a single PCIe device. PCI controllers cannot be hotplugged, so plan accordingly and add more pcie-root-ports if more than seven PCIe devices will be hotplugged. A pcie-to-pci-bridge controller can be added to support hotplugging legacy PCI devices. See https://libvirt.org/pci-hotplug.html for more information about PCI topology between QEMU machine types.

  1. Identify the host PCI or USB device to assign to the VM Guest. The following example identifies a DEC network card: 

    > sudo lspci -nn
 [...]
 03:07.0 Ethernet controller [0200]: Digital Equipment Corporation DECchip \
 21140 [FasterNet] [1011:0009] (rev 22)
 [...]

    Write down the device ID, for example, 03:07.0.

  2. Gather detailed information about the device using virsh nodedev-dumpxml ID. To get the ID, replace the colon and the period in the device ID (03:07.0) with underscores. Prefix the result with “pci_0000_”: pci_0000_03_07_0. 

    > sudo virsh nodedev-dumpxml pci_0000_03_07_0
 <device>
   <name>pci_0000_03_07_0</name>
   <path>/sys/devices/pci0000:00/0000:00:14.4/0000:03:07.0</path>
   <parent>pci_0000_00_14_4</parent>
   <driver>
     <name>tulip</name>
   </driver>
   <capability type='pci'>
     <domain>0</domain>
     <bus>3</bus>
     <slot>7</slot>
     <function>0</function>
     <product id='0x0009'>DECchip 21140 [FasterNet]</product>
     <vendor id='0x1011'>Digital Equipment Corporation</vendor>
     <numa node='0'/>
   </capability>
 </device>

    Write down the values for domain, bus and function.

  3. Detach the device from the host system: 

    > sudo virsh nodedev-detach pci_0000_03_07_0
   Device pci_0000_03_07_0 detached

  4. Convert the domain, bus, slot, and function values from decimal to hexadecimal. In this example, domain = 0, bus = 3, slot = 7, and function = 0. Ensure that the values are inserted in the right order. Use printf to output the PCI source address line that you need when updating the domain XML specification, for example: 

    > printf "<address domain='0x%x' bus='0x%x' slot='0x%x' function='0x%x'/>\n" 0 3 7 0
<address domain='0x0' bus='0x3' slot='0x7' function='0x0'/>

  5. Run virsh edit on your domain, and add the following device entry in the <devices> section using the result from the previous step: 

    <hostdev mode='subsystem' type='pci' managed='yes'>
   <source>
     <address domain='0x0' bus='0x03' slot='0x07' function='0x0'/>
   </source>
 </hostdev>

  6. Shut down the VM Guest: 

    > sudo shutdown -h now

  7. If SELinux is running on the VM Host Server, you need to allow managing the host device configuration: 

    > sudo setsebool -P virt_use_sysfs 1

  8. Start your VM Guest to make the assigned device available: 

    > sudo virsh start VM_NAME

4 Managed and unmanaged devices

 

When assigning a host device to a VM Guest, you can specify whether the device is managed or unmanaged by using the managed='yes' (default) or managed='no' option in the libvirt XML specification of the device, for example: 

<hostdev mode='subsystem' type='pci' managed='yes'>

Managed devices are automatically detached from the host driver and attached to the virtual vfio-pci driver of the guest during the domain startup. When the domain is terminated, the device is reattached to the original host driver.

For unmanaged devices, you need to take care of detaching the device from the host and attaching it to the vfio-pci driver manually, as suggested by the following steps:

  1. Before starting the domain, detach the device—for example, identified by pci_0000_03_07_0—from the host. 

    > sudo virsh nodedev-detach pci_0000_03_07_0

  2. Start the domain. It is automatically assigned the unmanaged device identified by pci_0000_03_07_0.

  3. After you stop the domain, you can make the device available to the host again. 

    > sudo virsh nodedev-reattach pci_0000_03_07_0

5 Troubleshooting

 

This topic lists common problems when assigning host devices to virtual machines.

5.1 PCI Pass-Through to fully virtualized (FV) Xen guest fails

 

This issue is identified with certain Intel processor chipsets where the iommu boot parameter needs to be disabled by default for PCI Pass-Through to work with paravirtualized (PV) guests. For both PV and FV guests to work with PCI Pass-Through, use the no-intremap option on the Xen hypervisor instead. No change is needed if PCI devices are only passed through to PV guests. For example: 

> sudo cat /boot/grub/menu.lst
###Don't change this comment - YaST2 identifier: Original name: xen###
title Xen -- SUSE Linux Enterprise Desktop 15 SP5 - 6.1.0-0.6.6
root (hd0,5)
kernel /boot/xen.gz vga=mode-0x314 iommu=no-intremap
module /boot/vmlinuz-6.1.0-0.6.6-xen root=/dev/sda6 resume=/dev/sda5 splash=silent showopts vga=0x314
module /boot/initrd-6.1.0-0.6.6-xen

5.2 Virtual machine complains that the passthrough device is missing

 

This often happens if you assign a removable host device—such as a USB flash drive—to a virtual machine (VM) and then try to start the VM with the assigned device unplugged. This breaks the VM start and you either need to remove the device assignment, or plug the device back in the host.