Applies to SUSE Linux Enterprise Server 15 SP6

18 Xen to KVM migration guide #

As the KVM virtualization solution is becoming more and more popular among server administrators, many of them need a path to migrate their existing Xen based environments to KVM. As of now, there are no mature tools to automatically convert Xen VMs to KVM. There is, however, a technical solution that helps convert Xen virtual machines to KVM. The following information and procedures help you to perform such a migration.

Important: Migration procedure not supported

The migration procedure described in this document is not fully supported by SUSE. We provide it as a guidance only.

18.1 Migration to KVM using `virt-v2v` #

This section contains information to help you import virtual machines from foreign hypervisors (such as Xen) to KVM managed by libvirt.

Tip: Microsoft Windows guests

This section is focused on converting Linux guests. Converting Microsoft Windows guests using virt-v2v is the same as converting Linux guests, except with regard to handling the Virtual Machine Driver Pack (VMDP). Additional details on converting Windows guests with the VMDP can be found separately at Virtual Machine Driver Pack documentation.

18.1.1 Introduction to `virt-v2v` #

virt-v2v is a command-line tool to convert VM Guests from a foreign hypervisor to run on KVM managed by libvirt. It enables paravirtualized virtio drivers in the converted virtual machine if possible. A list of supported operating systems and hypervisors follows:

Supported guest operating systems #

SUSE Linux Enterprise Server
openSUSE
Red Hat Enterprise Linux
Fedora
Microsoft Windows Server 2003 and 2008

Supported source hypervisor #

Supported target hypervisor #

KVM (managed by libvirt)

18.1.2 Installing `virt-v2v` #

The installation of virt-v2v is simple:

> sudo zypper install virt-v2v

Remember that virt-v2v requires root privileges, so you need to run it either as root, or via sudo.

18.1.3 Converting virtual machines to run under KVM managed by `libvirt` #

virt-v2v converts virtual machines from the Xen hypervisor to run under KVM managed by libvirt. To learn more about libvirt and virsh, see Part II, “Managing virtual machines with libvirt”. Additionally, all virt-v2v command line options are explained in the virt-v2v man page (man 1 virt-v2v).

Before converting a virtual machine, make sure to complete the following steps:

Procedure 18.1: Preparing the environment for the conversion #

Create a new local storage pool.
virt-v2v copies the storage of the source virtual machine to a local storage pool managed by libvirt (the original disk image remains unchanged). You can create the pool either with Virtual Machine Manager or virsh. For more information, see Section 9.2.2, “Managing storage with Virtual Machine Manager” and Section 9.2.1, “Managing storage with virsh”.
Prepare the local network interface.
Check that the converted virtual machine can use a local network interface on the VM Host Server. It is normally a network bridge and if it is not yet defined, create it with YaST › System › Network Settings › Add › Bridge.
Note: Mappings of network devices
Network devices on the source Xen host can be mapped during the conversion process to corresponding network devices on the KVM target host. For example, the Xen bridge br0 can be mapped to the default KVM network device. Sample mappings can be found in /etc/virt-v2v.conf. To enable these mappings, modify the XML rule and ensure the section is not commented out with  markers. For example:
```
 <network type='bridge' name='br0'>
   <network type='network' name='default'/>
 </network>
```
Tip: No network bridge
If there is no network bridge available, Virtual Machine Manager can optionally create it.

virt-v2v has the following basic command syntax:

virt-v2v -i INPUT_METHOD -os STORAGE_POOL SOURCE_VM

input_method: There are two input methods: libvirt or libvirtxml. See the SOURCE_VM parameter for more information.
storage_pool: The storage pool you already prepared for the target virtual machine.
source_vm: The source virtual machine to convert. It depends on the INPUT_METHOD parameter: for libvirt, specify the name of a libvirt domain. For libvirtxml, specify the path to an XML file containing a libvirt domain specification.

Note: Conversion time

Conversion of a virtual machine takes a lot of system resources, mainly for copying the whole disk image for a virtual machine. Converting a single virtual machine typically takes up to 10 minutes.Virtual machines using large disk images can take much longer.

18.1.3.1 Conversion based on the `libvirt` XML description file #

This section describes how to convert a local Xen virtual machine using the libvirt XML configuration file. This method is suitable if the host is already running the KVM hypervisor. Make sure that the libvirt XML file of the source virtual machine, and the libvirt storage pool referenced from it are available on the local host.

Obtain the libvirt XML description of the source virtual machine.
Tip: Obtaining the XML files
To obtain the libvirt XML files of the source virtual machine, you must run the host OS under the Xen kernel. If you already rebooted the host to the KVM-enabled environment, reboot back to the Xen kernel, dump the libvirt XML file, and then reboot back to the KVM environment.
First identify the source virtual machine under virsh:
```
# virsh list
 Id    Name                           State
----------------------------------------------------
[...]
  2     sles12_xen                     running
[...]
```
sles12_xen is the source virtual machine to convert. Now export its XML and save it to sles12_xen.xml:
```
# virsh dumpxml sles12_xen > sles12_xen.xml
```
Verify that all disk image paths are correct from the KVM host's perspective. This is not a problem when converting on one machine, but may require manual changes when converting using an XML dump from another host.
```
<source file='/var/lib/libvirt/images/XenPool/SLES.qcow2'/>
```
Tip: Copying images
To avoid copying an image twice, manually copy the disk image or images directly to the libvirt storage pool. Update the source file entries in the XML description file. The virt-v2v process detects the existing disks and converts them in place.

Run virt-v2v to convert to KVM virtual machine:

# virt-v2v sles12_xen.xml1 \
-i LIBVIRTXML2 \
-os remote_host.example.com:/exported_dir3 \
--bridge br04 \
-on sles12_kvm5

1	The XML description of the source Xen-based virtual machine.
2	`virt-v2v` reads the information about the source virtual machine from a `libvirt` XML file.
3	Storage pool where the target virtual machine disk image is placed. In this example, the image is placed on an NFS share `/exported_dir` on the `remote_host.example.com` server.
4	The target KVM-based virtual machine uses the network bridge `br0` on the host.
5	The target virtual machine is renamed to `sles12_kvm` to prevent name collision with the existing virtual machine of the same name.

18.1.3.2 Conversion based on the `libvirt` domain name #

This method is useful if you are still running libvirt under Xen, and plan to reboot to the KVM hypervisor later.

Find the libvirt domain name of the virtual machine you want to convert.

# virsh list
 Id    Name                           State
----------------------------------------------------
[...]
  2     sles12_xen                     running
[...]

sles12_xen is the source virtual machine to convert.

Run virt-v2v to convert to KVM virtual machine:

# virt-v2v sles12_xen1 \
-i libvirt2 \
-os storage_pool3 \
--network eth04 \
-of qcow25 \
-oa sparse6 \
-on sles12_kvm

1	The domain name of the Xen-based virtual machine.
2	`virt-v2v` reads the information about the source virtual machine directly from the active `libvirt` connection.
3	The target disk image is placed in a local `libvirt` storage pool.
4	All guest bridges (or networks) are connected to a locally managed network.
5	Format for the disk image of the target virtual machine. Supported options are `raw` or `qcow2`.
6	Whether the converted guest disk space is `sparse` or `preallocated`.

18.1.3.3 Converting a remote Xen virtual machine #

This method is useful if you need to convert a Xen virtual machine running on a remote host. As virt-v2v connects to the remote host via ssh, ensure the SSH service is running on the host.

Note: Passwordless SSH access

virt-v2v requires a passwordless SSH connection to the remote host. This means a connection using an SSH key added to the ssh-agent. See man ssh-keygen and man ssh-add for more details on this. More information is also available at Chapter 22, Securing network operations with OpenSSH.

To connect to a remote libvirt connection, construct a valid connection URI relevant for your remote host. In the following example, the remote host name is remote_host.example.com, and the user name for the connection is root. The connection URI then looks as follows:

xen+ssh://root@remote_host.example.com/

For more information on libvirt connection URIs, see https://libvirt.org/uri.html.

Find the libvirt domain name of the remote virtual machine you want to convert.

# virsh -c xen+ssh://root@remote_host.example.com/ list
 Id    Name                           State
----------------------------------------------------
  1     sles12_xen                     running
[...]

sles12_xen is the source virtual machine to convert.

The virt-v2v command for the remote connection looks like this:

# virt-v2v sles12_xen \
-i libvirt \
-ic xen+ssh://root@remote_host.example.com/ \
-os local_storage_pool \
--bridge br0

18.1.4 Running converted virtual machines #

After virt-v2v completes successfully, a new libvirt domain is created with the name specified with the -on option. If you did not specify -on, the same name as the source virtual machine is used. The new guest can be managed with standard libvirt tools, such as virsh or Virtual Machine Manager.

Tip: Rebooting the machine

If you completed the conversion under Xen as described in Section 18.1.3.2, “Conversion based on the libvirt domain name”, you may need to reboot the host machine and boot with the non-Xen kernel.

18.2 Xen to KVM manual migration #

18.2.1 General outline #

The preferred solution to manage virtual machines is based on libvirt; for more information, see https://libvirt.org/. It has several advantages over the manual way of defining and running virtual machines—libvirt is cross-platform, supports many hypervisors, has secure remote management, has virtual networking, and, most of all, provides a unified abstract layer to manage virtual machines. Therefore the main focus of this article is on the libvirt solution.

Generally, the Xen to KVM migration consists of the following basic steps:

Make a backup copy of the original Xen VM Guest.
Optionally, apply changes specific to paravirtualized guests.
Obtain information about the original Xen VM Guest and update it to KVM equivalents.
Shut down the guest on the Xen host, and run the new one under the KVM hypervisor.

Warning: No live migration

The Xen to KVM migration cannot be done live while the source VM Guest is running. Before running the new KVM-ready VM Guest, you are advised to shut down the original Xen VM Guest.

18.2.2 Back up the Xen VM Guest #

To back up your Xen VM Guest, follow these steps:

Identify the relevant Xen guest you want to migrate, and remember its ID/name.

> sudo virsh list --all
Id Name                 State
----------------------------------
 0 Domain-0             running
 1 SLES15SP3            running
[...]

Shut down the guest. You can do this either by shutting down the guest OS, or with virsh:
```
> sudo virsh shutdown SLES11SP3
```

Back up its configuration to an XML file.

> sudo virsh dumpxml SLES11SP3 > sles11sp3.xml

Back up its disk image file. Use the cp or rsync commands to create the backup copy. Remember that it is always a good idea to check the copy with the md5sum command.
After the image file is backed up, you can start the guest again with
```
> sudo virsh start SLES11SP3
```

18.2.3 Changes specific to paravirtualized guests #

Apply the following changes if you are migrating a paravirtualized Xen guest. You can do it either on the running guest, or on the stopped guest using guestfs-tools.

Important

After applying the changes described in this section, the image file related to the migrated VM Guest is not usable under Xen anymore.

18.2.3.1 Install the default kernel #

Warning: No booting

After installing the default kernel, the system fails to boot the Xen guest.

Before cloning the Xen guest disk image for use under the KVM hypervisor, make sure it is bootable without the Xen hypervisor. This is crucial for paravirtualized Xen guests as they normally contain a special Xen kernel, and often do not have a complete GRUB 2 boot loader installed.

For SLES 11, update the /etc/sysconfig/kernel file. Change the INITRD_MODULES parameter by removing all Xen drivers and replacing them with virtio drivers. Replace
```
INITRD_MODULES="xenblk xennet"
```
with
```
INITRD_MODULES="virtio_blk virtio_pci virtio_net virtio_balloon"
```
For SLES 12, 15 and openSUSE, search for xenblk xennet in /etc/dracut.conf.d/*.conf and replace them with virtio_blk virtio_pci virtio_net virtio_balloon
Paravirtualized Xen guests run a specific Xen kernel. To run the guest under KVM, you need to install the default kernel.
Note: Default kernel is already installed
You do not need to install the default kernel for a fully virtualized guest, as it is already installed.
Enter rpm -q kernel-default on the Xen guest to find out whether the default kernel is installed. If not, install it with zypper in kernel-default.
The kernel we are going to use to boot the guest under KVM must have virtio (paravirtualized) drivers available. Run the following command to find out. Do not forget to replace 6.4.0-150600.9 with your kernel version:
```
> sudo sudo find /lib/modules/6.4.0-150600.9-default/kernel/drivers/ -name virtio*
/lib/modules/6.4.0-150600.9-default/kernel/drivers/block/virtio_blk.ko.zst
/lib/modules/6.4.0-150600.9-default/kernel/drivers/bluetooth/virtio_bt.ko.zst
/lib/modules/6.4.0-150600.9-default/kernel/drivers/char/hw_random/virtio-rng.ko.zst
/lib/modules/6.4.0-150600.9-default/kernel/drivers/crypto/virtio
/lib/modules/6.4.0-150600.9/kernel/drivers/block/virtio_blk.ko
...
```
Update /etc/fstab. Change any storage devices from xvda to vda.
Update the boot loader configuration. Enter rpm -q grub2 on the Xen guest to find out whether GRUB 2 is already installed. If not, install it with zypper in grub2.
Now make the newly installed default kernel the default for booting the OS. Also remove/update the kernel command line options that may refer to Xen-specific devices. You can do it either with YaST (System › Boot Loader), or manually:
- Find the preferred Linux boot menu entry by listing them all:
```
> cat /boot/grub2/grub.cfg | grep 'menuentry '
```
  Remember the order number (counted from zero) of the one you newly installed.
- Set it as the default boot menu entry:
```
> sudo grub2-set-default N
```
  Replace N with the number of the boot menu entry you previously discovered.
- Open /etc/default/grubfor editing, and look for the GRUB_CMDLINE_LINUX_DEFAULT and GRUB_CMDLINE_LINUX_RECOVERY options. Remove or update any reference to Xen-specific devices. In the following example, you can replace
```
root=/dev/xvda1 disk=/dev/xvda console=xvc
```
  with
```
root=/dev/vda1 disk=/dev/vda
```
  Do not forget to remove all references to xvc-type consoles (such as xvc0).
Update device.map in either the /boot/grub2 or /boot/grub2-efi directory, whichever that VM uses. Change any storage devices from xvda to vda.
To import new default settings, run
```
grub2-mkconfig -o /boot/grub2/grub.cfg
```

18.2.3.2 Update the guest for boot under KVM #

Update the system to use the default serial console. List the configured consoles, and remove symbolic links to xvc? ones.

> sudo ls -l /etc/systemd/system/getty.target.wants/
getty@tty1.service -> /usr/lib/systemd/system/getty@.service
getty@xvc0.service -> /usr/lib/systemd/system/getty@xvc0.service
getty@xvc1.service -> /usr/lib/systemd/system/getty@xvc1.service

# rm /etc/systemd/system/getty.target.wants/getty@xvc?.service

Update the /etc/securetty file. Replace xvc0 with ttyS0.

18.2.4 Update the Xen VM Guest configuration #

This section describes how to export the configuration of the original Xen VM Guest, and what particular changes to apply to it so it can be imported as a KVM guest into libvirt.

18.2.4.1 Export the Xen VM Guest configuration #

First export the configuration of the guest and save it to a file. For example:

> sudo virsh dumpxml SLES11SP3
<domain type='xen'>
  <name>SLES11SP3</name>
  <uuid>fa9ea4d7-8f95-30c0-bce9-9e58ffcabeb2</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <bootloader>/usr/bin/pygrub</bootloader>
  <os>
    <type>linux</type>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/lib/xen/bin/qemu-dm</emulator>
    <disk type='file' device='disk'>
      <driver name='file'/>
      <source file='/var/lib/libvirt/images/SLES_11_SP2_JeOS.x86_64-0.0.2_para.raw'/>
      <target dev='xvda' bus='xen'/>
    </disk>
    <interface type='bridge'>
      <mac address='00:16:3e:2d:91:c3'/>
      <source bridge='br0'/>
      <script path='vif-bridge'/>
    </interface>
    <console type='pty'>
      <target type='xen' port='0'/>
    </console>
    <input type='mouse' bus='xen'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
  </devices>
</domain>

You can find detailed information on the libvirt XML format for VM Guest description at https://libvirt.org/formatdomain.html.

18.2.4.2 General changes to the guest configuration #

You need to make a few general changes to the exported Xen guest XML configuration to run it under the KVM hypervisor. The following applies to both fully virtualized and paravirtualized guests. The following XML elements are just an example and do not need to be in your specific configuration.

Tip: Conventions used

To refer to a node in the XML configuration file, an XPath syntax is used throughout this document. For example, to refer to a <name> inside the <domain> tag

<domain>
  <name>sles11sp3</name>
</domain>

an XPath equivalent /domain/name is used.

Change the type attribute of the /domain element from xento kvm.
Remove the /domain/bootloader element section.
Remove the /domain/bootloader_args element section.
Change the /domain/os/type element value from linux to hvm.
Add <boot dev="hd"/> under the /domain/os element.
Add the arch attribute to the /domain/os/type element. Acceptable values are arch=”x86_64” or arch=”i686”
Change the /domain/devices/emulator element from /usr/lib/xen/bin/qemu-dm' to /usr/bin/qemu-kvm.
For each disk associated with the paravirtualized (PV) guest, change the following:
- Change the name attribute of the /domain/devices/disk/driver element from file to qemu, and add a type attribute for the disk type. For example, valid options include raw and qcow2.
- Change the dev attribute of the /domain/devices/disk/target element from xvda to vda.
- Change the bus attribute of the /domain/devices/disk/target element from xen to virtio.
For each network interface card, make the following changes:
- If there is a model defined in /domain/devices/interface, change its type attribute value to virtio
```
<model type=”virtio”>
```
- Delete all /domain/devices/interface/script sections.
- Delete all /domain/devices/interface/target elements if the dev attribute starts with vif or vnet or veth. If using a custom network then change the dev value to that target.
Remove the /domain/devices/console element section if it exists.
Remove the /domain/devices/serial element section if it exists.
Change the bus attribute on the /domain/devices/input element from xen to ps2.
Add the following element for memory ballooning features under the /domain/devices element.
```
<memballoon model="virtio"/>
```

Tip: Device name

<target dev='hda' bus='ide'/> controls the device under which the disk is exposed to the guest OS. The dev attribute indicates the “logical” device name. The actual device name specified is not guaranteed to map to the device name in the guest OS. Therefore you may need to change the disk mapping on the boot loader command line. For example, if the boot loader expects a root disk to be hda2 but KVM still sees it as sda2, change the boot loader command line from

[...] root=/dev/hda2 resume=/dev/hda1 [...]

[...] root=/dev/sda2 resume=/dev/sda1 [...]

For paravirtualized xvda devices, change it to:

[...] root=/dev/vda2 resume=/dev/vda1 [...]

Otherwise the VM Guest refuses to boot in the KVM environment.

18.2.4.3 The target KVM guest configuration #

After having applied all the modifications mentioned above, you end up with the following configuration for your KVM guest:

<domain type='kvm'>
  <name>SLES11SP3</name>
  <uuid>fa9ea4d7-8f95-30c0-bce9-9e58ffcabeb2</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu cpuset='0-3'>1</vcpu>
  <os>
    <type arch=”x86_64”>hvm</type>
    <boot dev="hd"/>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type="raw"/>
      <source file='/var/lib/libvirt/images/SLES_11_SP2_JeOS.x86_64-0.0.2_para.raw'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='bridge'>
      <mac address='00:16:3e:2d:91:c3'/>
      <source bridge='br0'/>
    </interface>
    <input type='mouse' bus='usb'/>
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'/>
    <memballoon model="virtio"/>
  </devices>
</domain>

Save the configuration to a file in your home directory, as SLES11SP3.xml, for example. It gets copied to the default /etc/libvirt/qemu directory after the import.

18.2.5 Migrate the VM Guest #

After you updated the VM Guest configuration, and applied necessary changes to the guest OS, shut down the original Xen guest, and run its clone under the KVM hypervisor.

Shut down the guest on the Xen host by running shutdown -h now as root from the console.
Copy the disk images associated with the VM Guest if needed. A default configuration requires the Xen disk files to be copied from /var/lib/xen/images to /var/lib/kvm/images. The /var/lib/kvm/images directory may need to be created (as root) if you have not previously created a VM Guest.

Create the new domain, and register it with libvirt:

> sudo virsh define SLES11SP3.xml
 Domain SLES11SP3 defined from SLES11SP3.xml

Verify that the new guest is seen in the KVM configuration:
```
> virsh list –all
```

After the domain is created, you can start it:

> sudo virsh start SLES11SP3
 Domain SLES11SP3 started

18.3 More information #

For more information on libvirt, see https://libvirt.org.

You can find more details on the libvirt XML format at https://libvirt.org/formatdomain.html.

18 Xen to KVM migration guide #

18.1 Migration to KVM using virt-v2v #

18.1.1 Introduction to virt-v2v #

18.1.2 Installing virt-v2v #

18.1.3 Converting virtual machines to run under KVM managed by libvirt #

18.1.3.1 Conversion based on the libvirt XML description file #

18.1.3.2 Conversion based on the libvirt domain name #

18.1.3.3 Converting a remote Xen virtual machine #

18.1.4 Running converted virtual machines #

18.2 Xen to KVM manual migration #

18.2.1 General outline #

18.2.2 Back up the Xen VM Guest #

18.2.3 Changes specific to paravirtualized guests #

18.2.3.1 Install the default kernel #

18.2.3.2 Update the guest for boot under KVM #

18.2.4 Update the Xen VM Guest configuration #

18.2.4.1 Export the Xen VM Guest configuration #

18.2.4.2 General changes to the guest configuration #

18.2.4.3 The target KVM guest configuration #

18.2.5 Migrate the VM Guest #

18.3 More information #

18.1 Migration to KVM using `virt-v2v` #

18.1.1 Introduction to `virt-v2v` #

18.1.2 Installing `virt-v2v` #

18.1.3 Converting virtual machines to run under KVM managed by `libvirt` #

18.1.3.1 Conversion based on the `libvirt` XML description file #

18.1.3.2 Conversion based on the `libvirt` domain name #