Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
ContentsContents
Virtualization Guide
  1. Preface
  2. I Introduction
    1. 1 Virtualization technology
    2. 2 Virtualization scenarios
    3. 3 Introduction to Xen virtualization
    4. 4 Introduction to KVM virtualization
    5. 5 Virtualization tools
    6. 6 Installation of virtualization components
    7. 7 Virtualization limits and support
  3. II Managing virtual machines with libvirt
    1. 8 Starting and stopping libvirtd
    2. 9 Preparing the VM Host Server
    3. 10 Guest installation
    4. 11 Basic VM Guest management
    5. 12 Connecting and authorizing
    6. 13 Advanced storage topics
    7. 14 Configuring virtual machines with Virtual Machine Manager
    8. 15 Configuring virtual machines with virsh
    9. 16 Managing virtual machines with Vagrant
    10. 17 Xen to KVM migration guide
  4. III Hypervisor-independent features
    1. 18 Disk cache modes
    2. 19 VM Guest clock settings
    3. 20 libguestfs
    4. 21 QEMU guest agent
    5. 22 Software TPM emulator
  5. IV Managing virtual machines with Xen
    1. 23 Setting up a virtual machine host
    2. 24 Virtual networking
    3. 25 Managing a virtualization environment
    4. 26 Block devices in Xen
    5. 27 Virtualization: configuration options and settings
    6. 28 Administrative tasks
    7. 29 XenStore: configuration database shared between domains
    8. 30 Xen as a high-availability virtualization host
    9. 31 Xen: converting a paravirtual (PV) guest into a fully virtual (FV/HVM) guest
  6. V Managing virtual machines with QEMU
    1. 32 QEMU overview
    2. 33 Setting up a KVM VM Host Server
    3. 34 Guest installation
    4. 35 Running virtual machines with qemu-system-ARCH
    5. 36 Virtual machine administration using QEMU monitor
  7. Glossary
  8. A Virtual machine drivers
  9. B Configuring GPU Pass-Through for NVIDIA cards
  10. C XM, XL toolstacks, and the libvirt framework
  11. D GNU licenses
Navigation
Applies to SUSE Linux Enterprise Server 15 SP3

7 Virtualization limits and support

7.1 Architecture support

7.1.1 KVM hardware requirements

Currently, SUSE supports KVM full virtualization on AMD64/Intel 64, Arm AArch64, IBM Z, and LinuxONE hosts.

  • On the AMD64/Intel 64 architecture, KVM is designed around hardware virtualization features included in AMD* (AMD-V) and Intel* (VT-x) CPUs. It supports virtualization features of chipsets and PCI devices, such as an I/O Memory Mapping Unit (IOMMU) and Single Root I/O Virtualization (SR-IOV). You can test whether your CPU supports hardware virtualization with the following command:

    tux > egrep '(vmx|svm)' /proc/cpuinfo

    If this command returns no output, your processor either does not support hardware virtualization, or this feature has been disabled in the BIOS or firmware.

    The following Web sites identify AMD64/Intel 64 processors that support hardware virtualization: http://ark.intel.com/Products/VirtualizationTechnology (for Intel CPUs), and http://products.amd.com/ (for AMD CPUs).

  • On the Arm architecture, Armv8-A processors include support for virtualization.

Note
Note: KVM kernel modules not loading

The KVM kernel modules only load if the CPU hardware virtualization features are available.

The general minimum hardware requirements for the VM Host Server are the same as outlined in Abschnitt 2.1, „Hardware“. However, additional RAM for each virtualized guest is needed. It should at least be the same amount that is needed for a physical installation. It is also strongly recommended to have at least one processor core or hyper-thread for each running guest.

Note
Note: Arm AArch64

Arm AArch64 is a continuously evolving platform. It does not have a traditional standards and compliance certification program to enable interoperability with operating systems and hypervisors. Ask your vendor for the support statement on SUSE Linux Enterprise Server.

Note
Note: POWER

Running KVM or Xen hypervisors on the POWER platform is not supported.

7.1.2 Xen hardware requirements

SUSE supports Xen on AMD64/Intel 64.

7.2 Hypervisor limits

New features and virtualization limits for Xen and KVM are outlined in the Release Notes for each Service Pack (SP).

Only packages that are part of the official repositories for SUSE Linux Enterprise Server are supported. Conversely, all optional subpackages and plug-ins (for QEMU, libvirt) provided at packagehub are not supported.

For the maximum total virtual CPUs per host, see recommendations in the Virtualization Best Practices Guide regarding over-commitment of physical CPUs. The total number of virtual CPUs should be proportional to the number of available physical CPUs.

Note
Note: 32-bit hypervisor

With SUSE Linux Enterprise Server 11 SP2, we removed virtualization host facilities from 32-bit editions. 32-bit guests are not affected and are fully supported using the provided 64-bit hypervisor.

7.2.1 KVM limits

Supported (and tested) virtualization limits of a SUSE Linux Enterprise Server 15 SP3 host running Linux guests on AMD64/Intel 64. For other operating systems, refer to the specific vendor.

Table 7.1: KVM VM limits

Maximum virtual CPUs per VM

288

Maximum memory per VM

6 TiB

Note
Note

KVM host limits are identical to SUSE Linux Enterprise Server (see the corresponding section of release notes), except for:

7.2.2 Xen limits

Table 7.2: Xen VM limits

Maximum virtual CPUs per VM

128 (HVM), 64 (HVM Windows guest) or 512 (PV)

Maximum memory per VM

2 TiB (64-bit guest), 16 GiB (32-bit guest with PAE)

Table 7.3: Xen host limits

Maximum total physical CPUs

1024

Maximum total virtual CPUs per host

See recommendations in the Virtualization Best Practices Guide regarding over-commitment of physical CPUs in https://documentation.suse.com/sles-15/html/SLES-all/article-virtualization-best-practices.html#sec-vt-best-perf-cpu-assign. The total virtual CPUs should be proportional to the available physical CPUs

Maximum physical memory

16 TiB

7.3 Supported host environments (hypervisors)

This section describes the support status of SUSE Linux Enterprise Server 15 SP3 running as a guest operating system on top of various virtualization hosts (hypervisors).

Table 7.4: The following SUSE host environments are supported

SUSE Linux Enterprise Server

Hypervisors

SUSE Linux Enterprise Server 11 SP4

Xen and KVM

SUSE Linux Enterprise Server 12 SP1 to SP5

Xen and KVM

SUSE Linux Enterprise Server 15 GA to SP3

Xen and KVM

You can also search in the SUSE YES certification database

The level of support is as follows
  • Support for SUSE host operating systems is full L3 (both for the guest and host) in accordance with the respective product life cycle.

  • SUSE provides full L3 support for SUSE Linux Enterprise Server guests within third-party host environments.

  • Support for the host and cooperation with SUSE Linux Enterprise Server guests must be provided by the host system's vendor.

7.4 Supported guest operating systems

This section lists the support status for various guest operating systems virtualized on top of SUSE Linux Enterprise Server 15 SP3 for KVM and Xen hypervisors.

The following guest operating systems are fully supported (L3):
  • SUSE Linux Enterprise Server 11 SP4

  • SUSE Linux Enterprise Server 12 SP3, 12 SP4, 12 SP5

  • SUSE Linux Enterprise Server 15 GA, 15 SP1, 15 SP2, 15 SP3

  • Windows Server 2008 SP2+, 2008 R2 SP1+, 2012+, 2012 R2+, 2016, 2019, 2022

  • Oracle Linux 6, 7, 8 (KVM hypervisor only)

The following guest operating systems are supported as a technology preview (L2, fixes if reasonable):
  • SLED 15 SP3

Red Hat guest operating systems are fully supported (L3) if the customer has purchased Expanded Support:
  • Refer to the Expanded Support documentation for the list of available combinations and supported releases.

Note
Note: RHEL PV drivers

Starting from RHEL 7.2, Red Hat removed Xen PV drivers.

The following guest operating systems are supported on a best-effort basis (L2, fixes if reasonable):
  • Windows 8+, 8.1+, 10+

All other guest operating systems
  • In other combinations, L2 support is provided but fixes are available only if feasible. SUSE fully supports the host OS (hypervisor). The guest OS issues need to be supported by the respective OS vendor. If an issue fix involves both the host and guest environments, the customer needs to approach both SUSE and the guest VM OS vendor.

  • All guest operating systems are supported both fully virtualized and paravirtualized. The exception is Windows systems, which are only supported fully virtualized (but they can use PV drivers: https://www.suse.com/products/vmdriverpack/), and OES operating systems, which are supported only paravirtualized.

  • All guest operating systems are supported both in 32-bit and 64-bit environments, unless stated otherwise.

7.4.1 Availability of paravirtualized drivers

To improve the performance of the guest operating system, paravirtualized drivers are provided when available. Although they are not required, it is strongly recommended to use them. The paravirtualized drivers are available as follows:

SUSE Linux Enterprise Server 12 / 12 SP1 / 12 SP2

Included in kernel

SUSE Linux Enterprise Server 11 / 11 SP1 / 11 SP2 / 11 SP3 / 11 SP4

Included in kernel

SUSE Linux Enterprise Server 10 SP4

Included in kernel

Red Hat

Available since Red Hat Enterprise Linux 5.4. Starting from Red Hat Enterprise Linux 7.2, Red Hat removed the PV drivers.

Windows

SUSE has developed virtio-based drivers for Windows, which are available in the Virtual Machine Driver Pack (VMDP). For more information, see https://www.suse.com/products/vmdriverpack/.

Note
Note: PVops kernel

Starting from SUSE Linux Enterprise Server 12 SP2, we switched to a PVops kernel. We are no longer using a dedicated kernel-xen package. Dom0 kernel-default+kernel-xen was replaced with kernel-default, domU PV kernel-xen was replaced by kernel-default, domU HVM kernel-default+xen-kmp was replaced by kernel-default; dom0 toolstack still loads all back-end drivers.

7.5 Supported VM migration scenarios

SUSE Linux Enterprise Server supports migrating a virtual machine from one physical host to another.

7.5.1 Offline migration scenarios

SUSE supports offline migration, powering off a guest VM, then moving it to a host running a different SLE product, from SLE 12 to SLE 15 SPX. The following host operating system combinations are fully supported (L3) for migrating guests from one host to another:

Table 7.5: Offline supported migrating guests from SUSE Linux Enterprise Server 12 SP3

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 12 SP3

SLES 12 SP3

SLES 12 SP4

SLES 12 SP5

SLES 15 GA

Table 7.6: Offline supported migrating guests from SUSE Linux Enterprise Server 12 SP4

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 12 SP4

SLES 12 SP4

SLES 12 SP5

SLES 15 (KVM only)

SLES 15 SP1

Table 7.7: Offline supported migrating guests from SUSE Linux Enterprise Server 12 SP5

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 12 SP5

SLES 12 SP5

SLES 15 SP1

SLES 15 SP2

Table 7.8: Offline supported migrating guests from SUSE Linux Enterprise Server 15 GA

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 15 GA

SLES 15 GA

SLES 15 SP1

SLES 15 SP2

SLES 15 SP3

Table 7.9: Offline supported migrating guests from SUSE Linux Enterprise Server 15 SP1

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 15 SP1

SLES 15 SP1

SLES 15 SP2

SLES 15 SP3

Table 7.10: Offline supported migrating guests from SUSE Linux Enterprise Server 15 SP2

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 15 SP2

SLES 15 SP2

SLES 15 SP3

7.5.2 Live migration scenarios

This section lists support status of various live migration scenarios when running virtualized on top of SLES. Also, refer to the supported live migration requirements. The following host operating system combinations are fully supported (L3 in accordance with the respective product life cycle).

Table 7.11: Supported live-migrating guests from SLES 12 SP3

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 12 SP3

SLES 12 SP4

Table 7.12: supported live-migrating guests from SLES 12 SP4

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 12 SP4

SLES 12 SP4

SLES 12 SP5

SLES 15 GA (KVM only)

Table 7.13: Supported live-migrating guests from SLES 12 SP5

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 12 SP5

SLES 12 SP5

SLES 15 SP1

Table 7.14: Supported live-migrating guests from SLES 15

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 15 GA

SLES 15 GA

SLES 15 SP1

Table 7.15: Supported live-migrating guests from SLES 15 SP1

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 15 SP1

SLES 15 SP1

SLES 15 SP2

Table 7.16: Supported live-migrating guests from SLES 15 SP2

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 15 SP2

SLES 15 SP2

SLES 15 SP3

Table 7.17: Supported live-migrating guests from SLES 15 SP3

Host Source Product Name

To Host Target

SUSE Linux Enterprise Server 15 SP3

SLES 15 SP3

SLES 15 SP4 (when it is available)
Note
Note: Live migration
  • SUSE always supports live migration of virtual machines between hosts running SLES with successive service pack numbers. For example, from SLES 15 SP2 to 15 SP3.

  • SUSE strives to support live migration of virtual machines from a host running a service pack under LTSS to a host running a newer service pack, within the same major version of SUSE Linux Enterprise Server. For example, virtual machine migration from a SLES 12 SP2 host to a SLES 12 SP5 host. SUSE only performs minimal testing of LTSS-to-newer migration scenarios and recommends thorough on-site testing before attempting to migrate critical virtual machines.

Note
Note: Xen live migration

Live migration between SLE 11 and SLE 12 is not supported because of the different tool stack, see the Release notes for more details.

7.6 Feature support

7.6.1 Xen host (Dom0)

Table 7.18: Feature support—host (Dom0)

Features

Xen

Network and block device hotplugging

Yes

Physical CPU hotplugging

No

Virtual CPU hotplugging

Yes

Virtual CPU pinning

Yes

Virtual CPU capping

Yes

Intel* VT-x2: FlexPriority, FlexMigrate (migration constraints apply to dissimilar CPU architectures)

Yes

Intel* VT-d2 (DMA remapping with interrupt filtering and queued invalidation)

Yes

AMD* IOMMU (I/O page table with guest-to-host physical address translation)

Yes

Note
Note: Adding or removing physical CPUs at runtime is not supported

The addition or removal of physical CPUs at runtime is not supported. However, virtual CPUs can be added or removed for each VM Guest while offline.

7.6.2 Xen paravirtualized guest (DomU)

Table 7.19: Feature support—paravirtualized guest

Features

Xen

Virtual network and virtual block device hotplugging

Yes

Virtual CPU hotplugging

Yes

Virtual CPU over-commitment

Yes

Dynamic virtual memory resize

Yes

VM save and restore

Yes

VM live migration

Yes (See Section 11.7.1, “Migration requirements”)

Advanced debugging with GDBC

Yes

Dom0 metrics visible to VM

Yes

Memory ballooning

Yes

PCI Pass-Through

Yes (NetWare guests are excluded)

For live migration, both source and target system architectures need to match; that is, the processors (AMD* or Intel*) must be the same. Unless CPU ID masking is used, such as with Intel FlexMigration, the target should feature the same processor revision or a more recent processor revision than the source. If VMs are moved among different systems, the same rules apply for each move. To avoid failing optimized code at runtime or application start-up, source and target CPUs need to expose the same processor extensions. Xen exposes the physical CPU extensions to the VMs transparently. To summarize, guests can be 32-bit or 64-bit, but the VHS must be identical.

Note
Note: Intel flexMigration

For machines that support Intel FlexMigration, CPU-ID masking and faulting allow more flexibility in cross-CPU migration.

7.6.3 Fully virtualized guest

Table 7.20: Feature support—fully virtualized guest

Features

Xen

KVM

Virtual network and virtual block device hotplugging

Yes

Yes

Virtual CPU hotplugging

No

No

Virtual CPU over-commitment

Yes

Yes

Dynamic virtual memory resize

Yes

Yes

VM save and restore

Yes

Yes

VM Live Migration

Yes (See Section 11.7.1, “Migration requirements”)

Yes

VM snapshot

Yes

Yes

Advanced debugging with GDBC

Yes

Yes

Dom0 metrics visible to VM

Yes

Yes

PCI Pass-Through

Yes

Yes

Note
Note: Windows guest

Hotplugging of virtual network and virtual block devices, and resizing, shrinking, and restoring dynamic virtual memory are supported in Xen and KVM only if PV drivers are being used (VMDP).

For KVM, a detailed description of supported limits, features, recommended settings and scenarios, and other useful information is maintained in kvm-supported.txt. This file is part of the KVM package and can be found in /usr/share/doc/packages/kvm.

7.7 Nested virtualization

Nested virtualization allows you to run a virtual machine inside another VM while still using hardware acceleration from the host. It has very bad performance and adds more complexity while debugging. Nested virtualization is mostly used for testing purposes. In SUSE Linux Enterprise Server, nested virtualization is a Technology Preview. It is only provided for testing purposes and is not supported. Bugs can be reported but they will be treated with low priority. Any attempt to live-migrate or save/restore VMs in the presence of nested virtualization is also explicitly unsupported.

Print this page