Jump to content
documentation.suse.com / AMD Secure Encrypted Virtualization (AMD-SEV) Guide
SUSE Linux Enterprise Server 15 SP5

AMD Secure Encrypted Virtualization (AMD-SEV) Guide

Publication Date: June 27, 2024

AMD's Secure Encrypted Virtualization (SEV) allows the memory of virtual machines to be encrypted. SEV with Encrypted State (SEV-ES) goes one step further by encrypting the virtual machine's CPU register content. These technologies increase system security and are ideal for multi-tenant environments such as cloud computing. They enable protection from a variety of cross-VM and hypervisor-based attacks. As an example, a hostile VM that has escaped its hypervisor-enforced confines and is able to read arbitrary memory is unable to steal sensitive data from an SEV or SEV-ES VM.

This document aims to provide a basic understanding of how SEV and SEV-ES work, and how to enable and configure these features. It also mentions certain limitations and restrictions that the use of SEV and SEV-ES causes as compared to non-encrypted virtualization.

1 Introducing SEV

Encryption of computer data stored on disk is a widely deployed feature. However, data in RAM is stored in the clear. This can leave that data vulnerable to software or hardware probing by intruders on the host system, particularly in cloud computing environments where the physical resources are shared by many tenants. Consider a virtual machine of a hostile tenant escaping its sandbox because of a hypervisor bug and searching memory for sensitive data.

AMD's SEV (Secure Encrypted Virtualization) is a technology to protect Linux KVM virtual machines by transparently encrypting the memory of each VM with a unique key. SEV can also calculate a signature of the memory contents, which can be sent to the VM's owner as an attestation that the memory was encrypted correctly by the firmware. SEV is especially relevant to cloud computing environments, where VMs are hosted on remote servers which are not under the control of the VMs' owners. SEV can reduce the amount of trust VMs need to place in the hypervisor and administrator of their host system.

When a virtual machine is processing sensitive data, it can be present in CPU registers as well as memory. If the processing is halted, for example, to service an interrupt or share time with other processes, the virtual machine's CPU register contents are saved to hypervisor memory. This memory is readable by the hypervisor even if SEV is enabled. SEV-ES protects against this scenario by encrypting all CPU register contents when the processing of a virtual machine is halted. SEV-ES builds upon SEV to provide an even smaller attack surface for virtual machines running in a multi-tenant environment.

2 VM host requirements

The VM host hardware must support AMD's SEV technology. To detect if the host hardware supports SEV, check that the sev attribute is in the capabilities of libvirt and that its value is set appropriately:

<domainCapabilities>
   ...
   <features>
   ...
   <sev supported='yes'/>
   ...
   </sev>
   </features>
</domainCapabilities>

Additionally, ensure that the kvm_amd kernel module has the sev parameter enabled:

/sys/module/kvm_amd/parameters/sev = 1

3 VM requirements

The VM must be the modern Q35 machine type and must use UEFI firmware. libvirt can automatically select an appropriate SEV or SEV-ES enabled UEFI firmware, or one can be specified manually. Currently, the only firmware supported are /usr/share/qemu/ovmf-x86_64-code.bin and /usr/share/qemu/ovmf-x86_64-4m-code.bin. See Book “Virtualization Guide”, Chapter 10 “Guest installation”, Section 10.3.1 “Advanced UEFI Configuration” for more details on using UEFI firmware and the auto-selection feature.

Note
Note: No IDE support in Q35

The Q35 machine type does not have an IDE controller and does not support IDE disks.

All virtio-net devices need to be configured with the iPXE option ROM disabled. iPXE is currently not compatible with SEV and SEV-ES. All memory regions used by the VM must be locked for Direct Memory Access (DMA) and to prevent swapping. This includes memory for the VM and any memory regions allocated by QEMU to support running the VM, such as UEFI pflash for firmware and variable store, video RAM, etc.

4 VM configuration

Example 1: Sample configuration file

As an example, an SEV-encrypted VM configured with 4 GB of memory would contain the following XML configuration:

<domain type='kvm'>
    <memory unit='KiB'>4194304</memory>
    <currentMemory unit='KiB'>4194304</currentMemory>
    <memoryBacking>
    <locked/> 1
    </memoryBacking>
    <os>
    <type arch='x86_64' machine='pc-q35-2.11'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x86_64-ms-4m-code.bin</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/sles15-sev-guest_VARS.fd</nvram>
    <boot dev='hd'/>
    </os>
    <launchSecurity 2 type='sev'>
    <cbitpos>47</cbitpos> 3
    <reducedPhysBits>1</reducedPhysBits> 4
    <policy>0x0033</policy> 5
    <dhCert>AAAABBBB=CCCCCDDDDD</dhCert> 6
    <session>AAAABBBB=EEEEEFFFFF</session> 7
    </launchSecurity>
    <devices>
    <interface type='bridge'>
    <source bridge='br0'/>
    <model type='virtio'/>
    <rom enabled='no'/> 8
    </interface>
    ...
    </devices>
    ...
    </domain>

1

The memoryBacking element, along with its child element locking, is used to ease memory limit restrictions libvirt places on the VM's cgroup. Otherwise, VM creation would fail when QEMU attempts to lock the VM's memory regions along with other regions used to support the VM operation. See https://libvirt.org/kbase/launch_security_sev.html#memory for more information on VM memory configuration requirements for SEV VMs.

2

The launchSecurity type='sev' element and its contents enable encryption of the VM's memory contents.

3

When memory encryption is enabled, one of the physical address bits (also known as the C-bit) is used to mark if a memory page is protected. The required cbitpos element provides the location of the C-bit in a guest page table entry. For example, the value 47 indicates that bit position 47 in a page table entry determines whether that page is encrypted or not. The C-bit number is read from the host's CPUID and is thus hardware-dependent. The value of cbitpos is hypervisor-dependent, and can be obtained through the sev element in the capabilities of the domain.

4

When memory encryption is enabled, we lose certain bits of the physical address space. The required reducedPhysBits element provides this physical address bit reduction. Similarly to cbitpos, the value of reducedPhysBits is processor-family-dependent and can be obtained through the sev element in the domain capabilities.

5

The required policy element provides the guest policy which must be maintained by the SEV firmware. This policy is enforced by the firmware, and restricts what configuration and operational commands can be performed on the VM by the hypervisor. The guest policy provided when starting the VM is bound to that VM and cannot be changed throughout its lifetime.

6

The optional dhCert element provides the guest owner's base64-encoded Diffie-Hellman (DH) key. The key is used to negotiate a master secret key between the SEV firmware and guest owner. This master secret key is then used to establish a trusted channel between the SEV firmware and guest owner.

7

The optional session element provides the guest owner's base64-encoded session blob, as defined in the SEV API specification. See the LAUNCH_START section of the SEV specification for the session-blob format.

8

Besides the launchSecurity settings, SEV-encrypted VMs must have the iPXE option ROM disabled on all virtio-net devices. Currently, iPXE is not compatible with SEV-encrypted VMs.

The guest policy is four unsigned bytes with the following definition:

Table 1: Guest policy definitions

Bits

Definition

0

If set, debugging of the guest is disallowed

1

If set, sharing keys with other guests is disallowed

2

If set, SEV-ES is required

3

If set, sending the guest to another platform is disallowed

4

If set, the guest must not be transmitted to another platform that is not in the domain

5

If set, the guest must not be transmitted to another platform that is not SEV-capable

6-15

Reserved

16-32

The guest must not be transmitted to another platform with a lower firmware version

5 VM installation

virt-install supports the installation of SEV and SEV-ES virtual machines. In addition to your standard installation parameters, provide virt-install with options to satisfy the VM requirements and the --launchSecurity option.

The following example starts a network installation of a SLES 15 SP4 virtual machine protected with SEV-ES.

   virt-install --name sles15sp4-sev-es --location http://192.168.0.1/install/sles15sp4/x86_64 --disk size=20 --network=bridge=br0,model=virtio,rom.bar=off 1 --vcpus 4 --memory 4096 --noautoconsole --events on_reboot=destroy --machine q35 --memtune hard_limit=4563402 --launchSecurity sev,policy=0x07 2 --boot firmware=efi 3 --vnc --serial pty

1

The iPXE option ROM is not compatible with SEV-encrypted VMs and must be disabled on all virtio-net devices. While libvirt supports disabling option ROMs using either the enabled or bar attributes of the rom element, virt-install only supports disabling option ROMs using the bar attribute.

2

The launchSecurity option specifies the type and policy to be enforced by the SEV firmware. The policy setting is described in Table 1, “Guest policy definitions”.

3

The boot option allows specifying many boot-related settings, including the firmware used by the virtual machine. Specifying a firmware type efi allows libvirt's firmware auto-selection feature to select an appropriate SEV capable firmware for the virtual machine.

6 VM attestation

VM attestation is the process of verifying that trusted software components are correctly instantiated on a trusted compute platform. The process typically involves starting a VM in a paused state, retrieving a launch measurement of the instantiated software components, verifying the measurement, then providing a disk password or other key to unlock the VM and resume the paused boot process. The launch measurement includes cryptographic artifacts provided by the VM owner, with the cryptographic root of trust being the AMD SEV platform. The VM owner can be confident their software components have not been compromised and are running on a trusted platform once the launch measurement has been verified.

The overall attestation process is moderately complex with plenty of opportunity for error. Care must be taken to ensure the process itself is secure. For example, a secure attestation process cannot be executed directly on the hypervisor running the VM, since the goal is to demonstrate the hypervisor has not acted maliciously and contaminated the VM.

Although all the information and APIs required for attestation exist in SLES 15 SP4, SLES 15 SP5 introduces a simple utility called virt-qemu-sev-validate that can be used to satisfy several attestation use cases. For example, automated tests in quality assurance and small libvirt+KVM deployments that are not managed by large, commercial management stacks.

virt-qemu-sev-vailidate is included in the libvirt-client-qemu package and supports both offline and online attestation modes. virt-qemu-sev-validate requires all input for attestation as command-line parameters. Assuming it is invoked on a trusted machine, the results of virt-qemu-sev-validate can be trusted since no information is retrieved from untrusted sources. Online mode is less secure, particularly when executed directly on the hypervisor running the VM.

Regardless of mode, the attestation process of a libvirt+KVM VM starts with creating a VM or Guest Owner (GO) certificate and session blob that is unique for each start of the VM. The certificate and blob can be created with the sevctl utility, available in the sevctl package. The following example illustrates the use of the sevctl session command to create all the prelaunch SEV-related artifacts. The NAME parameter is optional and allows a prefix to be specified for the artifact file names. Using the VM name as a prefix is convenient for matching artifacts with VMs later. The path to the platform Diffie-Hellman (DH) certificate and the desired SEV policy are required parameters.

# sevctl session --name test-sev /data/sev/pdh.cert 7

The sevctl session command produces four files: tik.bin, tek.bin, godh.b64 and session.b64. If the optional NAME parameter was used, the files are prefixed with the specified value. The transport integrity key (tik.bin) and transport encryption key (tek.bin) are used in the verification stage of the attestation process. The guest owner Diffie-Hellman key (godh.b64) and session blob (session.b64) are copied to the VM XML configuration before starting the VM. See the dhCert and session subelements of the launchSecurity element in the VM configuration section for more details.

After the VM session artifacts have been created and VM XML configuration updated, the VM can be started in a paused state, for example:

# virsh -c qemu+ssh://USER_NAME@HOST_NAME/system create --paused /path/to/vm.xml

Creating the VM in a paused state allows retrieving the launch measurement from the hypervisor and comparing it to a measurement calculated on a trusted host using trusted inputs. If the measurements compare, the VM owner can be confident the VM has been properly instantiated on the hypervisor and execution can safely be started. The following command demonstrates using the virsh domlaunchsecinfo command to retrieve the paused VM launch measurement and other SEV-related information from the hosting hypervisor.

# virsh -c qemu+ssh://username@hostname/system domlaunchsecinfo sevtest
sev-measurement: VZjxMSlu+UuYkWHN2mAxDVVYXRmL3wqTu84kwk+5QS+4OMii7hs6cMAmXNpmmyS/
sev-api-major  : 1
sev-api-minor  : 51
sev-build-id   : 3
sev-policy     : 7

The retrieved launch measurement can then be given to virt-qemu-sev-validate to verify the VM has been securely instantiated. The following example demonstrates a full offline attestation of the measurement.

# virt-qemu-sev-validate --api-major 1 --api-minor 51 --build-id 3 --policy 7 \
 --firmware /usr/share/qemu/ovmf-x86_64-4m.bin --tik sevtest_tik.bin --tek sevtest_tek.bin --num-cpus 4 \
 --cpu-family 25 --cpu-model 1 --cpu-stepping 1 \
 --measurement QJ0oDpFmWj+bGZzFoMPbAxTuC6QD44W5w88x/hQM8toVsB75ci7V1YDfYoI9GTk

It is also possible to use virt-qemu-sev-validate in an online mode, where information needed to perform the VM attestation is retrieved from the hosting hypervisor. The following example demonstrates an online attestation of the VM, where only the hosting hypervisor URI, VM name, and associated TIK and TEK are specified. virt-qemu-sev-validate retrieves the remaining information, including the measurement itself, from the hosting hypervisor:

# virt-qemu-sev-validate --tik sevtest_tik.bin --tek sevtest_tek.bin \
 --connect qemu+ssh://USER_NAME@HOST_NAME/system --domain sevtest

Once the VM launch measurement has been verified, the VM owner can optionally inject a secret in the VM and resume VM execution. An example of injecting a secret would be providing a key to unlock an encrypted root disk.

# virsh -c qemu+ssh://USER_NAME@HOST_NAME/system domsetlaunchsecstate sevtest \
 --secrethdr hdr-str --secret secret-str
# virsh -c qemu+ssh://USER_NAME@HOST_NAME/system resume sevtest

7 SEV with KubeVirt

KubeVirt supports running SEV guests starting from the version 0.49.0. The functionality can be activated by enabling the WorkloadEncryptionSEV feature gate:

> kubectl edit kubevirt kubevirt -n kubevirt
[...]
spec:
  configuration:
    developerConfiguration:
      featureGates:
      - WorkloadEncryptionSEV
[...]

To run an SEV-encrypted guest, the virtual machine specification must include the entry sev: {} under the launchSecurity domain element. Additionally, you need to configure the firmware/bootloader parameters to use the efi option with the secureBoot flag set to disabled. The corresponding YAML snippet looks similar to the following:

[...]
spec:
  domain:
    firmware:
      bootloader:
        efi:
          secureBoot: false
    launchSecurity:
      sev: {}
[...]

8 Current limitations

SUSE does not recommend using the SEV and SEV-ES features with SUSE Linux products on the first generation AMD EPYC™ 7000 series of processors, code name Naples. It is recommended to use at least the second generation 7002 series processors, code name Rome. Additionally, the following limitations are placed on SEV and SEV-ES VMs.

  • The guest operating system running inside an SEV-encrypted VM must contain SEV support. SUSE Linux Enterprise Server 12 SP4 and newer, and all SUSE Linux Enterprise Server 15 releases support SEV.

  • Any operations that involve saving and restoring the memory and state of an instance are currently not supported. This means that SEV-encrypted VMs cannot be resumed from snapshots, saved/restored, or live migrated. Encrypted VMs can be shutdown and restarted on another host as normal.

  • SEV-encrypted VMs cannot contain directly accessible host devices (that is, PCI passthrough).

  • SEV-encrypted VMs are not compatible with Secure Boot. UEFI firmware containing Secure Boot support does not work with SEV or SEV-ES VMs.

  • SEV-ES VMs cannot be rebooted from within using reboot, shutdown -r now, etc. A reboot must be done by shutting down the VM and starting it again. This limitation does not apply to SEV VMs, only SEV-ES.

These limitations will be removed in the future as the hardware, firmware and specific layers of software receive new features.

9 More information

10 Legal notice

Copyright© 2006– 2024 SUSE LLC and contributors. All rights reserved.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled GNU Free Documentation License.

For SUSE trademarks, see https://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.

All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors, nor the translators shall be held liable for possible errors or the consequences thereof.