Commit 683412cc authored by Mingwei Zhang's avatar Mingwei Zhang Committed by Paolo Bonzini
Browse files

KVM: SEV: add cache flush to solve SEV cache incoherency issues 



Flush the CPU caches when memory is reclaimed from an SEV guest (where
reclaim also includes it being unmapped from KVM's memslots).  Due to lack
of coherency for SEV encrypted memory, failure to flush results in silent
data corruption if userspace is malicious/broken and doesn't ensure SEV
guest memory is properly pinned and unpinned.

Cache coherency is not enforced across the VM boundary in SEV (AMD APM
vol.2 Section 15.34.7). Confidential cachelines, generated by confidential
VM guests have to be explicitly flushed on the host side. If a memory page
containing dirty confidential cachelines was released by VM and reallocated
to another user, the cachelines may corrupt the new user at a later time.

KVM takes a shortcut by assuming all confidential memory remain pinned
until the end of VM lifetime. Therefore, KVM does not flush cache at
mmu_notifier invalidation events. Because of this incorrect assumption and
the lack of cache flushing, malicous userspace can crash the host kernel:
creating a malicious VM and continuously allocates/releases unpinned
confidential memory pages when the VM is running.

Add cache flush operations to mmu_notifier operations to ensure that any
physical memory leaving the guest VM get flushed. In particular, hook
mmu_notifier_invalidate_range_start and mmu_notifier_release events and
flush cache accordingly. The hook after releasing the mmu lock to avoid
contention with other vCPUs.

Cc: stable@vger.kernel.org
Suggested-by: default avatarSean Christpherson <seanjc@google.com>
Reported-by: default avatarMingwei Zhang <mizhang@google.com>
Signed-off-by: default avatarMingwei Zhang <mizhang@google.com>
Message-Id: <20220421031407.2516575-4-mizhang@google.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
parent d45829b3

arch/x86/include/asm/kvm-x86-ops.h

+1 −0
Original line number Diff line number Diff line
@@ -118,6 +118,7 @@ KVM_X86_OP_OPTIONAL(mem_enc_register_region) 
KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)

KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from)

KVM_X86_OP_OPTIONAL(vm_move_enc_context_from)

KVM_X86_OP_OPTIONAL(guest_memory_reclaimed)

KVM_X86_OP(get_msr_feature)

KVM_X86_OP(can_emulate_instruction)

KVM_X86_OP(apic_init_signal_blocked)

arch/x86/include/asm/kvm_host.h

+1 −0
Original line number Diff line number Diff line
@@ -1484,6 +1484,7 @@ struct kvm_x86_ops { 
	int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp);

	int (*vm_copy_enc_context_from)(struct kvm *kvm, unsigned int source_fd);

	int (*vm_move_enc_context_from)(struct kvm *kvm, unsigned int source_fd);

	void (*guest_memory_reclaimed)(struct kvm *kvm);



	int (*get_msr_feature)(struct kvm_msr_entry *entry);

arch/x86/kvm/svm/sev.c

+8 −0
Original line number Diff line number Diff line
@@ -2262,6 +2262,14 @@ static void sev_flush_encrypted_page(struct kvm_vcpu *vcpu, void *va) 
	wbinvd_on_all_cpus();

}



void sev_guest_memory_reclaimed(struct kvm *kvm)

{

	if (!sev_guest(kvm))

		return;



	wbinvd_on_all_cpus();

}



void sev_free_vcpu(struct kvm_vcpu *vcpu)

{

	struct vcpu_svm *svm;

arch/x86/kvm/svm/svm.c

+1 −0
Original line number Diff line number Diff line
@@ -4620,6 +4620,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { 
	.mem_enc_ioctl = sev_mem_enc_ioctl,

	.mem_enc_register_region = sev_mem_enc_register_region,

	.mem_enc_unregister_region = sev_mem_enc_unregister_region,

	.guest_memory_reclaimed = sev_guest_memory_reclaimed,



	.vm_copy_enc_context_from = sev_vm_copy_enc_context_from,

	.vm_move_enc_context_from = sev_vm_move_enc_context_from,

arch/x86/kvm/svm/svm.h

+2 −0
Original line number Diff line number Diff line
@@ -609,6 +609,8 @@ int sev_mem_enc_unregister_region(struct kvm *kvm, 
				  struct kvm_enc_region *range);

int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd);

int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd);

void sev_guest_memory_reclaimed(struct kvm *kvm);



void pre_sev_run(struct vcpu_svm *svm, int cpu);

void __init sev_set_cpu_caps(void);

void __init sev_hardware_setup(void);