Commit a937f37d authored by Marc Zyngier's avatar Marc Zyngier
Browse files

Merge branch kvm-arm64/dirty-ring into kvmarm-master/next 



* kvm-arm64/dirty-ring:
  : .
  : Add support for the "per-vcpu dirty-ring tracking with a bitmap
  : and sprinkles on top", courtesy of Gavin Shan.
  :
  : This branch drags the kvmarm-fixes-6.1-3 tag which was already
  : merged in 6.1-rc4 so that the branch is in a working state.
  : .
  KVM: Push dirty information unconditionally to backup bitmap
  KVM: selftests: Automate choosing dirty ring size in dirty_log_test
  KVM: selftests: Clear dirty ring states between two modes in dirty_log_test
  KVM: selftests: Use host page size to map ring buffer in dirty_log_test
  KVM: arm64: Enable ring-based dirty memory tracking
  KVM: Support dirty ring in conjunction with bitmap
  KVM: Move declaration of kvm_cpu_dirty_log_size() to kvm_dirty_ring.h
  KVM: x86: Introduce KVM_REQ_DIRTY_RING_SOFT_FULL

Signed-off-by: default avatarMarc Zyngier <maz@kernel.org>
parents 3bbcc8cc c57351a7

Documentation/virt/kvm/api.rst

+28 −8
Original line number Diff line number Diff line
@@ -7921,7 +7921,7 @@ regardless of what has actually been exposed through the CPUID leaf. 
8.29 KVM_CAP_DIRTY_LOG_RING/KVM_CAP_DIRTY_LOG_RING_ACQ_REL

----------------------------------------------------------



:Architectures: x86

:Architectures: x86, arm64

:Parameters: args[0] - size of the dirty log ring



KVM is capable of tracking dirty memory using ring buffers that are
@@ -8003,13 +8003,6 @@ flushing is done by the KVM_GET_DIRTY_LOG ioctl). To achieve that, one 
needs to kick the vcpu out of KVM_RUN using a signal.  The resulting

vmexit ensures that all dirty GFNs are flushed to the dirty rings.



NOTE: the capability KVM_CAP_DIRTY_LOG_RING and the corresponding

ioctl KVM_RESET_DIRTY_RINGS are mutual exclusive to the existing ioctls

KVM_GET_DIRTY_LOG and KVM_CLEAR_DIRTY_LOG.  After enabling

KVM_CAP_DIRTY_LOG_RING with an acceptable dirty ring size, the virtual

machine will switch to ring-buffer dirty page tracking and further

KVM_GET_DIRTY_LOG or KVM_CLEAR_DIRTY_LOG ioctls will fail.



NOTE: KVM_CAP_DIRTY_LOG_RING_ACQ_REL is the only capability that

should be exposed by weakly ordered architecture, in order to indicate

the additional memory ordering requirements imposed on userspace when
@@ -8018,6 +8011,33 @@ Architecture with TSO-like ordering (such as x86) are allowed to 
expose both KVM_CAP_DIRTY_LOG_RING and KVM_CAP_DIRTY_LOG_RING_ACQ_REL

to userspace.



After enabling the dirty rings, the userspace needs to detect the

capability of KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP to see whether the

ring structures can be backed by per-slot bitmaps. With this capability

advertised, it means the architecture can dirty guest pages without

vcpu/ring context, so that some of the dirty information will still be

maintained in the bitmap structure. KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP

can't be enabled if the capability of KVM_CAP_DIRTY_LOG_RING_ACQ_REL

hasn't been enabled, or any memslot has been existing.



Note that the bitmap here is only a backup of the ring structure. The

use of the ring and bitmap combination is only beneficial if there is

only a very small amount of memory that is dirtied out of vcpu/ring

context. Otherwise, the stand-alone per-slot bitmap mechanism needs to

be considered.



To collect dirty bits in the backup bitmap, userspace can use the same

KVM_GET_DIRTY_LOG ioctl. KVM_CLEAR_DIRTY_LOG isn't needed as long as all

the generation of the dirty bits is done in a single pass. Collecting

the dirty bitmap should be the very last thing that the VMM does before

considering the state as complete. VMM needs to ensure that the dirty

state is final and avoid missing dirty pages from another ioctl ordered

after the bitmap collection.



NOTE: One example of using the backup bitmap is saving arm64 vgic/its

tables through KVM_DEV_ARM_{VGIC_GRP_CTRL, ITS_SAVE_TABLES} command on

KVM device "kvm-arm-vgic-its" when dirty ring is enabled.



8.30 KVM_CAP_XEN_HVM

--------------------

Documentation/virt/kvm/devices/arm-vgic-its.rst

+4 −1
Original line number Diff line number Diff line
@@ -52,7 +52,10 @@ KVM_DEV_ARM_VGIC_GRP_CTRL 


    KVM_DEV_ARM_ITS_SAVE_TABLES

      save the ITS table data into guest RAM, at the location provisioned

      by the guest in corresponding registers/table entries.

      by the guest in corresponding registers/table entries. Should userspace

      require a form of dirty tracking to identify which pages are modified

      by the saving process, it should use a bitmap even if using another

      mechanism to track the memory dirtied by the vCPUs.



      The layout of the tables in guest memory defines an ABI. The entries

      are laid out in little endian format as described in the last paragraph.

arch/arm64/include/uapi/asm/kvm.h

+1 −0
Original line number Diff line number Diff line
@@ -43,6 +43,7 @@ 
#define __KVM_HAVE_VCPU_EVENTS



#define KVM_COALESCED_MMIO_PAGE_OFFSET 1

#define KVM_DIRTY_LOG_PAGE_OFFSET 64



#define KVM_REG_SIZE(id)						\

	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))

arch/arm64/kvm/Kconfig

+2 −0
Original line number Diff line number Diff line
@@ -32,6 +32,8 @@ menuconfig KVM 
	select KVM_VFIO

	select HAVE_KVM_EVENTFD

	select HAVE_KVM_IRQFD

	select HAVE_KVM_DIRTY_RING_ACQ_REL

	select NEED_KVM_DIRTY_RING_WITH_BITMAP

	select HAVE_KVM_MSI

	select HAVE_KVM_IRQCHIP

	select HAVE_KVM_IRQ_ROUTING

arch/arm64/kvm/arm.c

+3 −0
Original line number Diff line number Diff line
@@ -746,6 +746,9 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu) 


		if (kvm_check_request(KVM_REQ_SUSPEND, vcpu))

			return kvm_vcpu_suspend(vcpu);



		if (kvm_dirty_ring_check_request(vcpu))

			return 0;

	}



	return 1;