Commit 1dc61e7b authored by Dr. David Alan Gilbert's avatar Dr. David Alan Gilbert Committed by Michael S. Tsirkin
Browse files

postcopy shared docs



Add some notes to the migration documentation for shared memory
postcopy.

Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: default avatarMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: default avatarMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: default avatarMichael S. Tsirkin <mst@redhat.com>
parent 4275cd99
Loading
Loading
Loading
Loading
+41 −0
Original line number Diff line number Diff line
@@ -577,3 +577,44 @@ Postcopy now works with hugetlbfs backed memory:
     hugepages works well, however 1GB hugepages are likely to be problematic
     since it takes ~1 second to transfer a 1GB hugepage across a 10Gbps link,
     and until the full page is transferred the destination thread is blocked.

Postcopy with shared memory
---------------------------

Postcopy migration with shared memory needs explicit support from the other
processes that share memory and from QEMU. There are restrictions on the type of
memory that userfault can support shared.

The Linux kernel userfault support works on `/dev/shm` memory and on `hugetlbfs`
(although the kernel doesn't provide an equivalent to `madvise(MADV_DONTNEED)`
for hugetlbfs which may be a problem in some configurations).

The vhost-user code in QEMU supports clients that have Postcopy support,
and the `vhost-user-bridge` (in `tests/`) and the DPDK package have changes
to support postcopy.

The client needs to open a userfaultfd and register the areas
of memory that it maps with userfault.  The client must then pass the
userfaultfd back to QEMU together with a mapping table that allows
fault addresses in the clients address space to be converted back to
RAMBlock/offsets.  The client's userfaultfd is added to the postcopy
fault-thread and page requests are made on behalf of the client by QEMU.
QEMU performs 'wake' operations on the client's userfaultfd to allow it
to continue after a page has arrived.

.. note::
  There are two future improvements that would be nice:
    a) Some way to make QEMU ignorant of the addresses in the clients
       address space
    b) Avoiding the need for QEMU to perform ufd-wake calls after the
       pages have arrived

Retro-fitting postcopy to existing clients is possible:
  a) A mechanism is needed for the registration with userfault as above,
     and the registration needs to be coordinated with the phases of
     postcopy.  In vhost-user extra messages are added to the existing
     control channel.
  b) Any thread that can block due to guest memory accesses must be
     identified and the implication understood; for example if the
     guest memory access is made while holding a lock then all other
     threads waiting for that lock will also be blocked.