Commit 84cfc756 authored by Peter Maydell's avatar Peter Maydell
Browse files

Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20160526.1' into staging 



VFIO updates 2016-05-26

 - Infrastructure and quirks to support IGD assignment (Alex Williamson)
 - Fixes to 128bit handling, IOMMU replay, IOMMU translation sanity
   checking (Alexey Kardashevskiy)

# gpg: Signature made Thu 26 May 2016 18:50:29 BST using RSA key ID 3BB08B22
# gpg: Good signature from "Alex Williamson <alex.williamson@redhat.com>"
# gpg:                 aka "Alex Williamson <alex@shazbot.org>"
# gpg:                 aka "Alex Williamson <alwillia@redhat.com>"
# gpg:                 aka "Alex Williamson <alex.l.williamson@gmail.com>"

* remotes/awilliam/tags/vfio-update-20160526.1:
  vfio: Check that IOMMU MR translates to system address space
  memory: Fix IOMMU replay base address
  vfio: Fix 128 bit handling when deleting region
  vfio/pci: Add IGD documentation
  vfio/pci: Add a separate option for IGD OpRegion support
  vfio/pci: Intel graphics legacy mode assignment
  vfio/pci: Setup BAR quirks after capabilities probing
  vfio/pci: Consolidate VGA setup
  vfio/pci: Fix return of vfio_populate_vga()
  vfio: Create device specific region info helper
  vfio: Enable sparse mmap capability

Signed-off-by: default avatarPeter Maydell <peter.maydell@linaro.org>
parents aef11b8d f1f93650

docs/igd-assign.txt

0 → 100644
+133 −0
Original line number Diff line number Diff line 
Intel Graphics Device (IGD) assignment with vfio-pci

====================================================



IGD has two different modes for assignment using vfio-pci:



1) Universal Pass-Through (UPT) mode:



   In this mode the IGD device is added as a *secondary* (ie. non-primary)

   graphics device in combination with an emulated primary graphics device.

   This mode *requires* guest driver support to remove the external

   dependencies generally associated with IGD (see below).  Those guest

   drivers only support this mode for Broadwell and newer IGD, according to

   Intel.  Additionally, this mode by default, and as officially supported

   by Intel, does not support direct video output.  The intention is to use

   this mode either to provide hardware acceleration to the emulated graphics

   or to use this mode in combination with guest-based remote access software,

   for example VNC (see below for optional output support).  This mode

   theoretically has no device specific handling dependencies on vfio-pci or

   the VM firmware.



2) "Legacy" mode:



   In this mode the IGD device is intended to be the primary and exclusive

   graphics device in the VM[1], as such QEMU does not facilitate any sort

   of remote graphics to the VM in this mode.  A connected physical monitor

   is the intended output device for IGD.  This mode includes several

   requirements and restrictions:



    * IGD must be given address 02.0 on the PCI root bus in the VM

    * The host kernel must support vfio extensions for IGD (v4.6)

    * vfio VGA support very likely needs to be enabled in the host kernel

    * The VM firmware must support specific fw_cfg enablers for IGD

    * The VM machine type must support a PCI host bridge at 00.0 (standard)

    * The VM machine type must provide or allow to be created a special

      ISA/LPC bridge device (vfio-pci-igd-lpc-bridge) on the root bus at

      PCI address 1f.0.

    * The IGD device must have a VGA ROM, either provided via the romfile

      option or loaded automatically through vfio (standard).  rombar=0

      will disable legacy mode support.

    * Hotplug of the IGD device is not supported.

    * The IGD device must be a SandyBridge or newer model device.



For either mode, depending on the host kernel, the i915 driver in the host

may generate faults and errors upon re-binding to an IGD device after it

has been assigned to a VM.  It's therefore generally recommended to prevent

such driver binding unless the host driver is known to work well for this.

There are numerous ways to do this, i915 can be blacklisted on the host,

the driver_override option can be used to ensure that only vfio-pci can bind

to the device on the host[2], virsh nodedev-detach can be used to bind the

device to vfio drivers and then managed='no' set in the VM xml to prevent

re-binding to i915, etc.  Also note that IGD is also typically the primary

graphics in the host and special options may be required beyond simply

blacklisting i915 or using pci-stub/vfio-pci to take ownership of IGD as a

PCI class device.  Lower level drivers exist that may still claim the device.

It may therefore be necessary to use kernel boot options video=vesafb:off or

video=efifb:off (depending on host BIOS/UEFI) or these can be combined to

a catch-all, video=vesafb:off,efifb:off.  Error messages such as:



    Failed to mmap 0000:00:02.0 BAR <>. Performance may be slow



are a good indicator that such a problem exists.  The host files /proc/iomem

and /proc/ioports are often useful for identifying drivers consuming ranges

of the device to cause such conflicts.



Additionally, IGD device are known to generate small numbers of DMAR faults

when initially assigned.  It is believed that this is simply the IGD attempting

to access the reserved GTT space after reset, which it no longer has access to

when accessed from userspace.  So long as the DMAR faults are small in number

and most importantly, not ongoing, these are not an indication of an error.



Additionally++, analog VGA output (as opposed to digital outputs like HDMI,

DVI, or DisplayPort) may be unsupported in some use cases.  In the author's

experience, even DP to VGA adapters can be troublesome while adapters between

digital formats work well.



Usage

=====

The intention is for IGD assignment to be transparent for users and thus for

management tools like libvirt.  To make use of legacy mode, simply remove all

other graphics options and use "-nographic" and either "-vga none" or

"-nodefaults", along with adding the device using vfio-pci:



    -device vfio-pci,host=00:02.0,id=hostdev0,bus=pci.0,addr=0x2



For UPT mode, retain the default emulated graphics and simply add the vfio-pci

device making use of any other bus address other than 02.0.  libvirt will

default to assigning the device a UPT compatible address while legacy mode

users will need to manually edit the XML if using a tool like virt-manager

where the VM device address is not expressly specified.



An experimental vfio-pci option also exists to enable OpRegion, and thus

external monitor support, for UPT mode.  This can be enabled by adding

"x-igd-opregion=on" to the vfio-pci device options for the IGD device.  As

with legacy mode, this requires the host to support features introduced in

the v4.6 kernel.  If Intel chooses to embrace this support, the option may

be made non-experimental in the future, opening it to libvirt support.



Developer ABI

=============

Legacy mode IGD support imposes two fw_cfg requirements on the VM firmware:



1) "etc/igd-opregion"



   This fw_cfg file exposes the OpRegion for the IGD device.  A reserved

   region should be created below 4GB (recommended 4KB alignment), sized

   sufficient for the fw_cfg file size, and the content of this file copied

   to it.  The dword based address of this reserved memory region must also

   be written to the ASLS register at offset 0xFC on the IGD device.  It is

   recommended that firmware should make use of this fw_cfg entry for any

   PCI class VGA device with Intel vendor ID.  Multiple of such devices

   within a VM is undefined.



2) "etc/igd-bdsm-size"



   This fw_cfg file contains an 8-byte, little endian integer indicating

   the size of the reserved memory region required for IGD stolen memory.

   Firmware must allocate a reserved memory below 4GB with required 1MB

   alignment equal to this size.  Additionally the base address of this

   reserved region must be written to the dword BDSM register in PCI config

   space of the IGD device at offset 0x5C.  As this support is related to

   running the IGD ROM, which has other dependencies on the device appearing

   at guest address 00:02.0, it's expected that this fw_cfg file is only

   relevant to a single PCI class VGA device with Intel vendor ID, appearing

   at PCI bus address 00:02.0.



Footnotes

=========

[1] Nothing precludes adding additional emulated or assigned graphics devices

    as non-primary, other than the combination typically not working.  I only

    intend to set user expectations, others are welcome to find working

    combinations or fix whatever issues prevent this from working in the common

    case.

[2] # echo "vfio-pci" > /sys/bus/pci/devices/0000:00:02.0/driver_override

hw/ppc/spapr_iommu.c

+1 −1
Original line number Diff line number Diff line
@@ -278,7 +278,7 @@ static target_ulong put_tce_emu(sPAPRTCETable *tcet, target_ulong ioba, 
    tcet->table[index] = tce;



    entry.target_as = &address_space_memory,

    entry.iova = ioba & page_mask;

    entry.iova = (ioba - tcet->bus_offset) & page_mask;

    entry.translated_addr = tce & page_mask;

    entry.addr_mask = ~page_mask;

    entry.perm = spapr_tce_iommu_access_flags(tce);

hw/vfio/common.c

+124 −16
Original line number Diff line number Diff line
@@ -260,14 +260,20 @@ static void vfio_iommu_map_notify(Notifier *n, void *data) 
    VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n);

    VFIOContainer *container = giommu->container;

    IOMMUTLBEntry *iotlb = data;

    hwaddr iova = iotlb->iova + giommu->iommu_offset;

    MemoryRegion *mr;

    hwaddr xlat;

    hwaddr len = iotlb->addr_mask + 1;

    void *vaddr;

    int ret;



    trace_vfio_iommu_map_notify(iotlb->iova,

                                iotlb->iova + iotlb->addr_mask);

    trace_vfio_iommu_map_notify(iova, iova + iotlb->addr_mask);



    if (iotlb->target_as != &address_space_memory) {

        error_report("Wrong target AS \"%s\", only system memory is allowed",

                     iotlb->target_as->name ? iotlb->target_as->name : "none");

        return;

    }



    /*

     * The IOMMU TLB entry we have just covers translation through
@@ -294,21 +300,21 @@ static void vfio_iommu_map_notify(Notifier *n, void *data) 


    if ((iotlb->perm & IOMMU_RW) != IOMMU_NONE) {

        vaddr = memory_region_get_ram_ptr(mr) + xlat;

        ret = vfio_dma_map(container, iotlb->iova,

        ret = vfio_dma_map(container, iova,

                           iotlb->addr_mask + 1, vaddr,

                           !(iotlb->perm & IOMMU_WO) || mr->readonly);

        if (ret) {

            error_report("vfio_dma_map(%p, 0x%"HWADDR_PRIx", "

                         "0x%"HWADDR_PRIx", %p) = %d (%m)",

                         container, iotlb->iova,

                         container, iova,

                         iotlb->addr_mask + 1, vaddr, ret);

        }

    } else {

        ret = vfio_dma_unmap(container, iotlb->iova, iotlb->addr_mask + 1);

        ret = vfio_dma_unmap(container, iova, iotlb->addr_mask + 1);

        if (ret) {

            error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", "

                         "0x%"HWADDR_PRIx") = %d (%m)",

                         container, iotlb->iova,

                         container, iova,

                         iotlb->addr_mask + 1, ret);

        }

    }
@@ -380,6 +386,8 @@ static void vfio_listener_region_add(MemoryListener *listener, 
         */

        giommu = g_malloc0(sizeof(*giommu));

        giommu->iommu = section->mr;

        giommu->iommu_offset = section->offset_within_address_space -

                               section->offset_within_region;

        giommu->container = container;

        giommu->n.notify = vfio_iommu_map_notify;

        QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next);
@@ -433,6 +441,7 @@ static void vfio_listener_region_del(MemoryListener *listener, 
{

    VFIOContainer *container = container_of(listener, VFIOContainer, listener);

    hwaddr iova, end;

    Int128 llend, llsize;

    int ret;



    if (vfio_listener_skipped_section(section)) {
@@ -471,21 +480,25 @@ static void vfio_listener_region_del(MemoryListener *listener, 
    }



    iova = TARGET_PAGE_ALIGN(section->offset_within_address_space);

    end = (section->offset_within_address_space + int128_get64(section->size)) &

          TARGET_PAGE_MASK;

    llend = int128_make64(section->offset_within_address_space);

    llend = int128_add(llend, section->size);

    llend = int128_and(llend, int128_exts64(TARGET_PAGE_MASK));



    if (iova >= end) {

    if (int128_ge(int128_make64(iova), llend)) {

        return;

    }

    end = int128_get64(int128_sub(llend, int128_one()));



    llsize = int128_sub(llend, int128_make64(iova));



    trace_vfio_listener_region_del(iova, end - 1);

    trace_vfio_listener_region_del(iova, end);



    ret = vfio_dma_unmap(container, iova, end - iova);

    ret = vfio_dma_unmap(container, iova, int128_get64(llsize));

    memory_region_unref(section->mr);

    if (ret) {

        error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", "

                     "0x%"HWADDR_PRIx") = %d (%m)",

                     container, iova, end - iova, ret);

                     container, iova, int128_get64(llsize), ret);

    }

}
@@ -499,6 +512,54 @@ static void vfio_listener_release(VFIOContainer *container) 
    memory_listener_unregister(&container->listener);

}



static struct vfio_info_cap_header *

vfio_get_region_info_cap(struct vfio_region_info *info, uint16_t id)

{

    struct vfio_info_cap_header *hdr;

    void *ptr = info;



    if (!(info->flags & VFIO_REGION_INFO_FLAG_CAPS)) {

        return NULL;

    }



    for (hdr = ptr + info->cap_offset; hdr != ptr; hdr = ptr + hdr->next) {

        if (hdr->id == id) {

            return hdr;

        }

    }



    return NULL;

}



static void vfio_setup_region_sparse_mmaps(VFIORegion *region,

                                           struct vfio_region_info *info)

{

    struct vfio_info_cap_header *hdr;

    struct vfio_region_info_cap_sparse_mmap *sparse;

    int i;



    hdr = vfio_get_region_info_cap(info, VFIO_REGION_INFO_CAP_SPARSE_MMAP);

    if (!hdr) {

        return;

    }



    sparse = container_of(hdr, struct vfio_region_info_cap_sparse_mmap, header);



    trace_vfio_region_sparse_mmap_header(region->vbasedev->name,

                                         region->nr, sparse->nr_areas);



    region->nr_mmaps = sparse->nr_areas;

    region->mmaps = g_new0(VFIOMmap, region->nr_mmaps);



    for (i = 0; i < region->nr_mmaps; i++) {

        region->mmaps[i].offset = sparse->areas[i].offset;

        region->mmaps[i].size = sparse->areas[i].size;

        trace_vfio_region_sparse_mmap_entry(i, region->mmaps[i].offset,

                                            region->mmaps[i].offset +

                                            region->mmaps[i].size);

    }

}



int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region,

                      int index, const char *name)

{
@@ -525,13 +586,16 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region, 
            region->flags & VFIO_REGION_INFO_FLAG_MMAP &&

            !(region->size & ~qemu_real_host_page_mask)) {



            vfio_setup_region_sparse_mmaps(region, info);



            if (!region->nr_mmaps) {

                region->nr_mmaps = 1;

                region->mmaps = g_new0(VFIOMmap, region->nr_mmaps);



                region->mmaps[0].offset = 0;

                region->mmaps[0].size = region->size;

            }

        }

    }



    g_free(info);
@@ -1089,16 +1153,60 @@ int vfio_get_region_info(VFIODevice *vbasedev, int index, 
    *info = g_malloc0(argsz);



    (*info)->index = index;

retry:

    (*info)->argsz = argsz;



    if (ioctl(vbasedev->fd, VFIO_DEVICE_GET_REGION_INFO, *info)) {

        g_free(*info);

        *info = NULL;

        return -errno;

    }



    if ((*info)->argsz > argsz) {

        argsz = (*info)->argsz;

        *info = g_realloc(*info, argsz);



        goto retry;

    }



    return 0;

}



int vfio_get_dev_region_info(VFIODevice *vbasedev, uint32_t type,

                             uint32_t subtype, struct vfio_region_info **info)

{

    int i;



    for (i = 0; i < vbasedev->num_regions; i++) {

        struct vfio_info_cap_header *hdr;

        struct vfio_region_info_cap_type *cap_type;



        if (vfio_get_region_info(vbasedev, i, info)) {

            continue;

        }



        hdr = vfio_get_region_info_cap(*info, VFIO_REGION_INFO_CAP_TYPE);

        if (!hdr) {

            g_free(*info);

            continue;

        }



        cap_type = container_of(hdr, struct vfio_region_info_cap_type, header);



        trace_vfio_get_dev_region(vbasedev->name, i,

                                  cap_type->type, cap_type->subtype);



        if (cap_type->type == type && cap_type->subtype == subtype) {

            return 0;

        }



        g_free(*info);

    }



    *info = NULL;

    return -ENODEV;

}



/*

 * Interfaces for IBM EEH (Enhanced Error Handling)

 */

hw/vfio/pci-quirks.c

+642 −1

File changed.

Preview size limit exceeded, changes collapsed.

hw/vfio/pci.c

+96 −55
Original line number Diff line number Diff line
@@ -1440,8 +1440,6 @@ static void vfio_bar_setup(VFIOPCIDevice *vdev, int nr) 
                     vdev->vbasedev.name, nr);

    }



    vfio_bar_quirk_setup(vdev, nr);



    pci_register_bar(&vdev->pdev, nr, type, bar->region.mem);

}
@@ -1452,29 +1450,6 @@ static void vfio_bars_setup(VFIOPCIDevice *vdev) 
    for (i = 0; i < PCI_ROM_SLOT; i++) {

        vfio_bar_setup(vdev, i);

    }



    if (vdev->vga) {

        memory_region_init_io(&vdev->vga->region[QEMU_PCI_VGA_MEM].mem,

                              OBJECT(vdev), &vfio_vga_ops,

                              &vdev->vga->region[QEMU_PCI_VGA_MEM],

                              "vfio-vga-mmio@0xa0000",

                              QEMU_PCI_VGA_MEM_SIZE);

        memory_region_init_io(&vdev->vga->region[QEMU_PCI_VGA_IO_LO].mem,

                              OBJECT(vdev), &vfio_vga_ops,

                              &vdev->vga->region[QEMU_PCI_VGA_IO_LO],

                              "vfio-vga-io@0x3b0",

                              QEMU_PCI_VGA_IO_LO_SIZE);

        memory_region_init_io(&vdev->vga->region[QEMU_PCI_VGA_IO_HI].mem,

                              OBJECT(vdev), &vfio_vga_ops,

                              &vdev->vga->region[QEMU_PCI_VGA_IO_HI],

                              "vfio-vga-io@0x3c0",

                              QEMU_PCI_VGA_IO_HI_SIZE);



        pci_register_vga(&vdev->pdev, &vdev->vga->region[QEMU_PCI_VGA_MEM].mem,

                         &vdev->vga->region[QEMU_PCI_VGA_IO_LO].mem,

                         &vdev->vga->region[QEMU_PCI_VGA_IO_HI].mem);

        vfio_vga_quirk_setup(vdev);

    }

}



static void vfio_bars_exit(VFIOPCIDevice *vdev)
@@ -2061,9 +2036,7 @@ int vfio_populate_vga(VFIOPCIDevice *vdev) 
    struct vfio_region_info *reg_info;

    int ret;



    if (vbasedev->num_regions > VFIO_PCI_VGA_REGION_INDEX) {

        ret = vfio_get_region_info(vbasedev,

                                   VFIO_PCI_VGA_REGION_INDEX, &reg_info);

    ret = vfio_get_region_info(vbasedev, VFIO_PCI_VGA_REGION_INDEX, &reg_info);

    if (ret) {

        return ret;

    }
@@ -2089,14 +2062,35 @@ int vfio_populate_vga(VFIOPCIDevice *vdev) 
    vdev->vga->region[QEMU_PCI_VGA_MEM].nr = QEMU_PCI_VGA_MEM;

    QLIST_INIT(&vdev->vga->region[QEMU_PCI_VGA_MEM].quirks);



    memory_region_init_io(&vdev->vga->region[QEMU_PCI_VGA_MEM].mem,

                          OBJECT(vdev), &vfio_vga_ops,

                          &vdev->vga->region[QEMU_PCI_VGA_MEM],

                          "vfio-vga-mmio@0xa0000",

                          QEMU_PCI_VGA_MEM_SIZE);



    vdev->vga->region[QEMU_PCI_VGA_IO_LO].offset = QEMU_PCI_VGA_IO_LO_BASE;

    vdev->vga->region[QEMU_PCI_VGA_IO_LO].nr = QEMU_PCI_VGA_IO_LO;

    QLIST_INIT(&vdev->vga->region[QEMU_PCI_VGA_IO_LO].quirks);



    memory_region_init_io(&vdev->vga->region[QEMU_PCI_VGA_IO_LO].mem,

                          OBJECT(vdev), &vfio_vga_ops,

                          &vdev->vga->region[QEMU_PCI_VGA_IO_LO],

                          "vfio-vga-io@0x3b0",

                          QEMU_PCI_VGA_IO_LO_SIZE);



    vdev->vga->region[QEMU_PCI_VGA_IO_HI].offset = QEMU_PCI_VGA_IO_HI_BASE;

    vdev->vga->region[QEMU_PCI_VGA_IO_HI].nr = QEMU_PCI_VGA_IO_HI;

    QLIST_INIT(&vdev->vga->region[QEMU_PCI_VGA_IO_HI].quirks);

    }



    memory_region_init_io(&vdev->vga->region[QEMU_PCI_VGA_IO_HI].mem,

                          OBJECT(vdev), &vfio_vga_ops,

                          &vdev->vga->region[QEMU_PCI_VGA_IO_HI],

                          "vfio-vga-io@0x3c0",

                          QEMU_PCI_VGA_IO_HI_SIZE);



    pci_register_vga(&vdev->pdev, &vdev->vga->region[QEMU_PCI_VGA_MEM].mem,

                     &vdev->vga->region[QEMU_PCI_VGA_IO_LO].mem,

                     &vdev->vga->region[QEMU_PCI_VGA_IO_HI].mem);



    return 0;

}
@@ -2398,7 +2392,7 @@ static int vfio_initfn(PCIDevice *pdev) 
    ssize_t len;

    struct stat st;

    int groupid;

    int ret;

    int i, ret;



    if (!vdev->vbasedev.sysfsdev) {

        vdev->vbasedev.sysfsdev =
@@ -2560,6 +2554,43 @@ static int vfio_initfn(PCIDevice *pdev) 
        goto out_teardown;

    }



    if (vdev->vga) {

        vfio_vga_quirk_setup(vdev);

    }



    for (i = 0; i < PCI_ROM_SLOT; i++) {

        vfio_bar_quirk_setup(vdev, i);

    }



    if (!vdev->igd_opregion &&

        vdev->features & VFIO_FEATURE_ENABLE_IGD_OPREGION) {

        struct vfio_region_info *opregion;



        if (vdev->pdev.qdev.hotplugged) {

            error_report("Cannot support IGD OpRegion feature on hotplugged "

                         "device %s", vdev->vbasedev.name);

            ret = -EINVAL;

            goto out_teardown;

        }



        ret = vfio_get_dev_region_info(&vdev->vbasedev,

                        VFIO_REGION_TYPE_PCI_VENDOR_TYPE | PCI_VENDOR_ID_INTEL,

                        VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION, &opregion);

        if (ret) {

            error_report("Device %s does not support requested IGD OpRegion "

                         "feature", vdev->vbasedev.name);

            goto out_teardown;

        }



        ret = vfio_pci_igd_opregion_init(vdev, opregion);

        g_free(opregion);

        if (ret) {

            error_report("Device %s IGD OpRegion initialization failed",

                         vdev->vbasedev.name);

            goto out_teardown;

        }

    }



    /* QEMU emulates all of MSI & MSIX */

    if (pdev->cap_present & QEMU_PCI_CAP_MSIX) {

        memset(vdev->emulated_config_bits + pdev->msix_cap, 0xff,
@@ -2603,6 +2634,13 @@ static void vfio_instance_finalize(Object *obj) 
    vfio_bars_finalize(vdev);

    g_free(vdev->emulated_config_bits);

    g_free(vdev->rom);

    /*

     * XXX Leaking igd_opregion is not an oversight, we can't remove the

     * fw_cfg entry therefore leaking this allocation seems like the safest

     * option.

     *

     * g_free(vdev->igd_opregion);

     */

    vfio_put_device(vdev);

    vfio_put_group(group);

}
@@ -2677,6 +2715,8 @@ static Property vfio_pci_dev_properties[] = { 
                    VFIO_FEATURE_ENABLE_VGA_BIT, false),

    DEFINE_PROP_BIT("x-req", VFIOPCIDevice, features,

                    VFIO_FEATURE_ENABLE_REQ_BIT, true),

    DEFINE_PROP_BIT("x-igd-opregion", VFIOPCIDevice, features,

                    VFIO_FEATURE_ENABLE_IGD_OPREGION_BIT, false),

    DEFINE_PROP_BOOL("x-no-mmap", VFIOPCIDevice, vbasedev.no_mmap, false),

    DEFINE_PROP_BOOL("x-no-kvm-intx", VFIOPCIDevice, no_kvm_intx, false),

    DEFINE_PROP_BOOL("x-no-kvm-msi", VFIOPCIDevice, no_kvm_msi, false),
@@ -2687,6 +2727,7 @@ static Property vfio_pci_dev_properties[] = { 
                       sub_vendor_id, PCI_ANY_ID),

    DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice,

                       sub_device_id, PCI_ANY_ID),

    DEFINE_PROP_UINT32("x-igd-gms", VFIOPCIDevice, igd_gms, 0),

    /*

     * TODO - support passed fds... is this necessary?

     * DEFINE_PROP_STRING("vfiofd", VFIOPCIDevice, vfiofd_name),