Commit b87a0100 authored by Cédric Le Goater's avatar Cédric Le Goater Committed by David Gibson
Browse files

docs: updates on the POWER9 XIVE interrupt controller documentation 



This includes various small updates and a better description of the
chosen interrupt mode resulting from the combination of the 'ic-mode'
machine option, the 'kernel_irqchip' option, guest support and KVM
support.

Signed-off-by: default avatarCédric Le Goater <clg@kaod.org>
Message-Id: <20190612160425.27670-1-clg@kaod.org>
Signed-off-by: default avatarDavid Gibson <david@gibson.dropbear.id.au>
parent fad189d1

docs/specs/ppc-spapr-xive.rst

+106 −6
Original line number Diff line number Diff line
@@ -34,19 +34,118 @@ CAS Negotiation 
---------------



QEMU advertises the supported interrupt modes in the device tree

property "ibm,arch-vec-5-platform-support" in byte 23 and the OS

Selection for XIVE is indicated in the "ibm,architecture-vec-5"

property ``ibm,arch-vec-5-platform-support`` in byte 23 and the OS

Selection for XIVE is indicated in the ``ibm,architecture-vec-5``

property byte 23.



The interrupt modes supported by the machine depend on the CPU type

(POWER9 is required for XIVE) but also on the machine property

``ic-mode`` which can be set on the command line. It can take the

following values: ``xics``, ``xive``, ``dual`` and currently ``xics``

is the default but it may change in the future.

following values: ``xics``, ``xive``, and ``dual`` which is the

default mode. ``dual`` means that both modes XICS **and** XIVE are

supported and if the guest OS supports XIVE, this mode will be

selected.



The choosen interrupt mode is activated after a reconfiguration done

in a machine reset.



KVM negotiation

---------------



When the guest starts under KVM, the capabilities of the host kernel

and QEMU are also negotiated. Depending on the version of the host

kernel, KVM will advertise the XIVE capability to QEMU or not.



Nevertheless, the available interrupt modes in the machine should not

depend on the XIVE KVM capability of the host. On older kernels

without XIVE KVM support, QEMU will use the emulated XIVE device as a

fallback and on newer kernels (>=5.2), the KVM XIVE device.



As a final refinement, the user can also switch the use of the KVM

device with the machine option ``kernel_irqchip``.





XIVE support in KVM

~~~~~~~~~~~~~~~~~~~



For guest OSes supporting XIVE, the resulting interrupt modes on host

kernels with XIVE KVM support are the following:



==============  =============  =============  ================

ic-mode                            kernel_irqchip

--------------  ----------------------------------------------

/               allowed        off            on

                (default)

==============  =============  =============  ================

dual (default)  XIVE KVM       XIVE emul.     XIVE KVM

xive            XIVE KVM       XIVE emul.     XIVE KVM

xics            XICS KVM       XICS emul.     XICS KVM

==============  =============  =============  ================



For legacy guest OSes without XIVE support, the resulting interrupt

modes are the following:



==============  =============  =============  ================

ic-mode                            kernel_irqchip

--------------  ----------------------------------------------

/               allowed        off            on

                (default)

==============  =============  =============  ================

dual (default)  XICS KVM       XICS emul.     XICS KVM

xive            QEMU error(3)  QEMU error(3)  QEMU error(3)

xics            XICS KVM       XICS emul.     XICS KVM

==============  =============  =============  ================



(3) QEMU fails at CAS with ``Guest requested unavailable interrupt

    mode (XICS), either don't set the ic-mode machine property or try

    ic-mode=xics or ic-mode=dual``





No XIVE support in KVM

~~~~~~~~~~~~~~~~~~~~~~



For guest OSes supporting XIVE, the resulting interrupt modes on host

kernels without XIVE KVM support are the following:



==============  =============  =============  ================

ic-mode                            kernel_irqchip

--------------  ----------------------------------------------

/               allowed        off            on

                (default)

==============  =============  =============  ================

dual (default)  XIVE emul.(1)  XIVE emul.     QEMU error (2)

xive            XIVE emul.(1)  XIVE emul.     QEMU error (2)

xics            XICS KVM       XICS emul.     XICS KVM

==============  =============  =============  ================





(1) QEMU warns with ``warning: kernel_irqchip requested but unavailable:

    IRQ_XIVE capability must be present for KVM``

(2) QEMU fails with ``kernel_irqchip requested but unavailable:

    IRQ_XIVE capability must be present for KVM``





For legacy guest OSes without XIVE support, the resulting interrupt

modes are the following:



==============  =============  =============  ================

ic-mode                            kernel_irqchip

--------------  ----------------------------------------------

/               allowed        off            on

                (default)

==============  =============  =============  ================

dual (default)  QEMU error(4)  XICS emul.     QEMU error(4)

xive            QEMU error(3)  QEMU error(3)  QEMU error(3)

xics            XICS KVM       XICS emul.     XICS KVM

==============  =============  =============  ================



(3) QEMU fails at CAS with ``Guest requested unavailable interrupt

    mode (XICS), either don't set the ic-mode machine property or try

    ic-mode=xics or ic-mode=dual``

(4) QEMU/KVM incompatibility due to device destruction in reset. This

    needs to be addressed more cleanly with an error.





XIVE Device tree properties

---------------------------
@@ -92,10 +191,11 @@ for both interrupt mode. The different ranges are defined as follow : 
- ``0x0000 .. 0x0FFF`` 4K CPU IPIs (only used under XIVE)

- ``0x1000 .. 0x1000`` 1 EPOW

- ``0x1001 .. 0x1001`` 1 HOTPLUG

- ``0x1002 .. 0x10FF`` unused

- ``0x1100 .. 0x11FF`` 256 VIO devices

- ``0x1200 .. 0x127F`` 32 PHBs devices

- ``0x1200 .. 0x127F`` 32x4 LSIs for PHB devices

- ``0x1280 .. 0x12FF`` unused

- ``0x1300 .. 0x1FFF`` PHB MSIs

- ``0x1300 .. 0x1FFF`` PHB MSIs (dynamically allocated)



Monitoring XIVE

---------------

docs/specs/ppc-xive.rst

+5 −4
Original line number Diff line number Diff line
@@ -20,10 +20,11 @@ The XIVE IC is composed of three sub-engines, each taking care of a 
processing layer of external interrupts:



- Interrupt Virtualization Source Engine (IVSE), or Source Controller

  (SC). These are found in PCI PHBs, in the PSI host bridge

  controller, but also inside the main controller for the core IPIs

  and other sub-chips (NX, CAP, NPU) of the chip/processor. They are

  configured to feed the IVRE with events.

  (SC). These are found in PCI PHBs, in the Processor Service

  Interface (PSI) host bridge Controller, but also inside the main

  controller for the core IPIs and other sub-chips (NX, CAP, NPU) of

  the chip/processor. They are configured to feed the IVRE with

  events.

- Interrupt Virtualization Routing Engine (IVRE) or Virtualization

  Controller (VC). It handles event coalescing and perform interrupt

  routing by matching an event source number with an Event