Merge tag 'kvm-x86-fixes-6.2-1' of https://github.com/kvm-x86/linux into HEAD

``KVM_MSR_FILTER_READ``

  Filter read accesses to MSRs using the given bitmap. A 0 in the bitmap

  indicates that a read should immediately fail, while a 1 indicates that

  a read for a particular MSR should be handled regardless of the default

  indicates that read accesses should be denied, while a 1 indicates that

  a read for a particular MSR should be allowed regardless of the default

  filter action.

``KVM_MSR_FILTER_WRITE``

  Filter write accesses to MSRs using the given bitmap. A 0 in the bitmap

  indicates that a write should immediately fail, while a 1 indicates that

  a write for a particular MSR should be handled regardless of the default

  indicates that write accesses should be denied, while a 1 indicates that

  a write for a particular MSR should be allowed regardless of the default

  filter action.

``KVM_MSR_FILTER_READ | KVM_MSR_FILTER_WRITE``

  Filter both read and write accesses to MSRs using the given bitmap. A 0

  in the bitmap indicates that both reads and writes should immediately fail,

  while a 1 indicates that reads and writes for a particular MSR are not

  filtered by this range.

flags values for ``struct kvm_msr_filter``:

``KVM_MSR_FILTER_DEFAULT_ALLOW``

  If no filter range matches an MSR index that is getting accessed, KVM will

  fall back to allowing access to the MSR.

  allow accesses to all MSRs by default.

``KVM_MSR_FILTER_DEFAULT_DENY``

  If no filter range matches an MSR index that is getting accessed, KVM will

  fall back to rejecting access to the MSR. In this mode, all MSRs that should

  be processed by KVM need to explicitly be marked as allowed in the bitmaps.

  deny accesses to all MSRs by default.

This ioctl allows userspace to define up to 16 bitmaps of MSR ranges to deny

guest MSR accesses that would normally be allowed by KVM.  If an MSR is not

covered by a specific range, the "default" filtering behavior applies.  Each

bitmap range covers MSRs from [base .. base+nmsrs).

This ioctl allows user space to define up to 16 bitmaps of MSR ranges to

specify whether a certain MSR access should be explicitly filtered for or not.

If an MSR access is denied by userspace, the resulting KVM behavior depends on

whether or not KVM_CAP_X86_USER_SPACE_MSR's KVM_MSR_EXIT_REASON_FILTER is

enabled.  If KVM_MSR_EXIT_REASON_FILTER is enabled, KVM will exit to userspace

on denied accesses, i.e. userspace effectively intercepts the MSR access.  If

KVM_MSR_EXIT_REASON_FILTER is not enabled, KVM will inject a #GP into the guest

on denied accesses.

If this ioctl has never been invoked, MSR accesses are not guarded and the

default KVM in-kernel emulation behavior is fully preserved.

If an MSR access is allowed by userspace, KVM will emulate and/or virtualize

the access in accordance with the vCPU model.  Note, KVM may still ultimately

inject a #GP if an access is allowed by userspace, e.g. if KVM doesn't support

the MSR, or to follow architectural behavior for the MSR.

By default, KVM operates in KVM_MSR_FILTER_DEFAULT_ALLOW mode with no MSR range

filters.

Calling this ioctl with an empty set of ranges (all nmsrs == 0) disables MSR

filtering. In that mode, ``KVM_MSR_FILTER_DEFAULT_DENY`` is invalid and causes

an error.

As soon as the filtering is in place, every MSR access is processed through

the filtering except for accesses to the x2APIC MSRs (from 0x800 to 0x8ff);

x2APIC MSRs are always allowed, independent of the ``default_allow`` setting,

and their behavior depends on the ``X2APIC_ENABLE`` bit of the APIC base

register.

.. warning::

   MSR accesses coming from nested vmentry/vmexit are not filtered.

   MSR accesses as part of nested VM-Enter/VM-Exit are not filtered.

   This includes both writes to individual VMCS fields and reads/writes

   through the MSR lists pointed to by the VMCS.

If a bit is within one of the defined ranges, read and write accesses are

guarded by the bitmap's value for the MSR index if the kind of access

is included in the ``struct kvm_msr_filter_range`` flags.  If no range

cover this particular access, the behavior is determined by the flags

field in the kvm_msr_filter struct: ``KVM_MSR_FILTER_DEFAULT_ALLOW``

and ``KVM_MSR_FILTER_DEFAULT_DENY``.

Each bitmap range specifies a range of MSRs to potentially allow access on.

The range goes from MSR index [base .. base+nmsrs]. The flags field

indicates whether reads, writes or both reads and writes are filtered

by setting a 1 bit in the bitmap for the corresponding MSR index.

If an MSR access is not permitted through the filtering, it generates a

#GP inside the guest. When combined with KVM_CAP_X86_USER_SPACE_MSR, that

allows user space to deflect and potentially handle various MSR accesses

into user space.

   x2APIC MSR accesses cannot be filtered (KVM silently ignores filters that

   cover any x2APIC MSRs).

Note, invoking this ioctl while a vCPU is running is inherently racy.  However,

KVM does guarantee that vCPUs will see either the previous filter or the new

filter, e.g. MSRs with identical settings in both the old and new filter will

have deterministic behavior.

Similarly, if userspace wishes to intercept on denied accesses,

KVM_MSR_EXIT_REASON_FILTER must be enabled before activating any filters, and

left enabled until after all filters are deactivated.  Failure to do so may

result in KVM injecting a #GP instead of exiting to userspace.

4.98 KVM_CREATE_SPAPR_TCE_64

----------------------------

Used on x86 systems. When the VM capability KVM_CAP_X86_USER_SPACE_MSR is

enabled, MSR accesses to registers that would invoke a #GP by KVM kernel code

will instead trigger a KVM_EXIT_X86_RDMSR exit for reads and KVM_EXIT_X86_WRMSR

may instead trigger a KVM_EXIT_X86_RDMSR exit for reads and KVM_EXIT_X86_WRMSR

exit for writes.

The "reason" field specifies why the MSR trap occurred. User space will only

receive MSR exit traps when a particular reason was requested during through

The "reason" field specifies why the MSR interception occurred. Userspace will

only receive MSR exits when a particular reason was requested during through

ENABLE_CAP. Currently valid exit reasons are:

	KVM_MSR_EXIT_REASON_UNKNOWN - access to MSR that is unknown to KVM

vCPU execution. If the MSR write was unsuccessful, userspace also sets the

"error" field to "1".

See KVM_X86_SET_MSR_FILTER for details on the interaction with MSR filtering.

::

:Parameters: args[0] contains the mask of KVM_MSR_EXIT_REASON_* events to report

:Returns: 0 on success; -1 on error

This capability enables trapping of #GP invoking RDMSR and WRMSR instructions

into user space.

This capability allows userspace to intercept RDMSR and WRMSR instructions if

access to an MSR is denied.  By default, KVM injects #GP on denied accesses.

When a guest requests to read or write an MSR, KVM may not implement all MSRs

that are relevant to a respective system. It also does not differentiate by

To allow more fine grained control over MSR handling, userspace may enable

this capability. With it enabled, MSR accesses that match the mask specified in

args[0] and trigger a #GP event inside the guest by KVM will instead trigger

KVM_EXIT_X86_RDMSR and KVM_EXIT_X86_WRMSR exit notifications which user space

can then handle to implement model specific MSR handling and/or user notifications

to inform a user that an MSR was not handled.

args[0] and would trigger a #GP inside the guest will instead trigger

KVM_EXIT_X86_RDMSR and KVM_EXIT_X86_WRMSR exit notifications.  Userspace

can then implement model specific MSR handling and/or user notifications

to inform a user that an MSR was not emulated/virtualized by KVM.

The valid mask flags are:

	KVM_MSR_EXIT_REASON_UNKNOWN - intercept accesses to unknown (to KVM) MSRs

	KVM_MSR_EXIT_REASON_INVAL   - intercept accesses that are architecturally

                                invalid according to the vCPU model and/or mode

	KVM_MSR_EXIT_REASON_FILTER  - intercept accesses that are denied by userspace

                                via KVM_X86_SET_MSR_FILTER

7.22 KVM_CAP_X86_BUS_LOCK_EXIT

-------------------------------

This capability indicates that KVM supports that accesses to user defined MSRs

may be rejected. With this capability exposed, KVM exports new VM ioctl

KVM_X86_SET_MSR_FILTER which user space can call to specify bitmaps of MSR

ranges that KVM should reject access to.

ranges that KVM should deny access to.

In combination with KVM_CAP_X86_USER_SPACE_MSR, this allows user space to

trap and emulate MSRs that are outside of the scope of KVM as well as

		return;

	for (i = 0; i < npages; i++) {

		page_virtual = kmap_atomic(pages[i]);

		page_virtual = kmap_local_page(pages[i]);

		clflush_cache_range(page_virtual, PAGE_SIZE);

		kunmap_atomic(page_virtual);

		kunmap_local(page_virtual);

		cond_resched();

	}

}

static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu)

{

	if (to_svm(vcpu)->vmcb->control.exit_code == SVM_EXIT_MSR &&

	    to_svm(vcpu)->vmcb->control.exit_info_1)

	struct vmcb_control_area *control = &to_svm(vcpu)->vmcb->control;

	/*

	 * Note, the next RIP must be provided as SRCU isn't held, i.e. KVM

	 * can't read guest memory (dereference memslots) to decode the WRMSR.

	 */

	if (control->exit_code == SVM_EXIT_MSR && control->exit_info_1 &&

	    nrips && control->next_rip)

		return handle_fastpath_set_msr_irqoff(vcpu);

	return EXIT_FASTPATH_NONE;

		nested_ept_init_mmu_context(vcpu);

	/*

	 * This sets GUEST_CR0 to vmcs12->guest_cr0, possibly modifying those

	 * bits which we consider mandatory enabled.

	 * The CR0_READ_SHADOW is what L2 should have expected to read given

	 * the specifications by L1; It's not enough to take

	 * vmcs12->cr0_read_shadow because on our cr0_guest_host_mask we

	 * have more bits than L1 expected.

	 * Override the CR0/CR4 read shadows after setting the effective guest

	 * CR0/CR4.  The common helpers also set the shadows, but they don't

	 * account for vmcs12's cr0/4_guest_host_mask.

	 */

	vmx_set_cr0(vcpu, vmcs12->guest_cr0);

	vmcs_writel(CR0_READ_SHADOW, nested_read_cr0(vmcs12));

	vmx_switch_vmcs(vcpu, &vmx->vmcs01);

	/*

	 * If IBRS is advertised to the vCPU, KVM must flush the indirect

	 * branch predictors when transitioning from L2 to L1, as L1 expects

	 * hardware (KVM in this case) to provide separate predictor modes.

	 * Bare metal isolates VMX root (host) from VMX non-root (guest), but

	 * doesn't isolate different VMCSs, i.e. in this case, doesn't provide

	 * separate modes for L2 vs L1.

	 */

	if (guest_cpuid_has(vcpu, X86_FEATURE_SPEC_CTRL))

		indirect_branch_prediction_barrier();

	/* Update any VMCS fields that might have changed while L2 ran */

	vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, vmx->msr_autoload.host.nr);

	vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, vmx->msr_autoload.guest.nr);

		| FEAT_CTL_VMX_ENABLED_OUTSIDE_SMX;

	/*

	 * Note, KVM cannot rely on hardware to perform the CR0/CR4 #UD checks

	 * that have higher priority than VM-Exit (see Intel SDM's pseudocode

	 * for VMXON), as KVM must load valid CR0/CR4 values into hardware while

	 * running the guest, i.e. KVM needs to check the _guest_ values.

	 * Manually check CR4.VMXE checks, KVM must force CR4.VMXE=1 to enter

	 * the guest and so cannot rely on hardware to perform the check,

	 * which has higher priority than VM-Exit (see Intel SDM's pseudocode

	 * for VMXON).

	 *

	 * Rely on hardware for the other two pre-VM-Exit checks, !VM86 and

	 * !COMPATIBILITY modes.  KVM may run the guest in VM86 to emulate Real

	 * Mode, but KVM will never take the guest out of those modes.

	 * Rely on hardware for the other pre-VM-Exit checks, CR0.PE=1, !VM86

	 * and !COMPATIBILITY modes.  For an unrestricted guest, KVM doesn't

	 * force any of the relevant guest state.  For a restricted guest, KVM

	 * does force CR0.PE=1, but only to also force VM86 in order to emulate

	 * Real Mode, and so there's no need to check CR0.PE manually.

	 */

	if (!nested_host_cr0_valid(vcpu, kvm_read_cr0(vcpu)) ||

	    !nested_host_cr4_valid(vcpu, kvm_read_cr4(vcpu))) {

	if (!kvm_read_cr4_bits(vcpu, X86_CR4_VMXE)) {

		kvm_queue_exception(vcpu, UD_VECTOR);

		return 1;

	}

	/*

	 * CPL=0 and all other checks that are lower priority than VM-Exit must

	 * be checked manually.

	 * The CPL is checked for "not in VMX operation" and for "in VMX root",

	 * and has higher priority than the VM-Fail due to being post-VMXON,

	 * i.e. VMXON #GPs outside of VMX non-root if CPL!=0.  In VMX non-root,

	 * VMXON causes VM-Exit and KVM unconditionally forwards VMXON VM-Exits

	 * from L2 to L1, i.e. there's no need to check for the vCPU being in

	 * VMX non-root.

	 *

	 * Forwarding the VM-Exit unconditionally, i.e. without performing the

	 * #UD checks (see above), is functionally ok because KVM doesn't allow

	 * L1 to run L2 without CR4.VMXE=0, and because KVM never modifies L2's

	 * CR0 or CR4, i.e. it's L2's responsibility to emulate #UDs that are

	 * missed by hardware due to shadowing CR0 and/or CR4.

	 */

	if (vmx_get_cpl(vcpu)) {

		kvm_inject_gp(vcpu, 0);

	if (vmx->nested.vmxon)

		return nested_vmx_fail(vcpu, VMXERR_VMXON_IN_VMX_ROOT_OPERATION);

	/*

	 * Invalid CR0/CR4 generates #GP.  These checks are performed if and

	 * only if the vCPU isn't already in VMX operation, i.e. effectively

	 * have lower priority than the VM-Fail above.

	 */

	if (!nested_host_cr0_valid(vcpu, kvm_read_cr0(vcpu)) ||

	    !nested_host_cr4_valid(vcpu, kvm_read_cr4(vcpu))) {

		kvm_inject_gp(vcpu, 0);

		return 1;

	}

	if ((vmx->msr_ia32_feature_control & VMXON_NEEDED_FEATURES)

			!= VMXON_NEEDED_FEATURES) {

		kvm_inject_gp(vcpu, 0);

}

/*

 * Return the cr0 value that a nested guest would read. This is a combination

 * of the real cr0 used to run the guest (guest_cr0), and the bits shadowed by

 * its hypervisor (cr0_read_shadow).

 * Return the cr0/4 value that a nested guest would read. This is a combination

 * of L1's "real" cr0 used to run the guest (guest_cr0), and the bits shadowed

 * by the L1 hypervisor (cr0_read_shadow).  KVM must emulate CPU behavior as

 * the value+mask loaded into vmcs02 may not match the vmcs12 fields.

 */

static inline unsigned long nested_read_cr0(struct vmcs12 *fields)

{