KVM: x86/mmu: Make Host-writable and MMU-writable bit locations dynamic

2. Write-Protection: The SPTE is present and the fault is caused by

   write-protect. That means we just need to change the W bit of the spte.

What we use to avoid all the race is the SPTE_HOST_WRITEABLE bit and

SPTE_MMU_WRITEABLE bit on the spte:

What we use to avoid all the race is the Host-writable bit and MMU-writable bit

on the spte:

- SPTE_HOST_WRITEABLE means the gfn is writable on host.

- SPTE_MMU_WRITEABLE means the gfn is writable on mmu. The bit is set when

  the gfn is writable on guest mmu and it is not write-protected by shadow

  page write-protection.

- Host-writable means the gfn is writable in the host kernel page tables and in

  its KVM memslot.

- MMU-writable means the gfn is writable in the guest's mmu and it is not

  write-protected by shadow page write-protection.

On fast page fault path, we will use cmpxchg to atomically set the spte W

bit if spte.SPTE_HOST_WRITEABLE = 1 and spte.SPTE_WRITE_PROTECT = 1, to

restore the saved R/X bits if for an access-traced spte, or both. This is

safe because whenever changing these bits can be detected by cmpxchg.

bit if spte.HOST_WRITEABLE = 1 and spte.WRITE_PROTECT = 1, to restore the saved

R/X bits if for an access-traced spte, or both. This is safe because whenever

changing these bits can be detected by cmpxchg.

But we need carefully check these cases:

 * write-protects guest page to sync the guest modification, b) another one is

 * used to sync dirty bitmap when we do KVM_GET_DIRTY_LOG. The differences

 * between these two sorts are:

 * 1) the first case clears SPTE_MMU_WRITEABLE bit.

 * 1) the first case clears MMU-writable bit.

 * 2) the first case requires flushing tlb immediately avoiding corrupting

 *    shadow page table between all vcpus so it should be in the protection of

 *    mmu-lock. And the another case does not need to flush tlb until returning

 * So, there is the problem: the first case can meet the corrupted tlb caused

 * by another case which write-protects pages but without flush tlb

 * immediately. In order to making the first case be aware this problem we let

 * it flush tlb if we try to write-protect a spte whose SPTE_MMU_WRITEABLE bit

 * is set, it works since another case never touches SPTE_MMU_WRITEABLE bit.

 * it flush tlb if we try to write-protect a spte whose MMU-writable bit

 * is set, it works since another case never touches MMU-writable bit.

 *

 * Anyway, whenever a spte is updated (only permission and status bits are

 * changed) we need to check whether the spte with SPTE_MMU_WRITEABLE becomes

 * changed) we need to check whether the spte with MMU-writable becomes

 * readonly, if that happens, we need to flush tlb. Fortunately,

 * mmu_spte_update() has already handled it perfectly.

 *

 * The rules to use SPTE_MMU_WRITEABLE and PT_WRITABLE_MASK:

 * The rules to use MMU-writable and PT_WRITABLE_MASK:

 * - if we want to see if it has writable tlb entry or if the spte can be

 *   writable on the mmu mapping, check SPTE_MMU_WRITEABLE, this is the most

 *   writable on the mmu mapping, check MMU-writable, this is the most

 *   case, otherwise

 * - if we fix page fault on the spte or do write-protection by dirty logging,

 *   check PT_WRITABLE_MASK.

	rmap_printk("spte %p %llx\n", sptep, *sptep);

	if (pt_protect)

		spte &= ~SPTE_MMU_WRITEABLE;

		spte &= ~shadow_mmu_writable_mask;

	spte = spte & ~PT_WRITABLE_MASK;

	return mmu_spte_update(sptep, spte);

	 * spte from present to present (changing the spte from present

	 * to nonpresent will flush all the TLBs immediately), in other

	 * words, the only case we care is mmu_spte_update() where we

	 * have checked SPTE_HOST_WRITEABLE | SPTE_MMU_WRITEABLE

	 * instead of PT_WRITABLE_MASK, that means it does not depend

	 * on PT_WRITABLE_MASK anymore.

	 * have checked Host-writable | MMU-writable instead of

	 * PT_WRITABLE_MASK, that means it does not depend on PT_WRITABLE_MASK

	 * anymore.

	 */

	if (flush)

		kvm_arch_flush_remote_tlbs_memslot(kvm, memslot);

		nr_present++;

		host_writable = sp->spt[i] & SPTE_HOST_WRITEABLE;

		host_writable = sp->spt[i] & shadow_host_writable_mask;

		set_spte_ret |= set_spte(vcpu, &sp->spt[i],

					 pte_access, PG_LEVEL_4K,

static bool __read_mostly enable_mmio_caching = true;

module_param_named(mmio_caching, enable_mmio_caching, bool, 0444);

u64 __read_mostly shadow_host_writable_mask;

u64 __read_mostly shadow_mmu_writable_mask;

u64 __read_mostly shadow_nx_mask;

u64 __read_mostly shadow_x_mask; /* mutual exclusive with nx_mask */

u64 __read_mostly shadow_user_mask;

			kvm_is_mmio_pfn(pfn));

	if (host_writable)

		spte |= SPTE_HOST_WRITEABLE;

		spte |= shadow_host_writable_mask;

	else

		pte_access &= ~ACC_WRITE_MASK;

	spte |= (u64)pfn << PAGE_SHIFT;

	if (pte_access & ACC_WRITE_MASK) {

		spte |= PT_WRITABLE_MASK | SPTE_MMU_WRITEABLE;

		spte |= PT_WRITABLE_MASK | shadow_mmu_writable_mask;

		/*

		 * Optimization: for pte sync, if spte was writable the hash

				 __func__, gfn);

			ret |= SET_SPTE_WRITE_PROTECTED_PT;

			pte_access &= ~ACC_WRITE_MASK;

			spte &= ~(PT_WRITABLE_MASK | SPTE_MMU_WRITEABLE);

			spte &= ~(PT_WRITABLE_MASK | shadow_mmu_writable_mask);

		}

	}

	new_spte |= (u64)new_pfn << PAGE_SHIFT;

	new_spte &= ~PT_WRITABLE_MASK;

	new_spte &= ~SPTE_HOST_WRITEABLE;

	new_spte &= ~shadow_host_writable_mask;

	new_spte = mark_spte_for_access_track(new_spte);

	shadow_acc_track_mask	= 0;

	shadow_me_mask		= sme_me_mask;

	shadow_host_writable_mask = DEFAULT_SPTE_HOST_WRITEABLE;

	shadow_mmu_writable_mask  = DEFAULT_SPTE_MMU_WRITEABLE;

	/*

	 * Set a reserved PA bit in MMIO SPTEs to generate page faults with

	 * PFEC.RSVD=1 on MMIO accesses.  64-bit PTEs (PAE, x86-64, and EPT