Commit 946fbfdf authored by Oliver Upton's avatar Oliver Upton Committed by Marc Zyngier
Browse files

KVM: arm64: Make leaf->leaf PTE changes parallel-aware 



Convert stage2_map_walker_try_leaf() to use the new break-before-make
helpers, thereby making the handler parallel-aware. As before, avoid the
break-before-make if recreating the existing mapping. Additionally,
retry execution if another vCPU thread is modifying the same PTE.

Signed-off-by: default avatarOliver Upton <oliver.upton@linux.dev>
Reviewed-by: default avatarBen Gardon <bgardon@google.com>
Signed-off-by: default avatarMarc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20221107215934.1895478-1-oliver.upton@linux.dev
parent 0ab12f35

arch/arm64/kvm/hyp/pgtable.c

+12 −14
Original line number Diff line number Diff line
@@ -804,7 +804,6 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx, 
	else

		new = kvm_init_invalid_leaf_owner(data->owner_id);



	if (stage2_pte_is_counted(ctx->old)) {

	/*

	 * Skip updating the PTE if we are trying to recreate the exact

	 * same mapping or only change the access permissions. Instead,
@@ -814,8 +813,8 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx, 
	if (!stage2_pte_needs_update(ctx->old, new))

		return -EAGAIN;



		stage2_put_pte(ctx, data->mmu, mm_ops);

	}

	if (!stage2_try_break_pte(ctx, data->mmu))

		return -EAGAIN;



	/* Perform CMOs before installation of the guest stage-2 PTE */

	if (mm_ops->dcache_clean_inval_poc && stage2_pte_cacheable(pgt, new))
@@ -825,9 +824,8 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx, 
	if (mm_ops->icache_inval_pou && stage2_pte_executable(new))

		mm_ops->icache_inval_pou(kvm_pte_follow(new, mm_ops), granule);



	smp_store_release(ctx->ptep, new);

	if (stage2_pte_is_counted(new))

		mm_ops->get_page(ctx->ptep);

	stage2_make_pte(ctx, new);



	if (kvm_phys_is_valid(phys))

		data->phys += granule;

	return 0;