From 62ef67af1878fa2cd066642f2f59e33ade95f637 Mon Sep 17 00:00:00 2001 From: Sean Christopherson Date: Sun, 23 Aug 2026 22:07:51 +0900 Subject: [PATCH] KVM: x86: Check for invalid/obsolete root *after* making MMU pages available [ Upstream commit 2abd5287f08319fa35764566b15c6e22cb1068db ] Check for a "stale" page fault, i.e. for an invalid and/or obsolete root, after making MMU pages available for the shadow MMU. If reclaiming shadow pages zaps an in-use root, i.e. marks it invalid, then KVM will attempt to map memory into an invalid root. On its own, populating an invalid root is "fine", but because child shadow pages inherit their parent's role, any children created during the map/fetch will be created as invalid pages, thus violating KVM's invariant that invalid pages are never on the list of active MMU pages. Note, the underlying flaw has existed since KVM first started tracking invalid roots in 2008 (commit 2e53d63acba7, "KVM: MMU: ignore zapped root pagetables"), but the true badness only came along in 2020 (Linux 5.9) with the invariant that invalid shadow pages can't be on the list of active pages. Note #2, inheriting role.invalid when creating child shadow pages is also far from ideal; that flaw will be addressed separately. Reported-by: Hyunwoo Kim Fixes: f95eec9bed76 ("KVM: x86/mmu: Don't put invalid SPs back on the list of active pages") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Signed-off-by: Paolo Bonzini [backport note: upstream passes struct kvm_page_fault to is_page_fault_stale(); here individual parameters are used, as is_page_fault_stale() was backported with that interface in the preceding commit] Signed-off-by: Kenta Akagi Signed-off-by: Greg Kroah-Hartman --- arch/x86/kvm/mmu/mmu.c | 9 +++++---- arch/x86/kvm/mmu/paging_tmpl.h | 10 ++++++---- 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 019e24a76a96..8c381cf61e21 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4045,16 +4045,17 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, if (handle_abnormal_pfn(vcpu, is_tdp ? 0 : gpa, gfn, pfn, ACC_ALL, &r)) return r; - r = RET_PF_RETRY; write_lock(&vcpu->kvm->mmu_lock); - if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva)) - goto out_unlock; - r = make_mmu_pages_available(vcpu); if (r) goto out_unlock; + if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva)) { + r = RET_PF_RETRY; + goto out_unlock; + } + r = direct_map(vcpu, gpa, error_code, map_writable, max_level, pfn, prefault, is_tdp); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index c67b51bc666e..fd3f2c5bb7ed 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -923,16 +923,18 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code, walker.pte_access &= ~ACC_EXEC_MASK; } - r = RET_PF_RETRY; write_lock(&vcpu->kvm->mmu_lock); - if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva)) - goto out_unlock; - kvm_mmu_audit(vcpu, AUDIT_PRE_PAGE_FAULT); r = make_mmu_pages_available(vcpu); if (r) goto out_unlock; + + if (is_page_fault_stale(vcpu, pfn, mmu_seq, hva)) { + r = RET_PF_RETRY; + goto out_unlock; + } + r = FNAME(fetch)(vcpu, addr, &walker, error_code, max_level, pfn, map_writable, prefault); kvm_mmu_audit(vcpu, AUDIT_POST_PAGE_FAULT);