kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-26 19:14:18 +00:00
Code Issues Releases Packages Activity

KVM x86/mmu changes for 6.15

Browse source 
Add support for "fast" aging of SPTEs in both the TDP MMU and Shadow MMU, where
 "fast" means "without holding mmu_lock".  Not taking mmu_lock allows multiple
 aging actions to run in parallel, and more importantly avoids stalling vCPUs,
 e.g. due to holding mmu_lock for an extended duration while a vCPU is faulting
 in memory.
 
 For the TDP MMU, protect aging via RCU; the page tables are RCU-protected and
 KVM doesn't need to access any metadata to age SPTEs.
 
 For the Shadow MMU, use bit 1 of rmap pointers (bit 0 is used to terminate a
 list of rmaps) to implement a per-rmap single-bit spinlock.  When aging a gfn,
 acquire the rmap's spinlock with read-only permissions, which allows hardening
 and optimizing the locking and aging, e.g. locking an rmap for write requires
 mmu_lock to also be held.  The lock is NOT a true R/W spinlock, i.e. multiple
 concurrent readers aren't supported.
 
 To avoid forcing all SPTE updates to use atomic operations (clearing the
 Accessed bit out of mmu_lock makes it inherently volatile), rework and rename
 spte_has_volatile_bits() to spte_needs_atomic_update() and deliberately exclude
 the Accessed bit.  KVM (and mm/) already tolerates false positives/negatives
 for Accessed information, and all testing has shown that reducing the latency
 of aging is far more beneficial to overall system performance than providing
 "perfect" young/old information.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmfZizUACgkQOlYIJqCj
 N/1XkBAAwLxK3uKwkIIgzu+V6NPMiqPBsNtGiiQgRYMfvCwMW2vU6ztsBgNgs6zI
 eMnOCCo6fQaxPFvpKue8VN7TD33BcjzKZaPiuHZrzQIa/oYQeOlZ4oaN8lr9F9Ec
 5l1Lg/p2+z1GGDhWc2opNpg48sCtX7IQ0Tx46LkoB3VSDFP33GwW4+Ht2r71rNeL
 ofKB+T0hU5HOry5j0w0nTVwOEoNzlm1sVFqOHzgnK18Lmqw2CfOPm+46K+w8nOh+
 v+rwGuGa//1kcCjNCcGP1OuJdNAMgXBxND/l6LAkWcHfffIRbXlO07O05dAGqPeF
 rRn5JUl02OkI6lq99+935OmtEROe6mt+Bx0dhAzk4Z0CD6JY34ShZSAADSnltQlK
 2a1E95t63v8a7ZM5dwED7os2HBhxODoyeWQAlIHpkVdmeTJstkyvjPhubJc13+Js
 oDL6ehs3hhZ171ePn2aXo0NobA5fe7xbl4wugP3hNmBXjLvu04D+llcDmC095nBk
 ICuzFqFXCXzdjEwgWwPzTseWOCoWTkoRqeJ9lch4UD3mMMcmK0MbK6joocGvCFto
 cL/eZdElnf1MZwWYdo44X+NEc1jItZVvktkRrllpwCtpRSDINO6RYZGcRf/g0Lha
 XmaU7jICfi3AKc4N3S2l4KIkd/AeJQySM+kGArxIOYoaqFCe2Mc=
 =Iy57
 -----END PGP SIGNATURE-----

Merge tag 'kvm-x86-mmu-6.15' of https://github.com/kvm-x86/linux into HEAD

KVM x86/mmu changes for 6.15

Add support for "fast" aging of SPTEs in both the TDP MMU and Shadow MMU, where
"fast" means "without holding mmu_lock".  Not taking mmu_lock allows multiple
aging actions to run in parallel, and more importantly avoids stalling vCPUs,
e.g. due to holding mmu_lock for an extended duration while a vCPU is faulting
in memory.

For the TDP MMU, protect aging via RCU; the page tables are RCU-protected and
KVM doesn't need to access any metadata to age SPTEs.

For the Shadow MMU, use bit 1 of rmap pointers (bit 0 is used to terminate a
list of rmaps) to implement a per-rmap single-bit spinlock.  When aging a gfn,
acquire the rmap's spinlock with read-only permissions, which allows hardening
and optimizing the locking and aging, e.g. locking an rmap for write requires
mmu_lock to also be held.  The lock is NOT a true R/W spinlock, i.e. multiple
concurrent readers aren't supported.

To avoid forcing all SPTE updates to use atomic operations (clearing the
Accessed bit out of mmu_lock makes it inherently volatile), rework and rename
spte_has_volatile_bits() to spte_needs_atomic_update() and deliberately exclude
the Accessed bit.  KVM (and mm/) already tolerates false positives/negatives
for Accessed information, and all testing has shown that reducing the latency
of aging is far more beneficial to overall system performance than providing
"perfect" young/old information.
This commit is contained in:
Paolo Bonzini 2025-03-19 09:04:33 -04:00
commit 4286a3ec25
11 changed files with 375 additions and 171 deletions
Download patch file Download diff file Expand all files Collapse all files