If a vCPU stays scheduled out (or blocked) while the last pCPU it ran
on goes through a hotplug cycle (online->offline->online), and the vCPU
then resumes execution on the same pCPU, then it is possible for it to
run with an ASID that has now been assigned to a different vCPU,
resulting in stale TLB translations being used.
svm_enable_virtualization_cpu() resets asid_generation to 1 and sets
next_asid to max_asid + 1 on every CPU online event, including hotplug
cycles. Because next_asid starts beyond the pool boundary, the first
call to new_asid() after an online event always wraps the pool,
incrementing asid_generation to 2 and assigning ASIDs starting from
min_asid.
Consider two vCPUs from different VMs, vCPU-A pinned to CPU-X holding
asid_generation=2 and ASID=N from before the hotplug event:
1. CPU-X goes offline and back online: asid_generation resets to 1,
next_asid = max_asid + 1.
2. One or more vCPUs migrate to CPU-X and call new_asid(), wrapping
the pool and consuming ASIDs starting from min_asid. Eventually
vCPU-B from a different VM is assigned asid_generation=2, ASID=N
— the same ASID that vCPU-A held before the hotplug.
3. vCPU-A enters pre_svm_run() on CPU-X: current_vmcb->cpu is
unchanged so the migration branch is skipped. Its saved
asid_generation=2 matches sd->asid_generation=2, so the generation
check silently passes and vCPU-A continues running with ASID=N —
the same ASID just freshly assigned to vCPU-B.
Both vCPUs from different VMs now run on CPU-X with the same ASID,
causing them to share NPT TLB entries and producing stale translations.
The collision manifests as a KVM internal error (Suberror: 1, emulation
failure). The NPT page fault reports a faulting GPA far outside the
VM's physical memory range — a sign of stale TLB translations being
used. KVM falls back to instruction emulation, which fails on
FPU/XSave instructions (XRSTOR, STMXCSR) that the emulator does not
implement.
Fix this by incrementing asid_generation instead of resetting it to 1
in svm_enable_virtualization_cpu(). On module load, asid_generation
starts at 0 (memset) and the increment produces 1, identical to the
old behaviour. On subsequent hotplug cycles the generation advances
beyond any value a vCPU previously observed on this CPU, so the
generation check in pre_svm_run() reliably forces new_asid() on every
vCPU after every hotplug cycle.
Fixes: 774c47f1d7 ("[PATCH] KVM: cpu hotplug support")
Reported-by: Chandrakanth Silveru <Chandrakanth.Silveru@amd.com>
Tested-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Message-ID: <20260715063506.672432-1-nikunj@amd.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The intra-host migration/mirroring feature is not fully implemented for
SEV-SNP VMs. The proper migration requires additional SNP-specific
state such as guest_req_mutex, guest_req_buf, and guest_resp_buf to be
transferred or initialized on the destination.
The SNP VM mirroring requires vmsa features to be copied as well otherwise
ASID would be bound to SNP range while VM is detected as a SEV VM.
Reject SNP source VMs in migration/mirroring until proper SNP state
transfer is implemented.
Fixes: 1dfe571c12 ("KVM: SEV: Add initial SEV-SNP support")
Reported-by: Chris Mason <clm@meta.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Atish Patra <atishp@meta.com>
Link: https://patch.msgid.link/20260602-sev_snp_fixes-v3-1-24bfd3ae047c@meta.com
Cc: stable@vger.kernel.org
[sean: let lines poke past 80 chars, tag for stable]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Replace the #include of <linux/mod_devicetable.h> by the more specific
<linux/device-id/*.h> where applicable. For most cases the include
can be dropped completely, only a few drivers need one or two headers
added.
Acked-by: Danilo Krummrich <dakr@kernel.org>
Acked-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://patch.msgid.link/1a3f2007c5c5dcf555c09a4035ce3ae8ef1b6c49.1782808461.git.u.kleine-koenig@baylibre.com
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
* Fix S390_USER_OPEREXEC so it can now be enabled regardless of other
unrelated capabilities
* Fix handling of the _PAGE_UNUSED pte bit that could lead to guest
memory corruption in some scenarios
* A bunch of misc gmap fixes (locking, behaviour under memory pressure)
* Fix CMMA dirty tracking
x86:
* Tidy up some WARN_ON() and BUG_ON(), replacing them with WARN_ON_ONCE()
or KVM_BUG_ON(). All of these have obviously never triggered, or
somebody would have been annoyed earlier, but still.
* Fix missing interrupt due to stale CR8 intercept
* Add a statistic that can come in handy to debug leaks as well as the
vulnerability to a class of recently-discovered issues.
* Do not ask arch/x86/kernel to export default_cpu_present_to_apicid()
just for KVM.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmo7xaAUHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroN7bwf/Ua6PrRrgdzfUGiIruBDnFGtXnaKf
UPGpgl35Kl4ntPlmH6wRG6N0jP9sloa17Qj/Y33O1Um3Mi/eDV0UESW8o8tfFE2J
CWUPc3/adw2urhnkXsoleSFy21a89TgLd0p4tot832p+IHi3iNJ45fx7XxftBeGt
fa/liscncEX5tGwie0ZVkIEE/ob3+eX4ZYsTLXMi6FxAlyGPYNrp4jwJeYH9AFSd
ZYGqVbDjDq14vhnppKXM8DMfHTXN4OVYw2RJZ9Y7u2h7ku3sN0XakzVTAUz5gtoM
h+iu8dn5yslODBZiNGlls8UYm2bweOBCumH1ZITe/eTNuEdUOJGILorKRw==
=QzfV
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm fixes from Paolo Bonzini:
"s390:
- Fix S390_USER_OPEREXEC so it can now be enabled regardless of other
unrelated capabilities
- Fix handling of the _PAGE_UNUSED pte bit that could lead to guest
memory corruption in some scenarios
- A bunch of misc gmap fixes (locking, behaviour under memory
pressure)
- Fix CMMA dirty tracking
x86:
- Tidy up some WARN_ON() and BUG_ON(), replacing them with
WARN_ON_ONCE() or KVM_BUG_ON(). All of these have obviously never
triggered, or somebody would have been annoyed earlier, but still...
- Fix missing interrupt due to stale CR8 intercept
- Add a statistic that can come in handy to debug leaks as well as
the vulnerability to a class of recently-discovered issues
- Do not ask arch/x86/kernel to export
default_cpu_present_to_apicid() just for KVM"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (22 commits)
x86/apic: KVM: Use cpu_physical_id() to get APIC ID of running vCPU for AVIC
KVM: x86/mmu: Expose number of shadow MMU shadow pages as a stat
KVM: x86: Unconditionally recompute CR8 intercept on PPR update
KVM: VMX: Grab vmcs12 on CR8 interception update iff vCPU is in guest mode
KVM: x86: WARN (once) if RTC pending EOI tracking goes off the rails
KVM: x86: WARN and fail kvm_set_irq() if a PIC or I/O APIC vector is invalid
KVM: x86: Bug the VM, not the kernel, if the ISR count {under,over}flows
KVM: x86/mmu: Bug the VM, not the host kernel, if KVM write-protects upper SPTEs
KVM: x86: Replace BUG_ON() with WARN_ON_ONCE() on "bad" nested GPA translation
KVM: Replace guest-triggerable BUG_ON() in ioeventfd datamatch with get_unaligned()
KVM: s390: Return failure in case of failure in kvm_s390_set_cmma_bits()
KVM: s390: selftests: Fix cmma selftest
KVM: s390: Fix cmma dirty tracking
KVM: s390: Fix locking in kvm_s390_set_mem_control()
KVM: s390: Fix handle_{sske,pfmf} under memory pressure
KVM: s390: Fix code typo in gmap_protect_asce_top_level()
KVM: s390: Do not set special large pages dirty
KVM: s390: Fix dat_peek_cmma() overflow
s390/mm: Fix handling of _PAGE_UNUSED pte bit
KVM: s390: Fix typo in UCONTROL documentation
...
Use cpu_physical_id() instead of default_cpu_present_to_apicid() when
getting the APIC ID of the pCPU on which a vCPU is running/loaded, as the
kernel has gone way off the rails if a vCPU is loaded on a pCPU that has
been physically removed from the system. Even if the impossible were to
happen, the absolutely worst case scenario is that hardware will ring the
AIVC doorbell on the wrong pCPU, i.e. a severely broken system will
experience mild performance issues.
Kill off KVM's superfluous kvm_cpu_get_apicid() wrapper along with the
for-KVM export of default_cpu_present_to_apicid(), as they existed purely
for the wonky AVIC usage.
Cc: Kai Huang <kai.huang@intel.com>
Cc: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Acked-by: Naveen N Rao (AMD) <naveen@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Message-ID: <20260612185459.591892-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
If KVM attempts to translate what it thinks is an L2 GPA with a non-nested
MMU, simply WARN and return the GPA, i.e. trust the MMU more than the
caller, as there is zero reason to potentially panic the host kernel just
because KVM misused an API.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20260618185746.2023283-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
"This is a bit of an odd merge window on the KVM/arm64 front. There is
absolutely no new feature in the pull request. It is purely fixes,
because it is simply becoming too hard to review new stuff when so
many AI-fuelled fixes hit the list".
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX)
that caused the FPU load request.
- Some enhancements about interrupt injection.
- Some bug fixes and other small changes.
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic
PTE updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on
a memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes.
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace.
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root.
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both
of these are preparatory changes for upcoming APX support.
- Clean up KVM's register tracking and storage, primarily to prepare for
APX support, which expands the maximum number of GPRs from 16 to 32.
- Keep a single copy of the PDPTRs rather than two, since architecturally
there is just one.
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in shadow page table handling, where KVM fails to recursively
zap nested EPT/NPT shadow page tables when the nested hypervisor tears
down its own EPT/NPT page tables from the bottom up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in the
generic section; also run with MBEC enabled even for non-nested mode.
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Fix a largely benign bug where KVM TDX would incorrectly state it could
emulate several x2APIC MSRs.
- Use the "safe" WRMSR API when proxying LBR MSR writes as the to-be-written
value is guest controlled and completely unvalidated.
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in the
generic section; also run with GMET enabled even for non-nested mode.
- Fixes and minor cleanups to GHCB handling, on top of the earlier work
already merged into 7.1-rc.
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers.
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
- Don't advertise support for unusable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to follow
the kernel's nomenclature, e.g. for mmu_notifiers.
- Use guard() to cleanup up various KVM+VFIO flows.
- Minor cleanups.
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a gmem
range to multiple memslots, and fix the test that was supposed to ensure
KVM returns -EEXIST.
- Treat memslot binding offsets and sizes as unsigned values to fix a bug
where KVM interprets a large "offset + size" as a negative value and allows
a nonsensical offset.
- Use the inode number instead of the page offset for the NUMA interleaving
index to fix a bug where the effective index would jump by two for
consecutive pages (the caller also adds in the page offset).
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the first
pass, as always waiting only 1ms hid a KVM RISC-V bug as the test reaped the
bitmap before KVM could build up enough state to hit the bug.
- A pile of one-off fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmoypI0UHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroNCSAf8CyuvicojnZGFsNcoY4DZ0whBj5l8
IwVkHaptuyGLSwPAebC/DwtU2NBY4CQdbtLjGmxenf+fSQOtcz1YWKbdY4Nr0hJG
euAb/wO47RqKp1C4WPi7lL2P4XHgmUgD5XZ1qDnRV9gnGddZi3QLLmKgAtNq9Z4m
+pAdGD2/RyZeI0Dx61+5ufE/z8YBlH6GrzgBTxfvvRNKqNDFykBenxYHI7seEfHC
BPz0LxjHDD8BMDUo86qaylZnaeQSmFTGO/eTgvCvabxG6WbM1s0w3K1owmUmP7o3
p9o8274Xh53iK2JYD/qGOtAJXejIwbYKT5bnvUQX0O1EH2+ouvSFapLQ9Q==
=sSqM
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
- CPUID API updates (Ahmed S. Darwish):
- Introduce a centralized CPUID parser
- Introduce a centralized CPUID data model
- Introduce <asm/cpuid/leaf_types.h>
- Rename cpuid_leaf()/cpuid_subleaf() APIs
- treewide: Explicitly include the x86 CPUID headers
- Update to x86-cpuid-db v3.1 (Maciej Wieczor-Retman)
- Continued removal of pre-i586 support and related
simplifications (Ingo Molnar)
- Add Intel CPU model number for rugged Panther Lake
(Tony Luck)
- Misc fixes, updates and cleanups by Arnd Bergmann, Chao Gao,
Lukas Bulwahn, Sohil Mehta, Maciej Wieczor-Retman.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmou1LcRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1h16Q/9GvxrQDX13qd7/XrjOKNYtoLIGh1GhvHM
2ZdbYh80LX36bDOg7Mhoy0bvElwmzaz2x1Zrb5SAIqpQXYqjfuRCe7a1SPQxYtCK
A6j1YeD/CJMm36jaQITYkuCYxwaw5LQ62u0ShpEvyzZzarEJt8c8COWJjbE57FN2
qusr+6K1sBwpEUl5mLabbJWXqhpPRYCz+nl2GF9BCHe6z7Nw7Q8VZf4w1weONJOy
lMpC2X08w5TE2f4OwvnRbZLelyROz6pr1c3osUoQSVtBkprO9TsYHgd9yBva1E2E
KxdWm+uSSUXM7cFBzY3RLBzKpG6iLtqircoDFBVdxOlT8I8KFggzbYy4JPlDEHQS
FqUwWL+2duoRciOncnZn9hydxlg5So4w7OscvTVNp94/Pb2pMdpOB+bsN05WTpXc
VMQVxrfsmwT1rV1oppzRizTMnb6yGg8BQvZPE+wrMDdH+/wwhzok4GdZzKVCzUoD
nDFhJL7t8ikRwwuq5RAl1EcT7qLFJx3ba77B5REf2Uos2liTExO/XzcmQnzSV1M1
RXpajQ1ikEBtJPAD3Zy0ASm/QPjB27o+6Kj6hW3NLlPoi5l0Lu6xQzH3wNrpQpst
02NCWEn2X00YUQt2ukNcDFJiMHJzXAnBQDBSrWwwd97wol3sKnT9RR2NZVFzn7ls
GOF8yJ0Wuec=
=tpai
-----END PGP SIGNATURE-----
Merge tag 'x86-cpu-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 cpuid updates from Ingo Molnar:
- CPUID API updates (Ahmed S. Darwish):
- Introduce a centralized CPUID parser
- Introduce a centralized CPUID data model
- Introduce <asm/cpuid/leaf_types.h>
- Rename cpuid_leaf()/cpuid_subleaf() APIs
- treewide: Explicitly include the x86 CPUID headers
- Update to x86-cpuid-db v3.1 (Maciej Wieczor-Retman)
- Continued removal of pre-i586 support and related simplifications
(Ingo Molnar)
- Add Intel CPU model number for rugged Panther Lake (Tony Luck)
- Misc fixes, updates and cleanups by Arnd Bergmann, Chao Gao, Lukas
Bulwahn, Sohil Mehta, Maciej Wieczor-Retman.
* tag 'x86-cpu-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (25 commits)
x86/cpu: Make CONFIG_X86_CX8 unconditional
x86/cpu: Remove unused !CONFIG_X86_TSC code
x86/cpuid: Update bitfields to x86-cpuid-db v3.1
tools/x86/kcpuid: Update bitfields to x86-cpuid-db v3.1
x86/cpu: Make CONFIG_X86_TSC unconditional
MAINTAINERS: Drop obsolete FPU EMULATOR section
x86/cpu: Fix a F00F bug warning and clean up surrounding code
x86/cpu: Add Intel CPU model number for rugged Panther Lake
x86/cpuid: Introduce a centralized CPUID parser
x86/cpu: Introduce a centralized CPUID data model
x86/cpuid: Introduce <asm/cpuid/leaf_types.h>
x86/cpuid: Rename cpuid_leaf()/cpuid_subleaf() APIs
x86/cpu: Do not include the CPUID API header in asm/processor.h
Documentation: core-api/cpu_hotplug: Remove stale cpu0_hotplug docs
x86/cpu, cpufreq: Remove AMD ELAN support
x86/fpu: Remove the math-emu/ FPU emulation library
x86/fpu: Remove the 'no387' boot option
x86/fpu: Remove MATH_EMULATION and related glue code
treewide: Explicitly include the x86 CPUID headers
x86/cpu: Remove the CONFIG_X86_INVD_BUG quirk
...
PDPTRs are part of the CPU state. A bit unconventionally, they are
reached via vcpu->arch.walk_mmu instead of being stored in vcpu->arch
directly. That is nice in principle---it would allow TDP shadow paging
to have its own PDPTRs---but it is not necessary, because EPT has no
PDPTRs and NPT does not cache them.
Since kvm_pdptr_read does not otherwise need the MMU, drop the pdptrs
from the MMU altogether. There is however something to be careful
about, in that PDPTRs are now not stored separately in root_mmu and
nested_mmu for L1 and L2 guests. In practice this was already not
an issue:
- for EPT the VMCS0x has to keep them up to date; and for the purpose
of emulation they are always loaded from the VMCS on vmentry/vmexit,
thanks to the clearing of dirty and available register bitmaps in
vmx_switch_vmcs()
- for NPT, VCPU_EXREG_PDPTR is similarly cleared for nNPT, which does
not cache the PDPTRs; while for non-nNPT the PDPTRs are loaded
together with the load of CR3.
Note that page table PDPTRs are not affected, since they are stored
in pae_root.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260530165545.25599-6-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When L2 runs under nested NPT and uses PAE paging, KVM's cached PDPTRs
in mmu->pdptrs[] can hold stale or wrong values after nested
transitions and across migration restore, because both
nested_svm_load_cr3() and svm_get_nested_state_pages() only refresh
PDPTRs on the !nested_npt path.
The user-visible bug is on migration restore of an L2 running with nested
NPT and 32-bit PAE paging, if userspace uses KVM_SET_SREGS rather than
KVM_SET_SREGS2. In that case, load_pdptrs() leaves VCPU_EXREG_PDPTR
marked as available, and kvm_pdptr_read() will use a stale translation
that used L1 GPAs instead of L2 nGPAs. svm_get_nested_state_pages()
runs on first KVM_RUN but skips the refresh because nested_npt_enabled()
is true. The CPU itself reads L2's PDPTRs correctly from memory via
L1's NPT, but KVM-side walking of guest PAE page tables uses the bogus
cached values.
Unlike Intel's GUEST_PDPTR0..3 fields in the VMCS, SVM has no
VMCB-cached PDPTR state: the in-memory PDPTEs at the current CR3 are
the only source of truth, and svm_cache_reg(VCPU_EXREG_PDPTR) simply
reloads them from memory via load_pdptrs(). Clearing the avail
bit (and the dirty bit because !avail/dirty is invalid) to force
a reload when PDPTRs as needed fixes the bug.
Do the same for nested_svm_load_cr3()'s nested_npt branch, so that
the invariant "PDPTRs need reloading" is handled similarly for both
immediate and deferred loading.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260530165545.25599-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorQ78ACgkQOlYIJqCj
N/3bCxAAoXU8uJ37PIm58P7ok5hF2OFXeDLCYE1BpOfYOaT4b0vWW+Rb5I3y+2K6
511IxlBE2RcGBg6LHb1d1a/IEvs0NpzXOZFp7NseJC8sIn01Lndgn2SvoLoMVRy8
Yda4WyCNV1rSeq+duZKtMRhXiNY5Zdgsz6S5MdnslAK9OCbjS2XsQ6F7afMwM1g0
D3/kVUdBwUrKrllgmJ+XprLvYGEVji8xVCosdJm1ZAxb0xIr395/Oy7tnx3O2XNz
NjpEqabFRjcL8JMiOPnYdTfdqsNagr7qSqunJkCMUtuq+ZBaAA332ZKRbR/Rh83c
XprJMMw3+t77AjMSOHGmQY9ykVugT29bG7mjN/5bqhqqSzS1zjpGKd7ToEqvpKBn
lCnbUoSUQkvfBMZr6WUQppZvNfZaNi2Nuq4EKkZzqOiKUMO6ggfIh/1+M7s93mmT
rCa/35JEoBL3uwJcc75c25Y3QZWQyJjuNmFOuDt4s+yzDJ70F6II8FMykC5LGlFn
m9oZ/KIFEY1H9r8GyjoCIksQI/GBPxAKULA8wfY4WsMr+NzzKIhB39FfR6FnyZTN
XmPRlAKvWCGAqnrbOwgWvdu8Gsj18SpU38+AhngZRS7cVqB4+tQEZ4tcp9oSz6dy
UMYsZD0N8TFgGNqRf34DmgB+NSIamttAAYhMPWgAoqdvseHSvp0=
=4LNF
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-svm-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM SVM changes for 7.2
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
- Don't advertise support for unusuable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorQawACgkQOlYIJqCj
N/1e8RAAhQWGpKAs6kcyd+WhJCPm5bQPtDMRVNM8piCdXdZ1jy/qO2phuxWWXtxu
7a5Kvmz5WODuUBJxdyzMLJJv+o8+8j/ReNA5XkdDBZrnvxZeQpm6yfjq7ZTfmgBx
qc6gFETNjhu8gvb2tIqAfOe87TtBb2Ruw3fMiJR8+ATKIN4bgLrqOcnczBxHo80X
RQZqM3CNjac+6q5Gc4YEK2RbKvdzl/QNEuMrpk25O9fPZ5rnFO6K6ojZQQ2tTdCB
G6tnwg5zlPGSi41l4fjvbWfaj09UQ1/zupXaXfq6aoj4og773G8GmXGEvh9Kd6Dc
GurSJMhjnOPY9kJJ1YLHcxX4HtSc0M9WLqV/uZB8PtmbKfotUMJsHz5bv7rJa98N
XQnt+9U08JhURJaKF12p+s0bmO/K7Dmp7VrUhCP1jeNuHsSdaoXlrfC/qZp1oLip
t0jSxwZELkCSNsgbNNzpTmED1+xfj9DhqY+FvzZN9HPBedZb2ZHCuWFXJ4DMhn5p
TiS2cL6eu8gG7vFXCoOYIHJbZoLNm823nqEneFbSJdy9ZSn55/UTYyFuG0I6VY1W
rfv52Vz7AuhR4WGDAAqz6fPX5otJsHKkS6Kqy9dqTcV2jDsLvu97bef3qgGj/pe+
5q+J7eeOGTtyJeQZwxljJ7Z+SOSB8vEkKPX6d6bqM7YXWKzn6oo=
=SlF7
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-sev-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM SEV changes for 7.2
- Don't advertise support for unusuable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Ensure KVM's copy of CR0 and CR3 are up-to-date on SVM prior to invoking
fastpath handlers.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Misc fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorPacACgkQOlYIJqCj
N/2f4BAAiWMEfbazgtGLYQyiSd2QtIw41Dpcz9cnSBxvuFztulewTpd5z2B4/8kX
dSeZ3vyofgOVUVIj4AlD6Ap8ZO7XL3FizLcjWv9c8l4j/sTREL1ix0oejPJpBASg
5EmSueBjmj+8VbBNxHGCYMuU4okR0UBmU86sC0wBuho+CvqkNegSrmCpntHUh4Kn
34/HZ6QTik7iZNXV7KVT7/2De/bZV/9RJ7T8EAOkOGRfWft7cVUmkCXBP184pqee
Z9X1gcymXb17Z73GbOFbM3SEewcYLiy1LdDyk1Tid3Wk6nJlMj0Tc0djsI6Vz7+8
UvaKNoAXyWydPzJmYuCuTP+OnrZrha3RuEZvpP9UBJqGQgNjzs1jis92jiPL3yf4
38zcNesukLU8RWKMHw4PYMQtDIVwqPloSPN9tPZ6U+9IWZx88H/IRMd080Jxsbem
UNCuB88PawtlYV7gNk2XvQ0gcLib3LTjVjrWyTloUDWTjsMEXYxQ7Ehn6Jk+dfya
BMjivWY246SqsMBmkj4Lj0S6uD25z9Yv3uPcwshZ3nuXCjdmBKlWNSs7cFYopI1t
h4urP9xX/eURs63f++nB+6ZsqMR2ESQXmbybvsevR/+GFb0Iuy3obJk4ab2kKvP/
OhFpo8AmJ3jOK2RrIbZ/wUb5rfZbvnCxjAqSQ1YZHIMkrD9vlTY=
=IHTg
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-misc-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM misc x86 changes for 7.2
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Ensure KVM's copy of CR0 and CR3 are up-to-date on SVM prior to invoking
fastpath handlers.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Misc fixes and cleanups.
Merge the final part of the GHCB 7.2 fixes at
https://lore.kernel.org/kvm/20260529183549.1104619-1-pbonzini@redhat.com/.
Patches 1-17 have already been included in Linux 7.1; these are minor
cleanups, and fixes for behaviors that are suboptimal or contradicting
the specification.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that only two case-statements actually need a local "ret" variable,
refactor sev_handle_vmgexit() to have all flows return directly when
possible, and bury "ret" as "r" in the two paths that need to propagate a
return value from a helper.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-25-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-25-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that sev_es_validate_vmgexit() is only responsible for checking that
all required GHCB fields are marked valid, turn it into a predicate whose
name reflects exactly that.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-24-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-24-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Handle unknown #VMGEXIT reasons in sev_handle_vmgexit(), not in
sev_es_validate_vmgexit(). This makes it _much_ more obvious that KVM
simply funnels "legacy" exits to the standard SVM interception handlers,
and is the final preparatory change needed to reduce the scope of
sev_es_validate_vmgexit().
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-23-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-23-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Return INVALID_INPUT, not MISSING_INPUT, if the guest provides an unaligned
address for a GUEST_REQUEST, and/or attempts to use the same page for the
source and destination. The inputs are obviously invalid, not missing.
Opportunistically move the checks out of sev_es_validate_vmgexit(), to
continue the march towards reducing the scope of the helper, and to help
guide future changes into correctly handling bad input.
Fixes: 88caf544c9 ("KVM: SEV: Provide support for SNP_GUEST_REQUEST NAE event")
Fixes: 74458e4859 ("KVM: SEV: Provide support for SNP_EXTENDED_GUEST_REQUEST NAE event")
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-22-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-22-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signal INVALID_EVENT, not MISSING_INPUT, if a non-SNP guest attempts to
invoke an SNP-only #VMGEXIT. Opportunistically move the checks out of
sev_es_validate_vmgexit() to continue the march towards making said helper
a predicate whose sole purpose is to verify the guest has marked required
GHCB fields as valid.
Fixes: e366f92ea9 ("KVM: SEV: Support SEV-SNP AP Creation NAE event")
Fixes: 9b54e248d2 ("KVM: SEV: Add support to handle Page State Change VMGEXIT")
Fixes: 88caf544c9 ("KVM: SEV: Provide support for SNP_GUEST_REQUEST NAE event")
Fixes: 74458e4859 ("KVM: SEV: Provide support for SNP_EXTENDED_GUEST_REQUEST NAE event")
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-21-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-21-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the check to verify the guest's requested GHCB out of
sev_es_validate_vmgexit() as the first step towards making said helper a
predicate whose sole purpose is to verify the guest has marked required
GHCB fields as valid.
Using a single "validate" helper sounds good on paper, but in practice it's
difficult to verify that KVM is performing the necessary sanity checks (the
usage of state is far removed from the relevant checks), makes it difficult
to understand that "legacy" exits are simply routed to KVM's existing exit
handlers, and most importantly, has directly contributed to a number of
bugs as adding case-statements to the validation subtly removes them from
the default path that rejects unknown exit codes with INVALID_EVENT.
Deliberately extract the usage code check first so as to preserve the order
of KVM's checks, even though future code extraction will technically fix
bugs.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-20-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-20-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
If the guest attempts a non-MSR #VMGEXIT without the registered GHCB,
return a GHCB_HV_RESP_MALFORMED_INPUT+GHCB_ERR_NOT_REGISTERED error to the
guest instead of exiting KVM_RUN with -EINVAL (and in likelihood killing
the VM). KVM has already mapped the requested GHCB, i.e. can cleanly
report an error, and so exiting with -EINVAL is completely unjustified.
Fixes: 0c76b1d082 ("KVM: SEV: Add support to handle GHCB GPA register VMGEXIT")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-19-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-19-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Unmap and unpin the GHCB as needed when freeing a vCPU. If the VM is
destroyed after mapping+pinning the GHCB on #VMGEXIT, without re-running
the vCPU, KVM will effectively leak the GHCB and any mappings created for
the GHCB.
Fixes: 291bd20d5d ("KVM: SVM: Add initial support for a VMGEXIT VMEXIT")
Cc: stable@vger.kernel.org
Tested-by: Michael Roth <michael.roth@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-18-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-18-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Decouple synchronizing the GHCB SA from freeing/unpinning the SA, so that
the free/unpin path can be reused when freeing a vCPU.
Opportunistically add a WARN to harden KVM against stomping over (and thus
leaking) an already-allocated scratch area.
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-17-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-17-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Relocate sev_free_vcpu() down in sev.c so that it's definition comes after
sev_es_unmap_ghcb(). This will allow sharing unmap functionality between
the two functions without needing a forward declaration (or weird placement
of the common code).
No functional change intended.
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-16-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-16-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Rework Page State Change (PSC) handling to read the guest-provided start
and end indices exactly once, at the beginning of the request. Re-reading
the indices is "fine", _if_ the guest is well-behaved. KVM _should_ be
safe against concurrent guest modification of the indices, but there is
zero reason to introduce unnecessary risk.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-14-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-14-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Add a "psc" struct to vcpu_sev_es_state to avoid having to prefix all of
the fields with "psc_".
Take advantage of the code churn to opportunistically rename local
variables to "guest_psc" to make it more obvious that the buffer is guest
data, and more importantly, guest accessible!
Opportunistically rename inflight => batch_size as well, because there can
really only be one operation in-flight (per-vCPU), i.e. "inflight" _looks_
like a boolean, but in actuality is an integer tracking how many pages are
being handled by the current operation.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-13-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-13-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Increment the guest-visible "cur_entry" index outside of the for-loop
when processing Page State Change entries, and add a comment to make it
more obvious which code is operating on trusted data, and which code is
touching guest-accessible data.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-12-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-12-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that kvm_rax_read() truncates the output value to 32 bits if the
vCPU isn't in 64-bit mode, use it instead of the more verbose (and very
technically slower) kvm_register_read().
Note! VMLOAD, VMSAVE, and VMRUN emulation are still technically buggy,
as they can use EAX (versus RAX) in 64-bit mode via an operand size
prefix. Don't bother trying to handle that case, as it would require
decoding the code stream, which would open an entirely different can of
worms, and in practice no sane guest would shove garbage into RAX[63:32]
and then execute VMLOAD/VMSAVE/VMRUN with just EAX.
No functional change intended.
Cc: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260529222223.870923-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Make kvm_<reg>_{read,write}() mode-aware (where the value is truncated to
32 bits if the vCPU isn't in 64-bit mode), and convert all the intentional
"raw" accesses to kvm_<reg>_{read,write}_raw() versions. To avoid
confusion and bikeshedding over whether or not explicit 32-bit accesses
should use the "raw" or mode-aware variants, add and use "e" versions, e.g.
for things like RDMSR, WRMSR, and CPUID, where the instruction uses only
bits 31:0, regardless of mode.
No functional change intended (all use of "e" versions is for cases where
the value is already truncated due to bouncing through a u32).
Cc: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260529222223.870923-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Rename kvm_cache_regs.h to simply regs.h, as the "cache" nomenclature is
already a lie (the file deals with state/registers that aren't cached per
se), and so that more code/functionality can be landed in the header
without making it a truly horrible misnomer.
Deliberately drop the kvm_ prefix/namespace to align with other "local"
headers, and to further differentiate regs.h from the public/global
arch/x86/include/asm/kvm_vcpu_regs.h, which sadly needs to stay in asm/
so that the number of registers can be referenced by kvm_vcpu_arch.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260529222223.870923-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Check for full 64-bit mode, not just long mode, when truncating the
virtual address as part of INVLPGA emulation. Compatibility mode doesn't
support 64-bit addressing.
Note, the FIXME still applies, e.g. if the guest deliberately targeted
EAX while in 64-bit via an address size override. That flaw isn't worth
fixing as it would require decoding the code stream, which would open an
entirely different can of worms, and in practice no sane guest would shove
garbage into RAX[63:32] and execute INVLPGA.
Note #2, VMSAVE, VMLOAD, and VMRUN all suffer from the same architectural
flaw of not providing the full linear address in a VMCB exit information
field, because, quoting the APM verbatim:
the linear address is available directly from the guest rAX register
(VMSAVE, VMLOAD, and VMRUN take a physical address, but their behavior
with respect to rAX is otherwise identical).
Fixes: bc9eff67fc ("KVM: SVM: Use default rAX size for INVLPGA emulation")
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260529222223.870923-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Convert the repeated register save/restore sequences into macros,
trading some level of implementation trickiness for conciseness (more
than one register can be saved/restored with one invocation) and a
smaller chance of cut and paste errors between VMX and SVM files.
This becomes more useful with the upcoming support for APX, which
would need to add 32 lines to the VM entry/exit paths.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Convert the SEV-ES entry code to use macros for saving guest GPRs,
following VMX/SVM paths. Drop now-unused register offsets and
__VCPU_REGS_* defines.
No functional change intended.
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Link: https://patch.msgid.link/20260512011502.53072-4-chang.seok.bae@intel.com/
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Convert the register save/restore sequences in the SVM entry into macros,
following the VMX code. Drop the now-redundant register offset defines.
No functional change intended.
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Link: https://patch.msgid.link/20260512011502.53072-3-chang.seok.bae@intel.com/
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Convert the repeated register save/restore sequences into macros,
trading some level of implementation trickiness for conciseness (more
than one register can be saved/restored with one invocation) and a
smaller chance of cut and paste errors.
This is particularly useful in preparation for extended GPR support;
upcoming support for APX would need to add 32 lines to the VM entry/exit
paths.
No functional change intended.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Link: https://lore.kernel.org/6e67df0e-e5f0-43f5-aa86-22e8b01b75d2@redhat.com
Link: https://patch.msgid.link/20260512011502.53072-2-chang.seok.bae@intel.com/
[Keep kvm_vcpu_regs.h and put the macros in there. -Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Use READ_ONCE() when reading entries/indices from the guest-accessible
Page State Change buffer to defend against TOCTOU bugs.
Don't bother with READ_ONCE()/WRITE_ONCE() for cases where KVM is writing
(and not consuming the result!), as the guest isn't supposed to touch the
buffer while it's being processed. I.e. using READ_ONCE() is all about
protecting against misbehaving guests.
Fixes: 9b54e248d2 ("KVM: SEV: Add support to handle Page State Change VMGEXIT")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-11-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When processing Page State Change (PSC) requests, validate the PSC buffer
against the effective size of the scratch area, which could be less than
the maximum size if the guest provided a pointer that isn't exactly at the
start of the GHCB shared buffer.
Fixes: 9b54e248d2 ("KVM: SEV: Add support to handle Page State Change VMGEXIT")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-10-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Stop explicitly passing the PSC buffer to snp_begin_psc(): it *must*
be the scratch area. This will allow fixing a variety of bugs without
further complicating the code.
No functional change intended.
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-9-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that all paths in KVM properly validate the length needed for the
scratch area, and are guaranteed to pass in a non-zero length, WARN if KVM
attempts to configured the scratch area with min_len==0 to guard against
future bugs.
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-8-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When setting the length of the GHCB scratch area, and the area is in the
GHCB shared buffer, set the effective length of the scratch area to the max
possible size given the start of the guest-provided pointer, and the end of
the shared buffer.
The code was "fine" when first introduced, as KVM doesn't consult the
length of the buffer when emulating MMIO, because the passed in @len always
specifies the *max* size required. But for PSC requests, the incoming @len
is just the minimum length (to process the header), and KVM needs to know
the full size of the scratch area to avoid buffer overflows (spoiler alert).
Opportunistically rename @len => @min_len to better reflect its role.
Fixes: 9b54e248d2 ("KVM: SEV: Add support to handle Page State Change VMGEXIT")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-7-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When handling a Page State Change (PSC) #VMGEXIT use the size of the PSC
header as the minimum size for the scratch area. Per the GHCB spec, PSC
requests do NOT provide the length, i.e. using control->exit_info_2 for the
length is completely made up behavior. The existing code "works", e.g.
even though Linux-as-a-guest always passes '0', because KVM doesn't do
anything with the length when the request is in the GHCB's shared buffer.
Use the header as the min length. Once the header is retrieved, KVM can
use the specified indices to compute the full size of the request.
Fixes: 9b54e248d2 ("KVM: SEV: Add support to handle Page State Change VMGEXIT")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-6-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Explicitly ignore Port I/O requests of length '0' (or count '0'), so that
setting up the software scratch area (and other code) doesn't have to
worry about underflowing the length, and to allow for WARNing on trying
to configure the scratch area with len==0.
Fixes: 291bd20d5d ("KVM: SVM: Add initial support for a VMGEXIT VMEXIT")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-5-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When using GHCB v2+, reject MMIO requests that are larger than 8 bytes.
Per the GHCB spec:
SW_EXITINFO2 must be less than or equal to 0x7fffffff for version 1 and
less than or equal to 0x8 for all other versions.
Fixes: 4af663c2f6 ("KVM: SEV: Allow per-guest configuration of GHCB protocol version")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-4-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Explicitly ignore MMIO requests of length '0', so that setting up the
software scratch area (and other code) doesn't have to worry about
underflowing the length, and to allow for special casing '0' in the
future.
Fixes: 8f423a80d2 ("KVM: SVM: Support MMIO for an SEV-ES guest")
Cc: stable@vger.kernel.org
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
As per the GHCB spec, when using GHCB v2+ require the software scratch area
to reside in the GHCB's shared buffer. Note, things like Page State Change
(PSC) requests _rely_ on this behavior, as the guest can't provide a length
when making the request, i.e. the size of the guest payload is bounded by
the size of the shared buffer.
Failure to force usage of the GHCB, and a slew of other flaws, lets a
malicious SNP guest corrupt host kernel heap memory, and leak host heap
layout information.
setup_vmgexit_scratch() allocates a buffer via kvzalloc(exit_info_2),
where exit_info_2 is guest-controlled. With exit_info_2=24, this yields
a 24-byte allocation in kmalloc-cg-32 (32-byte slab objects). The buffer
holds an 8-byte psc_hdr followed by 8-byte psc_entry structs, so only
entries[0] and entries[1] are in-bounds.
snp_begin_psc() validates end_entry against VMGEXIT_PSC_MAX_COUNT (253)
but NOT against the actual buffer size:
idx_end = hdr->end_entry;
if (idx_end >= VMGEXIT_PSC_MAX_COUNT) { // checks 253, not buffer
snp_complete_psc(svm, ...);
return 1;
}
for (idx = idx_start; idx <= idx_end; idx++) {
entry_start = entries[idx]; // OOB when idx >= 2
The guest sets end_entry=10+, causing the host to iterate entries[2+]
which are OOB into adjacent slab objects. For each OOB entry:
- The host reads 8 bytes (OOB READ / info leak oracle)
- If the data passes PSC validation, __snp_complete_one_psc() writes
cur_page = 1 or 512 into the entry (OOB WRITE, sev.c:3806)
- If validation fails, the error response reveals whether adjacent
memory is zero vs non-zero (information disclosure to guest)
The guest controls allocation size (exit_info_2), entry range
(cur_entry/end_entry), and can fire unlimited VMGEXITs to repeatedly
hit different slab positions.
By exploiting the variety of bugs, a malicious SEV-SNP guest can:
- OOB read adjacent kmalloc-cg-32 objects (heap layout disclosure)
- OOB write cur_page bits into adjacent objects (heap corruption)
- Trigger use-after-free conditions across VMGEXITs
E.g. with KASAN enabled, a single insmod of the PoC guest module
produces 73 KASAN reports:
BUG: KASAN: slab-out-of-bounds in snp_begin_psc+0x126/0x890
Read of size 8 at addr ffff888219ffb5e0 by task qemu-system-x86/2199
BUG: KASAN: slab-out-of-bounds in snp_begin_psc+0x468/0x890
Write of size 8 at addr ffff888351566648 by task qemu-system-x86/2199
The buggy address belongs to the object at ffff888XXXXXXXXX
which belongs to the cache kmalloc-cg-32 of size 32
The buggy address is located N bytes to the right of
allocated 32-byte region [ffff888XXXXXXXXX, ffff888XXXXXXXXX)
Breakdown:
62 slab-out-of-bounds (reads + writes past allocation)
7 slab-use-after-free
4 use-after-free
All credit to Stan for the wonderful description and reproducer!
Reported-by: Stan Shaw <shawstan96@gmail.com>
Cc: Michael Roth <michael.roth@amd.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Peter Gonda <pgonda@google.com>
Cc: Jacky Li <jackyli@google.com>
Fixes: 4af663c2f6 ("KVM: SEV: Allow per-guest configuration of GHCB protocol version")
Cc: stable@vger.kernel.org
Signed-off-by: Michael Roth <michael.roth@amd.com>
[sean: write changelog]
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-2-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- Include the kernel's linux/mman.h in KVM selftests to ensure MADV_COLLAPSE
is defined, as older libc versions may not provide it.
- Include execinfo.h if and only if KVM selftests are building against glibc,
and provide a test_dump_stack() for non-glibc builds.
- Fudge around an RCU splat in the emegerncy reboot code that is technically
a legitimate flaw, but in practice is a non-issue and fixing the flaw, e.g.
by adding locking, would incur meaningful risk, i.e. do more harm than good.
- Rate-limit global clock updates once again (but without delayed work), as
KVM was subtly relying on the old rate-limiting for NPT correction to guard
against "update storms" when running without a master clock on systems with
overcommitted CPUs.
- Fix a brown paper bag goof where KVM checked if ERAPS is "dirty" instead of
marking it dirty when emulating INVPCID.
- Flush the TLB when transitioning from xAVIC => x2AVIC to ensure the CPU TLB
doesn't contain AVIC-tagged entries for the APIC base GPA.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmoZtdgACgkQOlYIJqCj
N/26sw/+IWOA5AxyoNW/lKAhhkzDTzGWrNCQkpMv+F4tOUbHYniTxI/pv4L3eMvf
ZLUXijYxhpJtnblLtrnPpSFl5tll4xQdMUv7+fljgpYmy6+erQHodtCgRi5wHDbM
NlD7DWOgwmpvzYLcybq1RfjZ3n+OBRvq95haQ6Ph4FtoYuIomtJ5tF2mnMlyxlc/
aIK5wzQ/JeYdQxwwz1ctlHkgE5bPnS+Sxr33+MRFQ5cIpuwdoS9zYRITNBM107kg
bLeei8Cxh91sgEidgwS8JToLvaEQH8AodkROjcScllwUxYsshPKsHeH7sTMbCOVd
DiH9VbheZo7d4kb6pvhGsY891ec00dR5E/l2gZYLWHg4v0lINTw6uBdoJuq3t2TO
Q3KmGVaUWz+c6dY/0qntVpws35zG106S8Pp4mx/1EnUHbJKZYDsUMC1ppwhrr3Pz
WEyQ9PFXhOyoSbrtOaEfU+wsFPeAfT9eYADu7oV1t7l75TJAKW1EEaSGfzOO/crj
3GK3vRq2B1cMHX9c4fwhSs4h8k5JvKlI/mtGPxZN3khVorx9dv/rTqOoeQEsFS5+
8s5XcNPPJlKfNXcu3Jq6rn8U/JA2HnbH298Nk5uXTCfTrZtDgbOnI8YVYWnoadOl
8xJoie5ccEsysVj1npNNh61LNMF1XBUUC+eNn0I1o0NzeRauxF8=
=QQUn
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-fixes-7.1-rc6' of https://github.com/kvm-x86/linux into HEAD
KVM x86 fixes for 7.1-rcN
- Include the kernel's linux/mman.h in KVM selftests to ensure MADV_COLLAPSE
is defined, as older libc versions may not provide it.
- Include execinfo.h if and only if KVM selftests are building against glibc,
and provide a test_dump_stack() for non-glibc builds.
- Fudge around an RCU splat in the emegerncy reboot code that is technically
a legitimate flaw, but in practice is a non-issue and fixing the flaw, e.g.
by adding locking, would incur meaningful risk, i.e. do more harm than good.
- Rate-limit global clock updates once again (but without delayed work), as
KVM was subtly relying on the old rate-limiting for NPT correction to guard
against "update storms" when running without a master clock on systems with
overcommitted CPUs.
- Fix a brown paper bag goof where KVM checked if ERAPS is "dirty" instead of
marking it dirty when emulating INVPCID.
- Flush the TLB when transitioning from xAVIC => x2AVIC to ensure the CPU TLB
doesn't contain AVIC-tagged entries for the APIC base GPA.
Now that KVM correctly handles Host-Only and Guest-Only bits in the
event selector MSRs, allow the guest to set them if the vCPU advertises
SVM and uses the mediated PMU.
Signed-off-by: Jim Mattson <jmattson@google.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260527234711.4175166-14-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reprogram PMU counters on nested transitions for the mediated PMU, to
re-evaluate Host-Only and Guest-Only bits and enable/disable the PMU
counters accordingly. For example, if Host-Only is set and Guest-Only is
cleared, a counter should be disabled when entering guest mode and
enabled when exiting guest mode.
According to the APM, when EFER.SVME is cleared, setting Host-Only or
Guest-Only disables the counter, so also trigger counter reprogramming
when EFER.SVME is toggled.
Counters setting any of Host-Only and Guest-Only bits are already being
tracked in pmc_has_mode_specific_enables, use the bitmap to reprogram
these counters.
Reprogram the counters synchronously on nested VMRUN/#VMEXIT and
EFER.SVME toggling. This is necessary as these instructions are counted
based on the new CPU state (after the instruction is retired in
hardware). Hence, the PMU needs to be updated before instruction
emulation is completed and kvm_pmu_instruction_retired() is called.
Defer reprogramming the counters when force leaving guest mode through
svm_leave_nested() to avoid potentially reading stale state (e.g.
incorrect EFER). All flows force leaving nested are non-architectural,
so accuracy is irrelevant.
Refactor a helper out of kvm_pmu_request_reprogram_counters() that
accepts a boolean allowing synchronous vs deferred reprogramming, and
use that from SVM code to support both scenarios.
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260527234711.4175166-13-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>