- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1
is present, guaranteeing that EL2-specific register bits are stateful
in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope
of those instructions remains 'in host' (i.e. L1 kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly in
the VGIC, our in-kernel generator of bugs (and sometimes interrupts)
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQSNXHjWXuzMZutrKNKivnWIJHzdFgUCaoZ/khccb2xpdmVyLnVw
dG9uQGxpbnV4LmRldgAKCRCivnWIJHzdFkK7AP9zvAcDf+DVte8+mZjCMDrvNB1n
UWsBnQkdZ1nDDYBJlwD8D1pvurK4ZV3UIl68IgpTqYKhMspCuOWDHrnBt1QyqQU=
=2iPX
-----END PGP SIGNATURE-----
Merge tag 'kvmarm-7.3' of https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 changes for 7.3
- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1
is present, guaranteeing that EL2-specific register bits are stateful
in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope
of those instructions remains 'in host' (i.e. L1 kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly in
the VGIC, our in-kernel generator of bugs (and sometimes interrupts)
* kvm-arm64/misc-7.3:
: Miscellaneous fixes for KVM/arm64, 7.3
:
: - Fixes for saving invalid table entries as part of saving the ITS
: tables (Fuad Tabba)
:
: - Don't reallocate the SPI array for re-attempted vgic_init(), avoiding
: a memory leak (Fuad Tabba)
:
: - Hold a reference on an LPI when saving the pending state (Qihang)
:
: - Don't WARN for out-of-range, guest-supplied INTID (Karl)
:
: - Avoid corrupting GPRs for 32-bit CP64 reads (Karl)
:
: - Reset 'in kernel' VGIC state when private IRQ allocation fails (Fuad)
:
: - Avoid kallsyms lookup in nVHE panic unless the host stage-2 is also
: disabled (Vincent)
:
: - Disregard Pending+Active state when computing maintenance IRQ for
: ICH_MISR_EL2.NP (Kajetan)
:
: - Various Sashiko-identified issues dealing with GICv5 (Sascha)
:
: - Fix CPU onlining in pKVM due to mismatched accesses when the MMU is
: disabled (Will)
KVM: arm64: Validate GICv5 timer PPIs before claiming ownership
KVM: arm64: vgic: Reject out-of-range GICv5 PPI IDs
KVM: arm64: vgic: Prevent speculative SPI array underflow
KVM: arm64: vgic: Free gic_kvm_info on initialization failure
KVM: arm64: Avoid mismatched accesses to 'struct kvm_nvhe_init_params'
KVM: arm64: vgic: Fix detection of MI on no pending LR
KVM: arm64: Drop %pB on nVHE panic when stage-2 is active
KVM: arm64: vgic: Reset in_kernel on private IRQ allocation failure
KVM: arm64: GICv2: Don't WARN on out-of-range GICV_DIR INTID
KVM: arm64: Preserve GPRs for AArch32 CP64 reads generating an UNDEF
KVM: arm64: vgic-v3: take an LPI reference in vgic_v3_save_pending_tables
KVM: arm64: vgic-its: Point saved ITEs at the next valid entry
KVM: arm64: vgic-its: Don't save collections the table cannot hold
KVM: arm64: vgic: Don't leak the SPI array when init is retried
KVM: arm64: vgic-its: Don't dereference a NULL collection on ITT save
Signed-off-by: Oliver Upton <oupton@kernel.org>
* kvm-arm64/vtr-patch:
: Inline patching of ICH_VTR_EL2 constant, courtesy of Marc Zyngier
:
: Unify readers of ICH_VTR_EL2 on an instruction-patched constant value,
: avoiding system register accesses known to trap under nested
: virtualization and sharing the implementation between pKVM and 'regular'
: KVM.
KVM: arm64: vgic-v3: Kill kvm_vgic_global_state.ich_vtr_el2
KVM: arm64: vgic-v3: Simplify initial GICv3 configuration sampling
KVM: arm64: Convert most ICH_VTR_EL2 accesses to inlined literal value
KVM: arm64: Add a helper providing an inlined literal value for ICH_VTR_EL2
KVM: arm64: Move GICv3 broken SEIS implementation detection to a CPU errrata
KVM: arm64: vgic-v3: Make vtr_to_* helpers use architectural field symbols
Signed-off-by: Oliver Upton <oupton@kernel.org>
* kvm-arm64/vncr-fixes:
: Fixes for handling L1 VNCR mappings, courtesy of Marc Zyngier
:
: - Recompute the VNCR software TLB when the MMU is toggled at stage-1
:
: - Improve TLB invalidation intersection to handle TLBIs affecting the
: end of the VA space
:
: - Fix race to invalidate the VNCR fixmap between TLBI emulation and
: vcpu_put()
:
: - Add missing sign extension for computing TLBI ranges
:
: - Make VNCR invalidation participate in the MMU notifier seqcount,
: preventing a concurrent VNCR TLB fill from consuming a stale
: translation
KVM: arm64: Correctly cap TLBI Range to the architural limit
KVM: arm64: Add VNCR TLB tracking again
KVM: arm64: Make VNCR invalidation participate in MMU invalidation retry
KVM: arm64: Sign-extend VA for range-based TLBI invalidation
KVM: arm64: Handle VNCR TLB invalidation race with vcpu_put() VNCR unmapping
KVM: arm64: Correctly handle end of VA space TLBI invalidation
KVM: arm64: Consider SCTLR_EL2.M when mapping the L1 VNCR page
KVM: arm64: Handle negative S1 walk levels in VNCR TLB size evaluation
KVM: arm64: Remove VM-wide VNCR mapping counter
Signed-off-by: Oliver Upton <oupton@kernel.org>
* kvm-arm64/pkvm-7.3: (26 commits)
: pKVM updates for 7.3
:
: - Avoid name collision on trace_clock() when CONFIG_NVHE_EL2_TRACING is
: disabled (Mostafa Saleh)
:
: - Clean up state tracking for whether the EL2 shadow VM has been
: created (Fuad Tabba)
:
: - Synchronize SCTLR_EL1 when injecting an exception to use current
: PAN/SSBS state (Fuad Tabba)
:
: - Avoid unnecessary cache maintenance when I/D-cache are known to be
: coherent in pKVM (Mostafa Saleh)
:
: - Lazy vCPU context save/restore for pKVM (Fuad Tabba)
:
: - Various fixes to the stage-2 MMU for pKVM (Fuad Tabba)
:
: - Allow counter offsetting of non-protected guests in protected mode
: (Mostafa Saleh)
:
: - Condition the 'broken CNTVOFF' mitigation on a VM actually having a
: nonzero offset, fixing boot failures of pVMs on affected hardware
: (Mostafa Saleh)
KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
KVM: arm64: Fix timer offsets for non-protected VMs
KVM: arm64: Make timer_get_offset() work in all contexts
KVM: arm64: selftests: Add stage-2 block transition test
KVM: arm64: Don't advertise eager page splitting under pKVM
KVM: arm64: Don't WARN on pKVM stage-2 map failures
KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled
KVM: arm64: Top up stage-2 memcache for dirty logging faults
KVM: arm64: Top up the memcache for pKVM permission faults
KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings
KVM: arm64: Implement lazy vCPU state sync for non-protected guests
KVM: arm64: Add primitives to flush/sync the VGIC state at EL2
KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
KVM: arm64: Add host and hypervisor vCPU lookup primitives
KVM: arm64: Move PSCI helper functions to a shared header
KVM: arm64: Factor out reusable vCPU reset helpers
KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
KVM: arm64: Extract MPIDR computation into a shared header
KVM: arm64: selftests: Add a userspace watchpoint test
KVM: arm64: Flush external_mdscr_el1 to the pKVM hyp vCPU
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
* kvm-arm64/pmu-7.3:
: vPMU updates for 7.3
:
: - Support for slot-based PMU events, relying on new UAPI that makes
: selection of a vPMU implementation mandatory (Congkai Tan)
KVM: arm64: Add KVM_ARM_VCPU_PMU_V3_STRICT vCPU feature
KVM: arm64: Ignore writes to PMCR_EL0.N when using strict UAPI
KVM: arm64: Advertise STALL_SLOT* in PMCEID1 under strict PMUv3 UAPI
KVM: arm64: Expose PMMIR_EL1.SLOTS under strict PMUv3 UAPI
Signed-off-by: Oliver Upton <oupton@kernel.org>
With GICv5, timer PPIs use architecturally defined IDs, and therefore
we require userspace to use the architected IDs too. We perform this
validation in timer_irqs_are_valid(), which validates the configured
PPI after claiming it with kvm_vgic_set_owner(). This is obviously the
wrong order, and could both result in a NULL pointer dereference for
an ID >= 64 and in an incorrect ownership claim.
Switch the order to first validate the PPI before setting the
owner. This both prevents an invalid GICv5 timer PPI from reaching the
IRQ lookup, and avoids claiming an IRQ for a timer configuration that
will be rejected.
Fixes: 06c85b58e0 ("KVM: arm64: Move GICv5 timer PPI validation into timer_irqs_are_valid()")
Link: https://sashiko.dev/#/message/20260810111614.DDC731F000E9%40smtp.kernel.org
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260811150941.941295-5-sascha.bischoff@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
GICv5 supports up to 128 PPIs, but KVM currently implements only the
first 64, which contain the architected PPIs it supports.
An encoded PPI with an ID outside that range passes irq_is_ppi(),
which only checks the encoded interrupt type. vgic_get_vcpu_irq()
therefore looks it up in private_irqs[], where array_index_nospec()
clamps the out-of-range index to zero and aliases PPI 0.
Include the supported PPI range in irq_is_ppi() so that KVM interfaces
reject unsupported PPIs. Also reject an out-of-range PPI in the lookup
as a safeguard against callers bypassing the predicate.
Fixes: 4d591252ba ("KVM: arm64: gic-v5: Implement PPI interrupt injection")
Fixes: eb8bce08ec ("KVM: arm64: gic: Introduce interrupt type helpers")
Link: https://sashiko.dev/#/patchset/20260724104819.1296803-1-sascha.bischoff@arm.com?part=27
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260811150941.941295-4-sascha.bischoff@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
For a non-GICv5 VM, SPI interrupt IDs include the private-interrupt
offset, while KVM's SPI array is indexed from zero. The lookup applies
array_index_nospec() to the absolute interrupt ID and subtracts the
private-interrupt offset afterwards.
On a speculative bypass of the range check for an interrupt ID below
the private range, the clamp preserves the small absolute value and
the subtraction underflows to an out-of-bounds SPI array index.
Convert the interrupt ID to a zero-based index into the SPI array
before applying array_index_nospec(). This way, we ensure that we
clamp to a reachable SPI ID, rather than an out-of-range SPI index.
Fixes: 41b87599c7 ("KVM: arm/arm64: vgic: fix possible spectre-v1 in vgic_get_irq()")
Link: https://sashiko.dev/#/patchset/20260724104819.1296803-1-sascha.bischoff@arm.com?part=27
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260811150941.941295-3-sascha.bischoff@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
vgic_set_kvm_info() allocates gic_kvm_info for use by
kvm_vgic_hyp_init(). When a maintenance interrupt is mandatory but not
provided, kvm_vgic_hyp_init() returns -ENXIO before reaching the
common cleanup path, leaking said allocation.
Route this error through that cleanup path so that gic_kvm_info is
freed and the global pointer is cleared.
Fixes: 0e5cb77706 ("irqchip/gic: Split vGIC probing information from the GIC code")
Link: https://sashiko.dev/#/patchset/20260724104819.1296803-1-sascha.bischoff@arm.com?part=1
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260811150941.941295-2-sascha.bischoff@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
When running with hVHE enabled, ___kvm_hyp_init() calls
__kvm_init_el2_state() on the CPU initialisation path during onlining
and resume from suspend.
In order to avoid clobbering the link register across this call, it is
stashed away in the 'tmp' member of 'struct kvm_nvhe_init_params',
however this save/restore operation is performed with the stage-1 MMU
disabled at EL2 and therefore gives rise to coherency problems because
the field is not aligned or padded to the CWG. For example, a cacheable
write to a physically-adjacent structure sharing the same cacheline
could lead to an eviction and subsequent write-back, overwriting the
saved LR while the incoming CPU is executing __kvm_init_el2_state().
Save the lr in far_el2 and remove the 'tmp' member from
'struct kvm_nvhe_init_params' altogether.
Cc: Oliver Upton <oupton@kernel.org>
Cc: Marc Zyngier <maz@kernel.org>
Fixes: afa9b48f32 ("KVM: arm64: Shave a few bytes from the EL2 idmap code")
Signed-off-by: Will Deacon <will@kernel.org>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260813131717.5885-1-will@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
- Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs.
- Harden the SNP and TDX "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion.
- Fix a variety of #DB priority bugs.
- Fix a class of races related to enabling Hyper-V emulation on a vCPU after
the vCPU is visible to the rest of KVM.
- Use static calls for nested virtualization ops.
- Move more KVM-internal code out of x86's kvm_host.h.
- Enumerate support for a variety of Zhaoxin instructions that don't require
explicit virtualization.
- Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path.
- Harden kvm_vcpu_map() against double-mapping and thus leaking references.
- Misc fixes and cleanups, e.g. for largely benign syzkaller splats.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp8uZQACgkQOlYIJqCj
N/2Pgw//ZmY9QZSvbSS69yYporgRhlTCMq1k3siLYfT19XShCpjO0KK9QTggs57L
0JXgbv4kwzvWWliwlRVK3e2nw027GI0j0kDRdcI3RAsaufZCjVT300cfUJBy/T4C
EQLULA8J3Q5pXStuW7SS2bMjI/D6J0+h80co1gfN4yeBDuKXUhtpmW9G/frCQyqI
3DduI57IBWO+pHImMWVx5jFc6WeDokQPxVASWE4zAU5PobXZSQ2K7KCbcx/rHxLt
3JZRw9sECF4Rn5DHm9Xag4bfZH0IR1jQ4P5fIzlvvYcW8yhj4CcTK648fVivBCL5
cpVQRCUVM6DyAX7l6tchP44XHv8a+D/bB2qZUlKvrMiSYA3usW7olKEDNUPI0dsm
vviat3br/XBu3msNkMZcYoP2WS+ka9zDAkSw7MkNWg1m62UfzvI70AQC0r7Fs/UG
+bUwMpJ11C/TOjwVL8DNMu0RT2uPgZ2xgtyD/IeVrqlm867xtlGxit+Uqs9V0vro
LC2YhNR2LYk8NhKO/IZsHI/9eTD4xQyPBn13U6dsZy19fqcPftSJM7avZfQNiA+l
nPN3v/FZmM22k32hc4NmGCx1/pBU1QtnTcZ0Z91pVzD5+DrvpYnMJpdhazrLL18j
IN2OK+Vy/WiP2xnAvCjOFcHCVgAe3SCorqIrjRlY2nMbnRmc03E=
=bQJe
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-misc-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM x86 misc changes for 7.3
- Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs.
- Harden the SNP and TDX "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion.
- Fix a variety of #DB priority bugs.
- Fix a class of races related to enabling Hyper-V emulation on a vCPU after
the vCPU is visible to the rest of KVM.
- Use static calls for nested virtualization ops.
- Move more KVM-internal code out of x86's kvm_host.h.
- Enumerate support for a variety of Zhaoxin instructions that don't require
explicit virtualization.
- Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path.
- Harden kvm_vcpu_map() against double-mapping and thus leaking references.
- Misc fixes and cleanups, e.g. for largely benign syzkaller splats.
- Remove a dying VM from the GA Log notifier list before the VM is actually
destroyed, to fix a potential use-after-free.
- Don't pass FOLL_WRITE when registering encrypted memory regions, i.e. when
pinning SEV/SEV-ES guest memory, to fix a regression with file-backed memory
introduced by KVM's (correct) usage of long-term pins.
- Allocate full pages for SEV/SEV-ES {DE,EN}CRYPT ops on SNP-enabled hosts to
fix a data corruption issue due to the PSP driver assigning to-be-written
pages to firmware (as required by the SNP specs).
- Unconditionally intercept ICBEP so that KVM generates the correct guest RIP
when handling an ICEBP-induced TASK_SWITCH #VMEXIT.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp80WUACgkQOlYIJqCj
N/3JuhAAv0zwXVVCQbE8Ni3CT/Wl03MEXGDsqlv1mBbsI5WkOte4CshCK8oHAYs4
LDhO6Y4BlIRJ78itpzk+FF5pvx0MPLi535PNx71o2IGKRZ2T/0u9GVFvHjXdM61F
OdMFSbRYTg4EHPhqskYN4vIy11Qd3vhaklwcDXxnxlZeTq2wtuVwa1GM7rkmiyRs
w3dtNrjSTM2axc4+H+gIN/WN5wM5jxsMVxhPdhPzK2isl4+btD9PG7BG64+aZVqJ
qW9CVTEz81yXdR04sTYutV0Hf56TzGE1RC75MH5nMi7AT9Lfq9L8eVSqPpCZBG/z
YTO8oSQtD9SrIabE8YQ/pBf+LH4DpDCnxjA5IDeiP/6BW54Via6PXpDQQGGrgqWF
pWs2jlc1nUxlrUPrPsQxx1yiFk0xDb/ketgwCHvVilyT5CV33ZO3w995FA3Lvjgy
KY12jxl1jwb8SgZfEkVhUQBNSIsfhj6Wb+jJdbvgcA5M7iUbKk7bBGaokhTbGsev
e63JCVFRydidINyk+LsvUfgjm1ua18x97A7ihcNMIsJY9CCALK8B/HtyPena2phM
F+EZ+IydI/JIrXU4C/o0tWm+mlKbGsxAoCcVx/OkIsckEQYhBgcYPlKr5hZvzy1S
deEmOEXB9cUIO11AOELYnRwEJl6xNuxa226It7WUtmuYQ9t5z48=
=MkIK
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-svm-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM SVM changes for 7.3
- Remove a dying VM from the GA Log notifier list before the VM is actually
destroyed, to fix a potential use-after-free.
- Don't pass FOLL_WRITE when registering encrypted memory regions, i.e. when
pinning SEV/SEV-ES guest memory, to fix a regression with file-backed memory
introduced by KVM's (correct) usage of long-term pins.
[This is correct because, while FOLL_WRITE was needed in the past to trigger
CoW unsharing, nowadays FOLL_LONGTERM does that already even without
FOLL_WRITE. And in fact, get_user_pages() actually disallows FOLL_WRITE
together with FOLL_LONGTERM. This change was acked by the MM maintainers.
For more inforamtion see commit ee1a586dd1.
- Paolo]
- Allocate full pages for SEV/SEV-ES {DE,EN}CRYPT ops on SNP-enabled hosts to
fix a data corruption issue due to the PSP driver assigning to-be-written
pages to firmware (as required by the SNP specs).
- Unconditionally intercept ICBEP so that KVM generates the correct guest RIP
when handling an ICEBP-induced TASK_SWITCH #VMEXIT.
- Service local TLB flushes on a failed nested VM-Enter to fix a bug where KVM
could miss a TLB on a future, successful VM-Enter with the same L2 VPID.
- Cap the maximum value shoved into the VMX Preemption Timer to workaround an
erratum that affects all existing Intel CPUs that support CPUID 0x15.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp82CwACgkQOlYIJqCj
N/1ekg//Z3ulUOnxKhywPtJ9T2bGlGoqJwslTygFy2JQfR4eNdnNYvtLU2A6Gsa2
1IaB/xJqMtSb+XxpgghYuGvU29J/i25QOJRM0RyJhaXxYvDkZPP4P1gwm9/qniFn
OTITTM37twD98uIYMjOXP6UsK7MPg2Kogv74hYwTpHVj/ti9weLwRX+x+bmlA2Aj
lPFUqPqQkLWFvhoSLV5uYW1rZfLtq/UdlV13DCF+vtp1u6IXaW6mFG5RMFog9kSt
arIW2ZG9Gx7SwrLrpRUSpQyRZugzTcNyl/lK+/sGzONfJO1Gw2QZ7bJFckhXoRt1
ktR5ZAXc8Zy7It7gsNozvQD1NsnT2+FTDzygv7h15IIOwWi+quWF63BCNz+in5RA
ivmFrUnRwmbGgoshpD+JjwR5qW7TK0lOLudWMWnH6ctIH3PvgJrNiIuTbKNGxtkI
D0WuDLrLcSqiR9M8q5eLTKiRjykYugG81KIHIMoTBszpO8/qIhdXyPUr3TPUkoHR
SeDL0m4KdURZR2cOTOvy6DIPTY2qn+rpxRFkqXJCUx/98kajsF0/0RDMBwUeaGF8
Y9e9bgR7YW8hMd6z64ODFTLUSMYXavC0AJpQq3spWg0bo15CcAFTAAnyAQOpTgyo
IGENMWREkCa62RrveZLhN3rFkYBbGMZSuBa0UQM+NvvJXDwdJ90=
=BvM6
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-vmx-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM VMX changes for 7.3
- Service local TLB flushes on a failed nested VM-Enter to fix a bug where KVM
could miss a TLB on a future, successful VM-Enter with the same L2 VPID.
- Cap the maximum value shoved into the VMX Preemption Timer to workaround an
erratum that affects all existing Intel CPUs that support CPUID 0x15.
- Fix a bug where KVM would walk a newly created rmap without holding the rmap
lock (or mmu_lock) during aging.
- Fix a bug where aging TDP MMU SPTEs could clobber FROZEN SPTEs.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp8wS8ACgkQOlYIJqCj
N/2SmQ//ecBzKzw5uQVwigrVGwWNZS1I00d4DZsYQpD/cAfSuKCYG4YGfG5IBtUy
tvMmoPswmTbullysJxB2bW0JOPjmGGEE6lsNhCH7CmgpKXdeBgqUo2x+KHVV8hFv
YlXXBA2rkbtObMjQgfaByIjv2M3d0trSkOfe39WvPEcfqjOBp3GK22km9eHw4qN2
PqidjSHtAzwcsd+id1cmw5uoel9RKOpux7FPM5mfVIWHJByDrY5sUU+u9Ug0p4F1
2wWbG6ghdnO0LsFFrGD2K2Ummux1KaBuFny/q4cqv3dg3IsMrLcX5xpmC9Z+N1Ey
mmq0D87VXzxEaYXDiLB7hWp6ZM+5dFXunSyhoX/Qu8ALUrUKyXY7MLOdLPXvsYpx
YEy5OTsZCdlebvziu2w8+aNOTxsctYF9cj8yraz8PtmKqgJ2Wg1seNLl7rfGwCWP
DGtvaPrS20cm+OIVD26or4zfR+6DGhilfz1BItoQejzA4ryVo3XEcYn9U11ph6mn
9Mu0W/XVl9AX44lliENPbcNmLDB5X65twv+1eIZkbevFRpSFsJYFg4Fdg6/0JK8g
B2zPyKmsTi+PgAx6dnEZeVNDhYY6P7E1OaGbXqIvNSdzGM6Ps3gbfIYiIOPD515B
kGi8y9kDKnNZ7PQrACTWU+u6stB8f4BoXBE7Au1+GNPbSRlqu9M=
=axEq
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-mmu-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM x86 MMU changes for 7.3
- Fix a bug where KVM would walk a newly created rmap without holding the rmap
lock (or mmu_lock) during aging.
- Fix a bug where aging TDP MMU SPTEs could clobber FROZEN SPTEs.
- Remove a defunct masterclock update from kvm_xen_shared_info_init() that
could result in corrupting kvmclock, for a lose definition or "corrupting",
due to triggering an unnecessary switch to/from masterclock mode.
- Skip Xen runstate time updates if time has effectively gone backwards, so
that the guest doesn't report 100% steal time for a very, very long time.
- Drop KVM's runtime updates of the Xen PV timing CPUID leaf, as KVM was
updating the wrong sub-leaf, and upstream KVM will soon provide all the
information needed by userspace to populate the CPUID field itself.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp7vSIACgkQOlYIJqCj
N/1FzRAAhtEbAmo1wQAfwCfC1zCGxxfNS45+KOUoa9qbBkmhqou0Vn648O4BxamK
7MJ2J9OZ25X5O+kkx0YTTqiRhPZ+z8NfAQeCAd2XhwqmLtKaY8PLea+Ei132ztLV
1UDHjxTjgrDrwZvV5V2q7IzHHWFxWm4HVX1BrArcsB2hndiq7Re+eYGF12IL2DEJ
nmL1Atc4xAVQYpeLqt9hT3mmbv09mifjEHGrzyP0xmCs6V8HKtpdM0tv9mdSvuRn
gTpnAmD9akfk4iT8xLW/1Di86a7KkWPqMU3te9GP5XBXeRld+iLZCPG1f5zq9wmT
I7wdScR8TPeEsawDIfwgW+SqbBE/452fbLk6spoIAJYysR8k5sXY5QMbmADznpBE
6MtDQgQ8IbevMZsb6NnfmYz2AFdBQcJN50/Zdk2SEXwXZJD6gZdNIVovlZURl0I5
9ApfHl7tSxW+Wz5aRL/iDCxQeoh+0cRXNbi4WRb6am3wgn8LVBvOgkZZNoTI1qE2
MN/Oc6WVavbMlOBC1mz9LuuiFugJnprWkxBjzvpzrlUBjt5m6lIni+9SctEArMuD
7tZbr1w8XGBwJU4xrMn16KC53+HAltuBgN0BK1umKtIGasn5K6cmGADpONclJ6RN
IKhcqIGwF+eqtYfthYIrGGsvMjrw9h1MGMMnLS3mt/J1GjX1VOo=
=oKIz
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-clocks-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM x86 PV clocks and timekeeping related changes for 7.3
- Remove a defunct masterclock update from kvm_xen_shared_info_init() that
could result in corrupting kvmclock, for a lose definition or "corrupting",
due to triggering an unnecessary switch to/from masterclock mode.
- Skip Xen runstate time updates if time has effectively gone backwards, so
that the guest doesn't report 100% steal time for a very, very long time.
- Drop KVM's runtime updates of the Xen PV timing CPUID leaf, as KVM was
updating the wrong sub-leaf, and upstream KVM will soon provide all the
information needed by userspace to populate the CPUID field itself.
- Forcefully invalidate SNP VMSA pages if their backing guest_memfd page is
zapped/invalidated, e.g. due to a PUNCH_HOLE in response to a Page-State
Change request.
- Rework the so called "prepare" and "invalidate" guest_memfd hooks to prepare
for in-place private<=>shared conversion, and clean up a few warts along the
way.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp7vdQACgkQOlYIJqCj
N/00vA/+NmxB+Zsr7+TZo4NFiFcs/FK7rBb6psvlLuJN1oNLPw7dzdmPclSyv7li
RDWRF6SmuujxCO2W4LMc1Fz5IjN8/iYPHsEHnYCCxjDgFdEl7x1MOJ0xuXQOWgQw
icpWdYgWh9EoaKepn0EFVeF7WZFK7KhxmpJNz5HZEx3p110HfrSHbjl0LikySDst
19Jck5ZzTCVrbF58e5k6phzvvR6mRNeLS7BKvOM+u5U70+7P9AcTbihd8qz7vy0k
LlJUBZbp7NywjgIcqMTZGmBvsWoOAcvykBX9hNgjDdobVhh3Ft+hsmOxNt7yOoAl
llJznwRQvZItWGfGO8o9l+/pJI7bvGzQ6isXqdg3TbxWH7/d6HOdWoGiC8ya2x+k
UBDvlRgAH9KZuuE7E2m/ggM0hoLbWyNDV2qVo2WnmqbGG4SQRSA5hyq/pIov0hRY
kr0anBd/Pw8ZoO02RtlrSQkKODbdR7l3/lQDzHzSWKidJ8Eh+iqA30/oIQzQMCOw
3s/SIB19Jab4UEgS3aqvFNbsj+zXN2tT6NLiiSoodSp+sSP5OMJ46U4Sb/txn9EP
L035UIyLRAGFsIybvutSckT/Guv9aVeeK0F2CNRvzKrPUhaj3Vi2a7RvQtZZWeVd
EJLzM07Rx8msGZYq8Eg+9QkrDe5IgVjJvsZ5RrKeS7VJKivpFRU=
=yUfX
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-coco-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM guest_memfd and x86 CoCo changes for 7.3
- Forcefully invalidate SNP VMSA pages if their backing guest_memfd page is
zapped/invalidated, e.g. due to a PUNCH_HOLE in response to a Page-State
Change request.
- Rework the so called "prepare" and "invalidate" guest_memfd hooks to prepare
for in-place private<=>shared conversion, and clean up a few warts along the
way.
- Remove kvm_debugfs_dir if kvm_init() fails after creating KVM's debugfs.
- Document some of the "fun" gotchas with the APIC base when creating IRQCHIPs
on x86.
- Add a per-VM bitmap to track which vCPU IDs have been "claimed" but for
which the vCPU isn't yet online, and use the bitmap to reject duplicate IDs
before calling into arch code. This allows arch code to consume vcpu_id
without having to worry about cross-vCPU clobbering (at least s390 and x86
have had related bugs).
- Zero a vCPU's entry in VMX's Posted Interrupt Descriptor table used for IPI
virtualization when the vCPU is freed to fix a use-after-free where hardware
will write to a freed vCPU's PID.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp7v0oACgkQOlYIJqCj
N/0d/Q/+MWjv5Ua3LXBIcdw34zm+ympYVwEs30x/GPM/434DR/JS7iKAL0ZaY/Wc
uQFlNOJ74Nwq59y63VFGhMJeJU3zh/T3aKS/yarlbeUn2kY5Mij856KnY1acKo1n
hVaW+S8xAYhyf4QWOugeheSU/nrQ6df13EA+zJWOcqWyeeWHm2cykpRAoNhglVJr
nu4lPewBqVBfLmNIMtWhV0L9YllFO9nhGA4Ho7mRCTp8McxXNoeqSmlqm5xbwa0U
iNgpyzEaZwXHBkjk/PbTNjmB4uRLIkhn3o+Ol0PRyt/R1OgJScwmlK+R04b+9nQa
wI0C4bh3lE5wyAFE78yIkju1DhTLHe5Vd748hONIyuwQ2fJoh5SCpFoPX59uND39
W4qU6I7DgAX4Oum0yuwBLMUaKTRtupgVHFlsXSIrx4Im95cYllUdfktLArmpdnfK
mFPOMPtD2x2RdFWhLQSGVLO+kG9IjRc06P0UVPwQPUc2CNfQKYgTqYjIPBn6KTef
U5ZQeN7TdZFg3Vchrwmis3PjYKeP/xdQPNOsMG9q7sXSdo8N4fCCsHfBjceEO5I5
s+DUbDJKruQW3MrwGTa6vCqOyLipNVxBXwgeOZQEDGbi+AynBB3QofY54jQSqwfL
SFbhfiTbcfu1JCogaCarZqclzVUp333LESNqqMljWMzDbCS9VwE=
=19dB
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-generic-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM arch-neutral and documentation changes for 7.3
- Remove kvm_debugfs_dir if kvm_init() fails after creating KVM's debugfs.
- Document some of the "fun" gotchas with the APIC base when creating IRQCHIPs
on x86.
- Add a per-VM bitmap to track which vCPU IDs have been "claimed" but for
which the vCPU isn't yet online, and use the bitmap to reject duplicate IDs
before calling into arch code. This allows arch code to consume vcpu_id
without having to worry about cross-vCPU clobbering (at least s390 and x86
have had related bugs).
- Zero a vCPU's entry in VMX's Posted Interrupt Descriptor table used for IPI
virtualization when the vCPU is freed to fix a use-after-free where hardware
will write to a freed vCPU's PID.
1. Advertise already-supported capabilities.
2. Some bug fixes about timer and MMIO.
3. Some hardening about interrupt injection.
4. Replace kvm_err() with kvm_pr_unimpl().
5. Add FPU/LSX/LASX test cases for selftests.
-----BEGIN PGP SIGNATURE-----
iQJKBAABCAA0FiEEzOlt8mkP+tbeiYy5AoYrw/LiJnoFAmp9ca4WHGNoZW5odWFj
YWlAa2VybmVsLm9yZwAKCRAChivD8uImelAuD/4jhprQWR9b8RMQ9bGYaBDml+98
fCRW8NPyicnct+97U/E9EXJrrMyyDOqJtOSnNQOVmTjz3IGQtzLoHULaJPqUv0Dh
7xUbpmb7qkqxrykUEUa81tDnClpISfLcgUBWKsg0VGLyF7MIEnbtECTadbzQEd10
hNXy/Pv8OiAIZe4rtlGWxOU5JxU6OVf0zEmXiS6cbqXrwxVl4STbF97/5ULqjU+D
4KMMs1IcOdf4klQeoE1oKvWsCiedZQBkKbtsv2hpDMmY4lUdheJg9+0atr8Oh7VS
TtlO6Fmt7ilK9eFOCapDcs4gf5WQgs8WXKAyVUYSoteyEXQ9kSF2bHMYiC+WaKhb
kMRR9ZyJa086FwDvW9/TyMrP8HLR4ORa7ksOeBnGt4Uv9A92ZbGqvEqLILPoKsIR
LBJUM17XuLlR9J56sYNtid3Rdq88VLo84Wfc0hW966b+6oNdOVQBPI2YRpk4wDT1
Mtra60dtzwQ3jCW/GTjJThueEC4lqOFKHpYPIP3rd4aL8oCvfgLYlYZ4TiXHUjpX
MgQ9rm8nDZAWipsjvt2yADFnv3YW4kD6hrn1S0EaDJXsTtT7pB6Mr0jlxQE/ZJdA
ibXSubO5cakIdpMnAylL78ZyGZOWn6YOSG1pEjHjkI+in73sZ5a89k4s5ErxKxtB
Hnc/FXwVWbylUdJkKw==
=eMRy
-----END PGP SIGNATURE-----
Merge tag 'loongarch-kvm-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson into HEAD
LoongArch KVM changes for v7.3
1. Advertise already-supported capabilities.
2. Some bug fixes about timer and MMIO.
3. Some hardening about interrupt injection.
4. Replace kvm_err() with kvm_pr_unimpl().
5. Add FPU/LSX/LASX test cases for selftests.
KVM: s390: Features and Fixes for 7.3
- merged kvms390/master to pick up additional fixes that came too late
for 7.2
- Fixes for vfio-ap
- Fixes for the gmap rework
- Fixes for vsie
- AI triggered fixes all over
- diag9c tracing
- code move preparation for the additional arm64 support
- enable CONTEXT_ANALYSIS
- update to vfio maintainer file location
KVM/riscv changes for 7.3
- Svadu/Zicfiss/Zicfilp FWFT support for Guest
- Use try_cmpxchg for IMSIC MRIF RMW
- More arch-specific tracepoints in KVM RISC-V
- Eager Page Splitting for KVM RISC-V
- Optimize hfence request handling for SMP Guests
- Improve dirty log clearing by skipping zero bits in mask
- Guard HFENCE range loops against overflow
- CPU PM notifiers in KVM RISC-V for non-retentive idle states
- Fix kernel-mode vector context save/restore for Guest
Running vector workloads like perf + mcf on KVM can result in an
unexpected termination due to a vtype corruption. This happens because
the kernel-mode vector (KMV) misattributes the guest's vcpu context as
the user's context and source from a wrong status.VS.
The simplified call chain that results in this problem is shown as
follow:
__riscv_sys_ioctl()
kvm_arch_vcpu_ioctl_run()
kvm_riscv_vcpu_exit()
kvm_riscv_vcpu_sbi_ecall()
kvm_riscv_vcpu_pmu_ctr_stop()
kvm_vcpu_write_guest()
__copy_to_user()
enter_vector_usercopy()
kernel_vector_begin()
kernel_vector_begin() should use the sstatus.VS from guest's vcpu
context instead of task_pt_reg(current). Also, it should not save
guest's v-reg into the user's context memory.
To resolve this, the vcpu context must be correctly saved when KMV is
serving a guest. However, invoking KVM functions directly from generic
RISC-V architecture code introduces a reverse dependency, breaking
builds when KVM is configured as N or M.
Address this by registering an RCU-protected callback for context
flushing. KVM registers this callback at module initialization and
unregisters it on exit. When KMV starts a kernel context, it can now
safely flush the vector context via the callback.
Fixes: ecd2ada8a5 ("riscv: Add support for kernel mode vector")
Signed-off-by: Andy Chiu <tchiu@tenstorrent.com>
Reviewed-by: Yong-Xuan Wang <yongxuan.wang@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260803215250.824417-4-tchiu@tenstorrent.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Similar to commit 7137a203b2 ("arm64/fpsimd: Permit kernel mode NEON
with IRQs off"), we are upgrading get/put_cpu_vector_context such that
kvm_arch_vcpu_load/put can be safely called under both irq off and
regular process context.
Also, export both symbols so the kvm module can call into it.
Signed-off-by: Andy Chiu <tchiu@tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260803215250.824417-3-tchiu@tenstorrent.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Refactor riscv_v_start_kernel_context() to drop `is_nested` variable and
simplify the logic.
This introduces no functional change and works as a preparatory patch for
the kernel-mode vector fix.
Signed-off-by: Andy Chiu <tchiu@tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260803215250.824417-2-tchiu@tenstorrent.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Guard guest prefix handling behind `KVM_S390_MANAGES_S390_GUEST`. This
enables other KVM implementations to use gmap without implementing
prefix handling. The prefix handling is integrated deeply in the gmap
implementation. Therefore, provide safe default implementations for the
guarded functions.
No functional changes.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Guard guest CMMA behind `KVM_S390_MANAGES_S390_GUEST`. This enables
other KVM implementations to use gmap without implementing CMMA.
No functional changes.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Guard guest storage key handling behind `KVM_S390_MANAGES_S390_GUEST`.
This enables other KVM implementations to use gmap without implementing
storage key infrastructure. Define KVM_S390_MANAGES_S390_GUEST to 1 for
KVM hosts managing s390 guests (in kvm_host_s390.h). A KVM
implementation not implementing those features must define
KVM_S390_MANAGES_S390_GUEST to 0 in its kvm_host_<guest_arch>.h.
No functional changes besides the new guard.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Refactor gmap code such that a second s390 (host) KVM implementation can
use the gmap code as well. Move mmu code from s390 to gmap so the other
KVM implementation can use it as well.
No functional change.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Move PGM code definitions from kvm_host_s390.h (back) to the generic
kvm_host.h. These definitions are needed by multiple KVM implementations
and should be in a shared location.
No functional change.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Move all the code required to run s390 KVM guests on s390 to a s390
subdirectory. Move gmap related code into a gmap directory to later
share gmap code between KVM implementations.
Update S390 VFIO-PCI MAINTAINERS filepath.
No functional change.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Rename kvm_host.h to kvm_host_s390.h and kvm_host_types.h to
kvm_host_s390_types.h to distinguish s390-specific KVM definitions
from the generic kvm_host.h that will be used for shared definitions.
No functional change.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Rename kvm-s390.c to s390.c and kvm-s390.h to s390.h for consistency
with the new directory structure. Update all include statements and
simplify Makefile ccflags from explicit paths to -I$(src).
No functional change.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Update include guard names in dat.h, faultin.h, and gmap.h to
ARCH_KVM_GMAP_* to reflect their upcoming relocation to a shared gmap
directory.
No functional change.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Move the kvm_s390_major_guest_pfault trace event from trace.h to a new
trace_gmap.h header. This separates gmap-specific tracing from general
KVM/s390 tracing, preparing for code sharing between multiple KVM
implementations.
The trace event definition is updated to use local defines for
parameters so that they can be replaced later with ease for when another
KVM implementation uses these traces.
No functional change.
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
TLB Invalidation by Range has a fairly powerful way of encoding pretty
large ranges in a small number of bits. This range can be based on an
arbitrary VA, which means it is pretty easy for a guest to generate an
overflow should the hypervisor be naive enough to add the range to the
base...
Make sure the range is capped to the limit dictated by the address bit
that determines the VA range. For an IPA invalidation, this is further
corrected down the line to ignore the upper range.
Fixes: 4ffa72ad8f ("KVM: arm64: nv: Add S1 TLB invalidation primitive for VNCR_EL2")
Reported-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://lore.kernel.org/r/yifz3wn5gk5sr6mapi32trgk5m5kp33bquctsjmkifebnsnndt@fix6u4rthx4g
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Reviewed-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://patch.msgid.link/20260810170616.746100-1-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
Register a CPU_PM notifier to restore hypervisor CSR content during CPU
non-retention idle states. When a CPU enters a deep idle state that
powers off the CPU domain, hypervisor CSRs and VS CSRs lose their state
and must be saved before entry and restored after exit.
This completes KVM's power management coverage for RISC-V:
- CPU hotplug: handled by kvm_online_cpu/kvm_offline_cpu (cpuhp callbacks)
- System suspend: handled by kvm_suspend/kvm_resume (syscore ops)
- CPU idle (retention): no action needed, CSRs are retained
- CPU idle (non-retention): handled by this CPU_PM notifier
Signed-off-by: Yong-Xuan Wang <yongxuan.wang@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260810-kvm-cpu-pm-v5-1-6c9e4c95caf5@sifive.com
Signed-off-by: Anup Patel <anup@brainfault.org>
In some circumstances, one bit of kernel stack could have been leaked
from dat_cond_set_storage_key().
Fix by clearing prev before use.
Fixes: 8e03e8316e ("KVM: s390: KVM page table management functions: storage keys")
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812104436.109741-9-imbrenda@linux.ibm.com>
kvm_s390_clear_pv_state() needs to also clear the dumping flag, to
allow the protected VM to be started again (as non-protected, with all
protected state safely destroyed) after a forced reboot while a
protected dump was ongoing and not completed.
Fixes: e40df9efd6 ("KVM: s390: pv: clear the state without memset")
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812104436.109741-8-imbrenda@linux.ibm.com>
When __inject_sigp_stop() is called for a Stop and Store Status
operation, if the vCPU is running, the interrupt is marked as pending
and the status is stored by the thread performing the KVM_RUN IOCTL.
If the vCPU is already stopped, the status is stored immediately.
Storing the status means writing into userspace, which might fault, and
__inject_sigp_stop() is called from do_inject_vcpu() which in turn is
always called holding a spinlock, which is obviously an issue.
Fix this by returning -EWOULDBLOCK from __inject_sigp_stop(), and
adding a bool flag to indicate whether a store status is needed. The
callers of do_inject_vcpu() are modified to pass the pointer to the
bool flag; whenever a Store Status operation is needed, the callers can
now perform it outside the spinlock.
Opportunistically refactor kvm_s390_set_irq_state() to use
scoped_guard() and __free().
Fixes: 6cddd432e3 ("KVM: s390: handle stop irqs without action_bits")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
[ Added Fixes tag while picking -- Claudio ]
Message-ID: <20260812104436.109741-7-imbrenda@linux.ibm.com>
In case of failure, pgste_get_trylock_multiple() will attempt to unlock
the locked PGSTEs based on whether the PCL is set. In some
circumstances this can lead to unlocking PGSTEs that were locked by
other threads.
Fix by unlocking the amount of PGSTEs that were actually locked,
ignoring the PCL bit in the array.
Fixes: 94fd9b16cc ("KVM: s390: KVM page table management functions: lifecycle management")
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Christoph Schlameuss <schlameuss@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812104436.109741-6-imbrenda@linux.ibm.com>
When attempting to report all pending floating interrupt to userspace,
the GISA IPM bits are atomically tested and cleared, and the
corresponding interrupt description is written in the output buffer. If
the output buffer is too small, an error is returned to userspace, but
the GISA IPM bits are now lost.
Moreover, the contract of KVM_DEV_FLIC_GET_ALL_IRQS, which is the only
path to get_all_floating_irqs(), states that:
> All interrupts remain pending, i.e. are not deleted from the list of
> currently pending interrupts.
Fix by non-destructively testing for the GISA IPM bits.
Fixes: 24160af6cb ("KVM: s390: add GISA interrupts to FLIC ioctl interface")
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812104436.109741-4-imbrenda@linux.ibm.com>
kvm_arch_vcpu_unlocked_ioctl() is called without further locks held, but
kvm_s390_inject_vcpu(), which is called from there, needs either the
kvm->srcu or the slots lock.
Fix by taking the kvm->srcu in kvm_arch_vcpu_unlocked_ioctl().
Fixes: ba5c1e9b6c ("KVM: s390: interrupt subsystem, cpu timer, waitpsw")
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Christoph Schlameuss <schlameuss@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812104436.109741-3-imbrenda@linux.ibm.com>
Some callers pass NULL as oldkey. Calling page_cond_set_storage_key()
will cause that NULL pointer to get dereferenced.
Fix by checking for NULL and assigning the pointer to a dummy local
variable to avoid crashes.
Fixes: 8e03e8316e ("KVM: s390: KVM page table management functions: storage keys")
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Christoph Schlameuss <schlameuss@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812104436.109741-2-imbrenda@linux.ibm.com>
kvm_sigset_activate() installs vcpu->sigset via sigprocmask() and
stashes the caller's mask in current->real_blocked; only
kvm_sigset_deactivate() restores it.
For KVM_RUN on a STOPPED vcpu the error path will not restore the
userspace mask. Re-arrange the error handling to also restore the
signal mask.
Fixes: 6352e4d2dd ("KVM: s390: implement KVM_(S|G)ET_MP_STATE for user space state control")
Cc: stable@vger.kernel.org
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260811153738.206885-5-borntraeger@linux.ibm.com>
The rc/rrc value is copied to the cmd location of the cmd in the kvm_pv_cmd
structure. Fix the offset.
Fixes: 8aba09588d ("KVM: s390: Add CPU dump functionality")
Cc: stable@vger.kernel.org
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260811153738.206885-4-borntraeger@linux.ibm.com>
When shadowing crypto access bits from a format0 apcb (crycb 0 or 1),
the bits 64..255 are unchanged from whatever is in the vsie page in the
crycb and thus in the apcb. This gives a nested guest potential access
to a device no longer available. Zero out the remaining bits.
Fixes: 6b79de4b05 ("KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2")
Cc: stable@vger.kernel.org
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260811153738.206885-3-borntraeger@linux.ibm.com>
Every other dat_* consumer in this file (kvm_s390_get_skeys,
set_skeys, get_cmma_bits, set_cmma_bits, MEM_CLR_CMMA,
kvm_s390_fixup_prefix, kvm_test_age_gfn, kvm_age_gfn) reads
kvm->arch.gmap->asce *inside* the mmu_lock read-side. keyop is the only
outlier.
gmap->asce is mutated under write_lock(mmu_lock) by gmap_set_limit()
and keyop might use a stale asce value for walking as KVM_S390_KEYOP
and KVM_S390_VM_MEM_LIMIT_SIZE can run concurrently. This can result
in memory corruption.
Fixes: 0ee4ddc164 ("KVM: s390: Storage key manipulation IOCTL")
Cc: stable@vger.kernel.org
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260811153738.206885-2-borntraeger@linux.ibm.com>
As per the ARM GICv3 spec, the maintenance interrupt identified by
ICH_MISR_EL2.NP is asserted when it is enabled and no List register is
in pending state. This is further described in the document as "no List
registers with the State field set to 0b01 (pending)". By checking only
the pending bit of the LR (bit 62), KVM currently asserts the MI when
there are no LRs in "pending" or "pending and active" states.
Fix the detection logic to consider only the "pending" state.
Cc: stable@vger.kernel.org
Fixes: 96c2f03311 ("KVM: arm64: nv: Plumb handling of GICv3 EL2 accesses")
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260810102923.2426475-2-kajetan.puchalski@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
