-----BEGIN PGP SIGNATURE-----
iQJRBAABCgA7FiEE+soXsSLHKoYyzcli6rmadz2vbToFAmq/0m8dHGFsZXhlaS5z
dGFyb3ZvaXRvdkBnbWFpbC5jb20ACgkQ6rmadz2vbTpVWxAAgk0avmO+opg/LpC6
vQ6kQO743IkD1uc+jnGcRbOSo1YftUu5W1HY7hpj75MAdzSuScN3sflWyFb8Ng7l
AG6HH9drhOIGzEMzc1847fDqM8BYO4DY72ofM/c6tAqlaO6v5EbT2zW/kXm8PPtS
sRE60PP0lMh4NCuD2cH9X19GcR44CQXDijioW91hWAg8jxbGKgz9Skw6W+RHqIrH
Gi5pkg7OV4Qz5+/XuLpJ0Sd7df8/LMtzAi+B6EG8EfldvkuSMT1vNyS04IKXjGN2
1BWyUhWuIuRs/ReJ7PbkIa+Byc5ToiMmwozLR8X0M2k5iI6lGSIxdjaAUN0W8sIn
fk0aIIHhXa7L5Kf/6sOpLX3zESx5f/i+/7GdMFLYRkXiQTLsMBi2lqxvz++wxxak
s/MzJ0luOrbYrU53ufb1tNpwX97mYpvu8YLu7PadfEuQuGf4g1yhrjYAu7wltOCo
vPOOCm8SSm+WW0BTNkjET7acYP4nXNHxnAx6KI1BPNMVMBa/C/b7PH7BrQhkkIfp
Fq1cXL2FFTRLr+CxhU9oDHfTekvSrY58OZG+lputgFSy4JMXahoioXImGWw2TaGP
00ExdJtA5lsTeskNuGoKDT0pZHWpgi0QlbGzdKWq4Qqv2fEOX5PWhgvnrucZ6+Ob
+aH1vPjhom3qsWPxLKFKW/UZsN8=
=+6qx
-----END PGP SIGNATURE-----
Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf
Pull bpf fixes from Alexei Starovoitov:
- Fix overflow of backward jump offset in constant blinding
(Alexei Starovoitov)
- Fix packet range of packet pointers sharing an id when var_off
tightens umax of one pointer and not the other (Alexei Starovoitov)
- Fix objects stuck in free_by_rcu_ttrace list of bpf memalloc
(Alexei Starovoitov)
- Fix use-after-free of progs detached from busy trampolines: wait for
an RCU tasks grace period before freeing trampoline progs, and patch
detached progs out of trampoline images that are still in use
(Florent Revest)
- Hold map BTF for the memory allocator destructor record to fix UAF in
deferred bpf_mem_alloc destruction (Kumar Kartikeya Dwivedi)
- Fix missing migration protection in resizable hashtab
lookup_and_delete batch operation (Ömer Mete Kaya)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
bpf: Fix missing migration protection in __rhtab_map_lookup_and_delete_batch()
selftests/bpf: Add a test for objects stuck in free_by_rcu_ttrace
bpf: Fix objects stuck in free_by_rcu_ttrace
bpf: Factor out __do_call_rcu_ttrace()
selftests/bpf: Test packet range of pointers sharing an id
bpf: Fix packet range of pointers sharing an id
selftests/bpf: Detach a trampoline prog while a task sleeps before it
bpf: Skip detached progs in trampoline images that are still in use
bpf: Wait for an RCU tasks grace period before freeing trampoline progs
bpf: Hold map BTF for the memory allocator destructor record
bpf: Fix overflow of jump offset in constant blinding
-----BEGIN PGP SIGNATURE-----
iQJPBAABCAA5FiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmq/uGEbFIAAAAAABAAO
bWFudTIsMi41KzEuMTIsMiwyAAoJEEn8cBKl3gOumEgQAK0gjA0n40rHWGlmgIJO
WTqMZGcbTvVXpPRyEXuNE7tTRsDDopen4UnmDqDGafXkBO1vU1LaxN72JonPjp7O
sg4EqCi0iciuiDvLrjwQ1oPtpkL/feyjcNuNzSH/h2E4hJCUcEnyhTpeaRHCm3t4
IGDXqqYYqjAQV4N6dWrzwN5K2oM75IOzccr0hyROkN/DBrqNZtupOco725MKxSsr
FGTgoh0a6GlN3Yp+AIjqaHcPca/aUZApPUKmBzoWeqZmSqAmpzUIJgaHsBZAQS88
oMcSh71Fzi2tOepiazjaAK9nZC3Nh2xeRU8cwUtvLvluzYA6wnBhcuTBOqThJmp3
gEYz8yPbvil0aDIhfXB3xW2dxxsHaJrKJDLz0Smhi43D2lLKteQT3BE/2jQQ7OSa
6FcV905hys3lIxlHeCa2wE/7NgiNAy3qE1aazkIKRfI8dRJxIuhogvW3VpKGCAue
TwvjfvuoVtkaPAW/gLyDGRsMzrUvGPFa8yVvP5JTPrYQh5P+bEIM8CozWO2ScVcH
Cdu0/TDe9TTS5mIwh0CrSX3shV/80bvHTINL14sTAKBGtN0N0vJgA8NSdIYt8OaF
8qDDMg/XV5LY7sbMB3/Kru//wyfUw4LJdYhZSIJ0tZvzaRRIRiZGL1lT+X12PCrq
uoaJ4AoDCf/ZiDaVTXEVf+28
=GmSm
-----END PGP SIGNATURE-----
Merge tag 'random-7.3-rc6-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/crng/random
Pull random number generator fixes from Jason Donenfeld:
- VMGENID memory needs to be mapped with the decrypted tag, so that
SEV-SNP machines can boot
- A fix for an initialization race in VMGENID, followed by a cleanup
- Trivial kernel doc cleanups in siphash and random.c
- A fix for a new compilation failure with recent clang on PPC and
RISC-V, due to generating an out-of-line memset in the vDSO
* tag 'random-7.3-rc6-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/crng/random:
random: vDSO: avoid call to memset() when zeroing reserved parameter
random: fix vgetrandom_opaque_params kernel-doc
random: vDSO: fix repeated word 'to' in comment
siphash: clean up kernel-doc comments
virt: vmgenid: move to using dev_set/get_drvdata
virt: vmgenid: set driver_data before registering notification handlers
virt: vmgenid: remap memory as decrypted
After a recent change in LLVM [1], builds with the random vDSO
implementation, such as PowerPC and RISC-V, fail when checking the vDSO:
arch/powerpc/kernel/vdso/vdso32.so.dbg: dynamic relocations are not supported
arch/riscv/kernel/vdso/vdso.so.dbg: dynamic relocations are not supported
memset() is now generated when zeroing params->reserved for some builds
because LLVM has an optimization (now run in more instances) that can
recognize at compile time when it is assigning a static value to a
contiguous area of memory and turn that into a call to memset(). Both
clang and GCC assume memset() is always available [2].
Clang has an internal fiddly hook, -max-store-memset, which we can set
to a high number, to disable generating out of line memset calls [3].
Similarly, GCC has -finline-stringops=memset to do the same [4], should
this issue ever hit future version of GCC. While these options wouldn't
make sense for normal kernel code, it is fine for the extremely limited
and intentionally compact vDSO code.
Link: 90cebef141 [1]
Link: https://gcc.gnu.org/onlinedocs/gcc-16.2.0/gcc/Standards.html#index-ffreestanding [2]
Link: b28eeb28be [3]
Link: https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-finline-stringops [4]
Closes: https://github.com/ClangBuiltLinux/linux/issues/2183
Cc: stable@vger.kernel.org # v6.12+
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
- Take a reference on the last IRQ loaded into an LR to prevent it
from being freed while running the guest (Marc Zyngier)
- Ensure that the ITS MOVALL command only affects LPIs that were
previously affined to the source redistributor (Marc Zyngier)
- Fix + test for honoring the host's trap configuration when running
non-protected VMs while KVM is in protected mode (Fuad Tabba)
- Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
stage-2 (Mostafa Saleh)
x86:
Various bugfixes where the guest could do stupid things on purpose to
cause problems in the host:
- Failed VMRUNs can cause pending TLB flushes to be dropped, and in
general some actions done through VMCB control fields have to be redone
if VMRUN fails
- Toggling MSR interceptions or eVMCS execution controls can cause the
host to use a stale MSR permission bitmap
- Bad page tables can cause a WARN.
Also fix issues in last week's pull request (my fault, for changing
email workflow and thus missing feedback sent to kvm@ but not LKML).
Generic:
- Take kvm_lock when creating vCPUs. For almost two decades everybody
thought it was not done for some unspecified performance reasons, but
in reality it was only done because kvm_lock was originally a spinlock.
This is a better fix than 97d65b544f ("KVM: Check for duplicate vcpu_id
as early as possible", from the 7.3 merge window), and does not waste
2K per VM, hence its inclusion here.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmq+lW0UHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroPUbAf/Vap8HMmHlfQ0/A+25CZtJjWNOuNM
Zj3YMFLGjiqTHTVjWafWCpvS2kWdV/MyTcgFxEKyx/QJ7cxofLAHCtak5TYoUyJ7
oUQaHf2eK6VdtA1yz7yBf0zQSpKZhOoF3Ze2kQCUaR1ZEHm3apcFeh7jnjFammdx
YpeHm1ERKJW9KQNc4KXH9dIx8IGjOW5C6hAcxgZKUeTy+5oz7FMOqPQc/wNowOG5
ldxzSDa+BoHZ/okOkeBWgTfYFPH0MU3CO/ZX9+eJ+HIIEZ+XkVzo3h8KBpmWPTqj
dAksp8oZeg9B9XLzxklCAnpJaoNQHhQlQJAwJMivwwNwT99drizphQY7Bw==
=1X9h
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm fixes from Paolo Bonzini:
"The most intrusive change is reverting a commit from 7.3-rc1 that made
struct kvm a bit too large, and fixing the same issue otherwise.
There are again a lot of selftests lines; the sheer number of commits
is not small but I don't expect much more for 7.3 due to people
travelling to Plumbers next week.
ARM:
- Take a reference on the last IRQ loaded into an LR to prevent it
from being freed while running the guest (Marc Zyngier)
- Ensure that the ITS MOVALL command only affects LPIs that were
previously affined to the source redistributor (Marc Zyngier)
- Fix + test for honoring the host's trap configuration when running
non-protected VMs while KVM is in protected mode (Fuad Tabba)
- Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
stage-2 (Mostafa Saleh)
x86:
Various bugfixes where the guest could do stupid things on purpose to
cause problems in the host:
- Failed VMRUNs can cause pending TLB flushes to be dropped, and in
general some actions done through VMCB control fields have to be
redone if VMRUN fails
- Toggling MSR interceptions or eVMCS execution controls can cause
the host to use a stale MSR permission bitmap
- Bad page tables can cause a WARN.
Also fix issues in last week's pull request (my fault, for changing
email workflow and thus missing feedback sent to kvm@ but not LKML).
Generic:
- Take kvm_lock when creating vCPUs. For almost two decades everybody
thought it was not done for some unspecified performance reasons,
but in reality it was only done because kvm_lock was originally a
spinlock.
This is a better fix than 97d65b544f ("KVM: Check for duplicate
vcpu_id as early as possible", from the 7.3 merge window), and does
not waste 2K per VM, hence its inclusion here"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (29 commits)
KVM: arm64: Use stage-1 leaf size for VM_PFNMAP
KVM: arm64: selftests: Check a feature hidden in an ID register is UNDEF
KVM: arm64: Use the host's HCR_EL2 for non-protected VMs in pKVM
KVM: arm64: Clear HCR_EL2.RW for 32-bit non-protected vCPUs
KVM: arm64: Apply the fine-grained UNDEFs without FEAT_FGT
KVM: arm64: vgic-its: Fix MOVALL handling of source redistributor
KVM: arm64: vgic: Take a refcount on IRQs referenced by last_lr_irq
KVM: arm64: vgic: Allow last_lr_irq to be NULL when LRs are not overflowing
KVM: SEV: Do cache maintenance on the source VM *before* clearing SEV state
KVM: SEV: Nullify "have run CPUs" mask pointer when freeing it
KVM: selftests: Extend nested x2APIC test to validate using eVMCS for vmcs12
KVM: selftests: Extend nested x2APIC test to validate disabling x2APIC virt
KVM: selftests: Verify that L0's TPR doesn't get clobbered
KVM: selftests: Run the nested x2APIC with and without APICv being inhibited in L2
KVM: selftests: Add x2APIC MSR test for inhibiting APICv while nested
KVM: nVMX: Force MSR bitmap refresh if runtime eVMCS controls are modified
KVM: SVM: Use the active VMCB's MSR bitmap when checking if MSR is intercepted
KVM: SVM: Sync guest's PERF_CNTR_GLOBAL_CTL from h/w only on successful VMRUN
KVM: SVM: Don't mark ASID fields as dirty when setting control.tlb_ctl
KVM: SVM: Update control fields on #VMEXIT if and only if VMRUN succeeded
...
bpf_tramp_image_put() makes sure a trampoline image is not freed while
a task may still be running in it, but nothing similar is done for the
progs called by that image. Detach drops the last prog reference right
away and the prog is freed after grace periods, on the basis that a
task still in the traced function skips the fexit progs once the nop at
ip_after_call is patched to a jump.
That leaves out a task sleeping in a sleepable prog that runs before
the detached one, which no grace period waits for:
CPU 0 CPU 1
in image I, sleeping in prog S
detach P from I's trampoline
-> new image, bpf_tramp_image_put(I)
bpf_prog_put(P), last ref
grace periods, P freed
back from S
__bpf_prog_enter(P)
call P->bpf_func
If S and P are fexit progs the task is already past the patched jump,
and fentry only images don't have one. On x86 this is an int3 in
poisoned bpf_prog_pack memory:
Oops: int3: 0000 [#1] SMP NOPTI
CPU: 18 UID: 0 PID: 94573 Comm: x169 Not tainted 6.18.44 #1 PREEMPT(lazy)
RIP: 0010:0xffffffffc0601d8d
Call Trace:
<TASK>
? bpf_trampoline_6442515411+0x1a4/0x21b
bpf_lsm_bprm_committed_creds+0x5/0x10
security_bprm_committed_creds+0x5f/0x70
begin_new_exec+0x2d6/0x410
...
We hit this in production when progs attached through trampolines got
detached while their hooks were busy, and it was independently found
with a fuzzer and KASAN.
Have the JITs emit a patchable nop in front of each prog call sequence
and record it in the image. When a prog is detached, patch its nop to a
jump over the call sequence. Progs that stay attached keep running for
the tasks that are in the image, and ip_after_call isn't needed anymore.
The task can be in any image that isn't freed yet, not only in the
current one. It sleeps in image I1 that calls S, P and Q, then P is
detached and the trampoline moves to image I2, then Q is detached and
its call is still in I1. So the trampoline keeps a list of its images
until they are freed, and detaching a prog patches its nop in all of
them. Images hold a reference on the trampoline for that long.
On riscv and loongarch a jump of any range takes several instructions,
and a task preempted in the middle of them could resume into half of the
new sequence. The nop is a single instruction there, patched to a near
branch through arch_bpf_trampoline_skip().
With the extra nops, BPF_MAX_TRAMP_LINKS progs no longer fit in a page
on x86 and arm64 (and already didn't on powerpc), so lower the limit
there like s390 does.
Fixes: e21aa34178 ("bpf: Fix fexit trampoline.")
Reported-by: Sechang Lim <rhkrqnwk98@gmail.com>
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260926135605.1217928-3-florent.revest@linux.dev
Closes: https://lore.kernel.org/bpf/20260815071927.147049-1-zirajs7@gmail.com/
- Invalidate the ITS translation cache when the guest changes the
base address of the ITS tables (Fuad Tabba)
- Skip saving ITS devices with device IDs that are out-of-bounds
rather than failing the entire ITS save ioctl (Fuad Tabba)
- Close race between VM teardown and invalidations of nested MMUs
when handling MMU operations that are allowed to block
(Lorenzo Stoakes)
- Various fixes for the handling of the host's untrusted SVE
configuration in pKVM (Fuad Tabba)
- Make sure that empty SMCCC ranges based at 0 are rejected by the
kvm_smccc_set_filter() (Karl Mehltretter)
- Revoke the host mapping for pKVM's private stack pages, along
with a new sanity check that all mappings in the hyp's private
VA range have been correctly marked as hyp-owned (Fuad Tabba)
- Lifetime fixes for the array of shadow stage-2 MMUs, ensuring that
concurrent vCPU initialization cannot relocate in-use MMUs. Defer
the freeing of shadow stage-2 MMUs to the point that no other users
(e.g. MMU notifier) could reference them (Marc Zyngier)
- Drop useless WARN when rejecting an unsupported ioctl for pKVM
(Fuad Tabba)
- Fix the steal_time selftest to install correctly-sized mappings for
non-4K hosts (Sebastian Ott)
- Correct mapping of fine-grained trap for GCSPOPX instruction
(Mark Brown)
- Fix KVM_BUG_ON() due to missing handling of DBGBXVR<n> from 32-bit
guests (Karl Mehltretter)
RISC-V:
- Synchronize hrtimer during VCPU teardown.
- Fix the conversion between vsip and hvip values.
- Serialize IMSIC attributes with vCPU migration.
- Release unused page after MMU invalidation.
- Propagate interrupted G-stage faults to KVM user-space as EINTR.
- Fix nested acceleration hfence entry update order.
- Fix sdata leak and stale snapshot_addr in snapshot_set_shmem.
- Preserve firmware counter value across PMU counter stop/start.
- Report PMU snapshot write failure to the guest.
- Fix perf-backed counter accounting across PMU stop and read.
- Correctly propagate error of a hart status SBI call.
s390:
- Ensure that accesses through kvm_arch_set_irq_inatomic mark as dirty
the pages that contain indicator and summary bits.
- Fix compile warning for kvm_s390_update_cmma_dirty().
- Fix incorrect propagation of ENOENT from _gaccess_shadow_fault() to
userspace.
- Move s390_kvm_mmu_commit_memory_region() into
s390_kvm_mmu_prepare_memory_region() so that it can fail instead of WARN.
- Add missing srcu in kvm_s390_set_irq_state().
- Fix potential races in storage functions.
- Fix race in _destroy_pages_crste().
- Fix issues in the handling of KVM interrupt and page resources, when a
queue that is assigned to a mediated device (mdev) is removed from the
host's AP configuration.
- Fix loop condition in uv_find_secrets.
- Prevent potential out-of-bounds read.
x86:
- Fix a brown paper bag bug where KVM would incorrectly treat Intel PMU MSRs
as valid on AMD.
- Fix a regression in the hardware disable selftest where it checked the wrong
macro when detecting glibc support (breaks at least musl).
- Never clear KVM_REQ_VM_DEAD so that dead VMs stay dead, which is especially
important for KVM_BUG_ON() flows, which often guard more dangerous bugs.
- Re-pend GET_NESTED_STATE_PAGES if getting the pages fails, to fix a bug
where KVM would let userspace run a broken setup with stale vmcs12 pages.
- Fix a class of bugs where KVM would fail to fill kvm_run exit fields if
getting nested pages failed.
- Treat reserved entries in the memory attributes xarray as "no attributes",
to fix false positives when checking for mixed attributes.
- Fix memcg accounting for the memory attributes xarray (the xarray library
subtly requires the xarray to be configured for accounting upfront; the gfp
flags taken at runtime are used only rarely).
- Don't pre-reserve xarray entries when storing empty attributes, as storing
NULL must not require memory allocation (KVM and other subsystems heavily
rely on this behavior).
- Fix a memory leak and a cache maintenance issue related to doing intra-host
migration on an SEV guest.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmq3Ta0UHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroMcygf/Z8GovgUACbWs/PSgVxZr5++MsbnM
/N+vUP0VhsBxWCt191fBmKSBMNyTL734CKe0fkmisIAIPxf63uA3lttfMshNXKct
VJ3TQkbA7qiSEGf2Um1dp4k6egqDDbTS1a4NF7CM7AEL4Wm3YDfqv2h4rZicEBsd
Spy4yV8sKoiEHMnduhF2m5r0cWkrj194e2dolJy0p1WiNeEykf/nMTmNq5JWusBh
TDDzZf1p2BianOmQ6+fkzAfWDSIdv0OPmoH7iqvQ/xhvm4dLWq2oeOz5Q2/uWAp0
BqU4Z5zBrGRUWyXM0cDZEqBfmNH4v7oR9PfLgvD4qpMoqONuMj1+mIO68A==
=XtLD
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm fixes from Paolo Bonzini:
"Arm:
- Invalidate the ITS translation cache when the guest changes the
base address of the ITS tables (Fuad Tabba)
- Skip saving ITS devices with device IDs that are out-of-bounds
rather than failing the entire ITS save ioctl (Fuad Tabba)
- Close race between VM teardown and invalidations of nested MMUs
when handling MMU operations that are allowed to block (Lorenzo
Stoakes)
- Various fixes for the handling of the host's untrusted SVE
configuration in pKVM (Fuad Tabba)
- Make sure that empty SMCCC ranges based at 0 are rejected by the
kvm_smccc_set_filter() (Karl Mehltretter)
- Revoke the host mapping for pKVM's private stack pages, along with
a new sanity check that all mappings in the hyp's private VA range
have been correctly marked as hyp-owned (Fuad Tabba)
- Lifetime fixes for the array of shadow stage-2 MMUs, ensuring that
concurrent vCPU initialization cannot relocate in-use MMUs. Defer
the freeing of shadow stage-2 MMUs to the point that no other users
(e.g. MMU notifier) could reference them (Marc Zyngier)
- Drop useless WARN when rejecting an unsupported ioctl for pKVM
(Fuad Tabba)
- Fix the steal_time selftest to install correctly-sized mappings for
non-4K hosts (Sebastian Ott)
- Correct mapping of fine-grained trap for GCSPOPX instruction (Mark
Brown)
- Fix KVM_BUG_ON() due to missing handling of DBGBXVR<n> from 32-bit
guests (Karl Mehltretter)
RISC-V:
- Synchronize hrtimer during VCPU teardown
- Fix the conversion between vsip and hvip values
- Serialize IMSIC attributes with vCPU migration
- Release unused page after MMU invalidation
- Propagate interrupted G-stage faults to KVM user-space as EINTR
- Fix nested acceleration hfence entry update order
- Fix sdata leak and stale snapshot_addr in snapshot_set_shmem
- Preserve firmware counter value across PMU counter stop/start
- Report PMU snapshot write failure to the guest
- Fix perf-backed counter accounting across PMU stop and read
- Correctly propagate error of a hart status SBI call
s390:
- Ensure that accesses through kvm_arch_set_irq_inatomic mark as
dirty the pages that contain indicator and summary bits
- Fix compile warning for kvm_s390_update_cmma_dirty()
- Fix incorrect propagation of ENOENT from _gaccess_shadow_fault() to
userspace
- Move s390_kvm_mmu_commit_memory_region() into
s390_kvm_mmu_prepare_memory_region() so that it can fail instead of
WARN
- Add missing srcu in kvm_s390_set_irq_state()
- Fix potential races in storage functions
- Fix race in _destroy_pages_crste()
- Fix issues in the handling of KVM interrupt and page resources,
when a queue that is assigned to a mediated device (mdev) is
removed from the host's AP configuration
- Fix loop condition in uv_find_secrets
- Prevent potential out-of-bounds read
x86:
- Fix a brown paper bag bug where KVM would incorrectly treat Intel
PMU MSRs as valid on AMD
- Fix a regression in the hardware disable selftest where it checked
the wrong macro when detecting glibc support (breaks at least musl)
- Never clear KVM_REQ_VM_DEAD so that dead VMs stay dead, which is
especially important for KVM_BUG_ON() flows, which often guard more
dangerous bugs
- Re-pend GET_NESTED_STATE_PAGES if getting the pages fails, to fix a
bug where KVM would let userspace run a broken setup with stale
vmcs12 pages
- Fix a class of bugs where KVM would fail to fill kvm_run exit
fields if getting nested pages failed
- Treat reserved entries in the memory attributes xarray as "no
attributes", to fix false positives when checking for mixed
attributes
- Fix memcg accounting for the memory attributes xarray (the xarray
library subtly requires the xarray to be configured for accounting
upfront; the gfp flags taken at runtime are used only rarely)
- Don't pre-reserve xarray entries when storing empty attributes, as
storing NULL must not require memory allocation (KVM and other
subsystems heavily rely on this behavior)
- Fix a memory leak and a cache maintenance issue related to doing
intra-host migration on an SEV guest"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (54 commits)
KVM: SEV: Do cache maintenance on the source VM during intra-host migration
KVM: SEV: Free have_run_cpus during VM destruction even if VM is no longer SEV
KVM: Don't pre-reserve xarray entries when storing empty/NULL attributes
KVM: Ensure memory attributes xarray nodes are accounted to the caller's memcg
KVM: Don't treat reserved xarray entries as having memory attributes
KVM: x86: Fill kvm_run exit fields in common get_nested_state_pages() error paths
KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails
KVM: arm64: Fix AArch32 DBGBXVR<n> handling
KVM: arm64: Fix FGT mapping for HFGITR_EL2.nGCSEPP
KVM: selftests: fix steal_time for arm64 with host page size > 4K
KVM: arm64: Don't WARN on an unknown VM ioctl in protected mode
KVM: arm64: nv: Delay freeing of shadow S2 structures until VM destruction
KVM: arm64: nv: Fix life cycle of the nested_mmus array
KVM: arm64: Check every private mapping is hyp-owned at pKVM init
KVM: arm64: Move the private VA allocation cursor to __io_map_next
KVM: arm64: Match hyp text by physical address in fix_host_ownership()
KVM: arm64: Transfer the hyp stack pages out of the host stage-2
KVM: arm64: selftests: Test empty SMCCC filter range at base 0
KVM: arm64: Return -EINVAL for an empty SMCCC filter range at base 0
KVM: arm64: Derive GUEST_HAS_SVE from the SVE feature bit at EL2
...
When creating a vCPU, don't drop kvm->lock to when doing the bulk of actual
vCPU creation, as allowing multiple vCPUs to be created in parallel adds
significant complexity in KVM (as evidenced by the many related bugs), and
all known VMMs fully serialize vCPU creation. Remove all manually locking
of kvm->lock from kvm_arch_vcpu_{post,}create() for obvious reasons.
For many years, "everyone" has assumed that dropping kvm->lock was done for
performance reasons optimization, e.g. to allow userspace to create all
vCPUs concurrently for latency purposes. But as above, no known VMM does
that. Looking at the history of this code, before commit 11ec280471
("KVM: Convert vm lock to a mutex"), kvm->lock was a spinlock. I.e. KVM
*had* to drop kvm->lock when doing the bulk of vCPU creation, otherwise KVM
couldn't do normal memory allocations. When kvm->lock got turned into a
mutex for unrelated reasons, no one took advantage updated of the change to
simplify vCPU creation. And 19 years later, everyone just assumed that KVM
continued to deal with the complexity for performance reasons.
Furthermore, naively parallelizing vCPU creation in userspace is likely a
net negative due to the overheads of task creation. Unless a VMM carefully
avoids the extra overhead related to parallelization, e.g. spawns each
vCPU's thread before creating the vCPU, creating vCPUs concurrently is a
net *negative* up until about ~64 vCPUs, after which the times are a wash.
The absolute speed of light _is_ faster if KVM doesn't hold kvm-lock, but
at vCPU counts of ~16 or less, it's probably in the noise when considering
total VM creation time, as the added latency is less than 1ms up until 16
or so vCPUs.
On top of all that, KVM has had a *lot* of fatal bugs (most often found by
syzkaller) related to vCPUs being created while trying to do per-VM
operations (basically, see every flow that locks all vCPUs). I.e. the
parallel vCPU creation "support" is actively harmful as the only "use case"
is for misbehaving userspace to exploit KVM bugs.
Serializing vCPU creation will allow reverting commit 97d65b544f ("KVM:
Check for duplicate vcpu_id as early as possible"), which had "minor" math
error: the worst case scenario isn't "256 bytes per VM", it's "256 unsigned
longs per VM", i.e. 2048 bytes per VM, which doubles the size of each VM
and pushes several architectures into order-1 allocations.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Tested-by: Jean-Christophe Guillain <jean-christophe@guillain.net>
Tested-by: Naveen N Rao (AMD) <naveen@kernel.org>
Message-ID: <20260921174445.911676-5-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- Fix several bugs in PCI error recovery SCLP reporting: don't report
success on skipped recovery, report errors when no pdev is associated,
add missing device lock, and fix struct pci_dev reference leak in
zpci_report_status()
- Fix several bugs in CIO code: fix use of invalid SCHIB data, guard PMCW
field accesses, check device number valid bit in PMWC before accessing
other fields, and fix NULL pointer dereference in
ccw_device_get_util_str()
- Fix virtual vs physical address confusion in channel measurement
facility code on kernels with CONFIG_RANDOMIZE_IDENTITY_BASE=y
- Fix couple of bugs in s390dbf: fix copy of failed static debug areas,
skip view registration on failure, and reject NULL pointer in
debug_dump()
- Fix sriov_numvfs attribute name in zPCI documentation
- Fix typos in comments
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEECMNfWEw3SLnmiLkZIg7DeRspbsIFAmq2V1AACgkQIg7DeRsp
bsJ7zg/7BrnPs0WyKv9NZvVUudYbz3nUIAleJo6HcDK2SUDr+EX3W4U7OjsXL6Fv
H76hx5AqF6ddtUOJ2uevCAb3PKm0eA7vmK9YiERAclrBA+tWGW0xBwCqqyc84k38
CPxOBr87Ae1jIIcfDxfBWujtKTFi/4BpSvKvKgg3mnd2egny4UnHnABx5ZcClEee
p6aVjCKycQFHwNLMoJA1BxvGSpcXdiKXbm6VRJ4gCaBnVuzkLUv1ZMp/Pi1fWVbV
E1AEztZ/03Tfry51q7wQLv6wD5lR3Z/Py2vZ4YIbelQwVMAMI+rFtgd5oFfo9qkF
gcFJ+faE9m0E/nsoaaVCaa2Ja4+hV/vZ5ik0bq1ljEqEvInaa2XTuxNAljId351Y
JrLvCdaip0FkMNMqEym0iydMAlpc491GAhzNrT3yvq6t03VVpxprPPA6VIwHg+zk
UBMP+QdxtnaV8AAUzDD9E255d88N4DECBICUgf1XjFm/I7xLJFWyUvbzcqNn9VnY
EGVBOeQmoByhCL9lN0+nzEM1sgT+hQB/C3WmLqiuLxrxjnc8dVr8rVHcPXFgARPw
Ht3W9jxdTdl4VBwckOWmbvG4L6spXFLdEzI51cP46L1RKWhFjLl/6RoY8fUHf4Vz
YnHutQevSoPI2D6G4v8K1FBz9wclxCojBPbXCmevfbL9B4ChPoQ=
=r3uP
-----END PGP SIGNATURE-----
Merge tag 's390-7.3-4' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
Pull s390 fixes from Heiko Carstens:
- Fix several bugs in PCI error recovery SCLP reporting: don't report
success on skipped recovery, report errors when no pdev is
associated, add missing device lock, and fix struct pci_dev reference
leak in zpci_report_status()
- Fix several bugs in CIO code: fix use of invalid SCHIB data, guard
PMCW field accesses, check device number valid bit in PMWC before
accessing other fields, and fix NULL pointer dereference in
ccw_device_get_util_str()
- Fix virtual vs physical address confusion in channel measurement
facility code on kernels with CONFIG_RANDOMIZE_IDENTITY_BASE=y
- Fix couple of bugs in s390dbf: fix copy of failed static debug areas,
skip view registration on failure, and reject NULL pointer in
debug_dump()
- Fix sriov_numvfs attribute name in zPCI documentation
- Fix typos in comments
* tag 's390-7.3-4' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
s390/cio: Fix NULL pointer dereference in ccw_device_get_util_str()
s390/debug: Fix NULL pointer dereference in debug_info_copy()
s390/debug: Do not register views for failed static debug areas
s390/debug: Reject NULL debug info in debug_dump()
s390/cmf: Fix virtual vs physical address confusion
s390/pci: Don't report recovery success on skipped recovery
s390/pci: Report SCLP status on error events when no pdev is associated
s390/pci: Fix missing device lock in zpci_report_status()
s390/pci: Fix leak of struct pci_dev reference in zpci_report_status()
s390/cio: Guard PMCW field accesses with dnv check
s390/cio: Check pmcw.dnv before pmcw.ena in I/O entry points
s390/cio: Fix cio_update_schib() to not cache invalid schib
s390/pci/docs: Fix sriov_numvfs attribute name
s390: Fix typos in comments
When debug_register_static() fails, it clears areas, active_pages and
active_entries but leaves the area bounds unchanged. Copying such an
area, either by opening its view file or via debug_dump(), makes
debug_info_copy() dereference the NULL pointers.
Skip the copy loop when the source has no areas.
Closes: https://lore.kernel.org/r/20260903132123.12F271F00A3F@smtp.kernel.org
Fixes: d72541f945 ("s390/debug: add early tracing support")
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
__REGISTER_STATIC_DEBUG_INFO() calls debug_register_view()
unconditionally, even when debug_register_static() has failed. In that
case _debug_register() was never reached and id->debugfs_root_entry is
still NULL, so debugfs_create_file() places the view file in the debugfs
root directory. For sclp_err this leaves a /sys/kernel/debug/hex_ascii
file with nothing to indicate which debug log it belongs to.
debug_register_static() is not exported and the macro is its only
caller, so let it return an error code and skip the view registration
when it fails. No debugfs files are created for such an area then.
Reproduce by booting with s390dbf=sclp_err::100000000. The sclp_err
registration fails, no s390dbf/sclp_err/ directory is created, and a
hex_ascii file appears in the debugfs root instead.
Fixes: d72541f945 ("s390/debug: add early tracing support")
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
debug_dump() passes id on to debug_info_copy(), which dereferences
in->name unchecked. debug_unregister(), debug_set_size() and
debug_register_view() guard against a NULL id but debug_dump() does not.
Nothing reaches this today, but add the check for consistency.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260910110147.96E851F000FF@smtp.kernel.org/
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
When a PCI device is already in the permanent failure state, recovery is
skipped, but the SCLP recovery report still shows success. Fix this by
changing the status string to explicitly state that recovery was skipped
due to permanent failure.
Cc: stable@vger.kernel.org
Fixes: 4ec6054e73 ("s390/pci: Report PCI error recovery results via SCLP")
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Reviewed-by: Farhan Ali <alifm@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
With commit 4ec6054e73 ("s390/pci: Report PCI error recovery results
via SCLP") SCLP reports are generated when recovery is performed in
response to an error event. If such an error event arrives but no pdev
is currently associated with the zdev, e.g. because it was removed or
not yet probed, no report is generated. Fix this by generating a report
specific to an error event for a zdev without an associated pdev.
Cc: stable@vger.kernel.org
Fixes: 4ec6054e73 ("s390/pci: Report PCI error recovery results via SCLP")
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Reviewed-by: Farhan Ali <alifm@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
When pdev is non-NULL, zpci_report_status() accesses the device's driver.
To get a consistent state matching the recovery, the device lock needs to
be held. Do so by expanding the existing device lock critical section.
The lock only needs to be held when the pdev is non-NULL, so extract
the pdev-specific reporting into a helper function which also adds a
lockdep assertion to detect calls without the device lock held.
Cc: stable@vger.kernel.org
Fixes: 4ec6054e73 ("s390/pci: Report PCI error recovery results via SCLP")
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Reviewed-by: Farhan Ali <alifm@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
In zpci_report_status(), a reference to the pdev associated with the
zdev being reported about is acquired using pci_get_slot(). This
reference needs to be dropped with pci_dev_put(), but this call is
missing, thus leaking the reference. On subsequent hot unplug, this will
cause the struct pci_dev to not be released, leaking memory and
preventing reattach.
At the same time, the only existing caller already holds a pdev
reference. So instead of reacquiring and then dropping another reference,
simply pass the existing pdev pointer to zpci_report_status(). This gets
rid of the need for pci_get_slot() as well as the zdev->zbus check.
Cc: stable@vger.kernel.org
Fixes: 4ec6054e73 ("s390/pci: Report PCI error recovery results via SCLP")
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Reviewed-by: Farhan Ali <alifm@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
- Invalidate the ITS translation cache when the guest changes the
base address of the ITS tables (Fuad Tabba)
- Skip saving ITS devices with device IDs that are out-of-bounds
rather than failing the entire ITS save ioctl (Fuad Tabba)
- Close race between VM teardown and invalidations of nested MMUs
when handling MMU operations that are allowed to block
(Lorenzo Stoakes)
- Various fixes for the handling of the host's untrusted SVE
configuration in pKVM (Fuad Tabba)
- Make sure that empty SMCCC ranges based at 0 are rejected by the
kvm_smccc_set_filter() (Karl Mehltretter)
- Revoke the host mapping for pKVM's private stack pages, along
with a new sanity check that all mappings in the hyp's private
VA range have been correctly marked as hyp-owned (Fuad Tabba)
- Lifetime fixes for the array of shadow stage-2 MMUs, ensuring that
concurrent vCPU initialization cannot relocate in-use MMUs. Defer
the freeing of shadow stage-2 MMUs to the point that no other users
(e.g. MMU notifier) could reference them (Marc Zyngier)
- Drop useless WARN when rejecting an unsupported ioctl for pKVM
(Fuad Tabba)
- Fix the steal_time selftest to install correctly-sized mappings for
non-4K hosts (Sebastian Ott)
- Correct mapping of fine-grained trap for GCSPOPX instruction
(Mark Brown)
- Fix KVM_BUG_ON() due to missing handling of DBGBXVR<n> from 32-bit
guests (Karl Mehltretter)
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQSNXHjWXuzMZutrKNKivnWIJHzdFgUCaq7S2hccb2xpdmVyLnVw
dG9uQGxpbnV4LmRldgAKCRCivnWIJHzdFuyFAPwMTw52duiQXiJd2MLUe7qndEWW
s5vcAPDZIdDOWY4HewD/Xh7j5EwanNDM2DDXvdrgjnsTrwsJBW2ULIBKS6lvOA4=
=fF21
-----END PGP SIGNATURE-----
Merge tag 'kvmarm-fixes-7.3-1' of https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 changes for 7.3, take #2
- Invalidate the ITS translation cache when the guest changes the
base address of the ITS tables (Fuad Tabba)
- Skip saving ITS devices with device IDs that are out-of-bounds
rather than failing the entire ITS save ioctl (Fuad Tabba)
- Close race between VM teardown and invalidations of nested MMUs
when handling MMU operations that are allowed to block
(Lorenzo Stoakes)
- Various fixes for the handling of the host's untrusted SVE
configuration in pKVM (Fuad Tabba)
- Make sure that empty SMCCC ranges based at 0 are rejected by the
kvm_smccc_set_filter() (Karl Mehltretter)
- Revoke the host mapping for pKVM's private stack pages, along
with a new sanity check that all mappings in the hyp's private
VA range have been correctly marked as hyp-owned (Fuad Tabba)
- Lifetime fixes for the array of shadow stage-2 MMUs, ensuring that
concurrent vCPU initialization cannot relocate in-use MMUs. Defer
the freeing of shadow stage-2 MMUs to the point that no other users
(e.g. MMU notifier) could reference them (Marc Zyngier)
- Drop useless WARN when rejecting an unsupported ioctl for pKVM
(Fuad Tabba)
- Fix the steal_time selftest to install correctly-sized mappings for
non-4K hosts (Sebastian Ott)
- Correct mapping of fine-grained trap for GCSPOPX instruction
(Mark Brown)
- Fix KVM_BUG_ON() due to missing handling of DBGBXVR<n> from 32-bit
guests (Karl Mehltretter)
The API partial block handling requires a intermediate chaining
value (CV). The internal function hash_data() sets the function code
correctly, so also call cpacf_kimd() instruction for intermediate CV
generation, as cpacf_klmd() always generate the final hash value.
Cc: stable@vger.kernel.org # 6.15+
Fixes: 08811169ac ("crypto: s390/hmac - Use API partial block handling")
Signed-off-by: Holger Dengler <dengler@linux.ibm.com>
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When hardware persistently returns -EBUSY after exhausting retries,
the error propagates to crypto_finalize_*_request(). The crypto API's
completion wrapper treats -EBUSY as a queueing status and swallows it,
preventing the completion callback from firing. This causes callers
using crypto_wait_req() to block indefinitely.
Translate persistent -EBUSY to -EIO after retry exhaustion to ensure
proper error propagation and callback invocation.
Fixes: 6cd87cb5ef ("s390/crypto: Rework protected key AES for true asynch support")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.16+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When crypto_finalize_hash_request() or
crypto_finalize_skcipher_request() explicitly completes a request, the
do_one_request callback must return 0 to indicate successful
handling. Returning a negative error code causes the crypto engine to
assume the driver failed to take ownership and triggers a second
completion via crypto_request_complete(), resulting in a double
completion. This pattern occurs in paes_s390.c 4 times and once in
phmac_s390.c.
Fixed in phmac_do_one_request() and all four paes do_one_request
callbacks (ecb, cbc, ctr, xts) by returning 0 after explicit
finalization instead of propagating the error code.
Fixes: 6cd87cb5ef ("s390/crypto: Rework protected key AES for true asynch support")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.16+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When a request is transferred to the engine via
crypto_transfer_hash_request_to_engine() there are two return codes
signaling a successful transfer: EINPROGRESS and EBUSY. However the
correct handling of EBUSY was missing and has been added as a return
code indicating a successful transfer to the crypto engine.
Fixes: cbbc675506 ("crypto: s390 - New s390 specific protected key hash phmac")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.17+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When a request is transferred to the engine via
crypto_transfer_skcipher_request_to_engine() there are two return
codes signaling a successful transfer: EINPROGRESS and EBUSY. However
the correct handling of EBUSY was missing and has been added as a
return code indicating a successful transfer to the crypto engine.
Fixes: 6cd87cb5ef ("s390/crypto: Rework protected key AES for true asynch support")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.16+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The 4 algorithms implemented in paes_s390 never had any cra_flags
set. So add code which sets the cra_flag to CRYPTO_ALG_ASYNC and
CRYPTO_ALG_NO_FALLBACK.
Fixes: 4ccd065a69 ("crypto: ahash - Add support for drivers with no fallback")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.17+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The PAES CTR implementation used a mutex to lock one page of exclusive
memory for fast CTR processing. Unfortunately a mutex is not save to
use in atomic or interrupt context. So use a binary semaphore instead
which is save to use in such environments.
Furthermore rework the code to get rid of conditional locking. So
restructure the PAES CRT code by extracting the main loop into a
separate function and just give in information about the (locked) page
can be used or not (is not locked).
Fixes: 6cd87cb5ef ("s390/crypto: Rework protected key AES for true asynch support")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.16+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
In function ctr_paes_do_crypt() there is a buffer used to process
remaining bytes < AES_BLOCK_SIZE. This buffer was not scrubbed and
thus could lead to expose of unwanted data. Rework the code to
explicitly scrub the buffer at the end of the function to avoid
exposure of maybe sensitive data.
In function __xts_2keys_prep_param() change the existing scrub to
clean the whole param block instead of just the key field.
Fixes: 6cd87cb5ef ("s390/crypto: Rework protected key AES for true asynch support")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.16+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
All the 4 PAES cipher processing loops were not checking the return
value of skcipher_walk_done() immediately after calling it. This could
lead to error masking when both the walk operation failed and a
subsequent key conversion was needed (k < n condition).
Add immediate error checks after skcipher_walk_done() in all main
processing loops (ECB, CBC, CTR, XTS modes) to ensure walk errors are
properly propagated and not masked by subsequent operations.
With that comes a slight rework around the skcipher_walk_done()
invocation. It is now necessary to check if the walk has already been
finalized (walk->nbytes is then 0) or not to avoid double
de-allocation of resources held by the walk.
Fixes: 6cd87cb5ef ("s390/crypto: Rework protected key AES for true asynch support")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.16+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The AES CTR implementation used a mutex to lock one page of exclusive
memory for fast CTR processing. Unfortunately a mutex is not save to
use in atomic or interrupt context. So use a binary semaphore instead
which is save to use in such environments.
Furthermore rework the code to get rid of conditional locking. So
restructure the AES CRT code by extracting the main loop into a
separate function and just give in information about the (locked) page
can be used or not (is not locked).
Fixes: 7988fb2c03 ("crypto: s390/aes - convert to skcipher API")
Suggested-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 5.5+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
In function ctr_aes_crypt() there is a buffer used to process
remaining bytes < AES_BLOCK_SIZE. This buffer was not scrubbed and
thus could lead to expose of unwanted data. When the buffer is used
explicitly scrub it at the end of the code block to avoid exposure of
maybe sensitive data.
In a similar way the function gcm_aes_crypt() hat an error path where
the CPACF param block was not scrubbed. Instead of return early now
these error paths go to end of function where explicit scrubbing is
done. Similar with the buffers which are part of the gcm_sg_walk
structs from the variables gw_in and gw_out.
Fixes: d07f951903 ("crypto: s390/aes - Fix buffer overread in CTR mode")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 6.8+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The return codes from skcipher_walk_virt() were not properly checked
before entering the processing loops in ecb_aes_crypt() and ctr_aes_crypt().
If skcipher_walk_virt() fails, the walk structure may be in an undefined
state, and attempting to process data could lead to incorrect behavior
or accessing uninitialized memory.
Add proper return code checking to ensure correct handling of the walk
initialization and walk advance and eventually return to the caller
with that return code.
Fixes: 7988fb2c03 ("crypto: s390/aes - convert to skcipher API")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 5.5+
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Trace functions check for non-NULL id->areas without lock to minimize
overhead. This opens a race window where a NULL pointer dereference
occurs if id->areas is set to NULL (e.g. via echo 0 > ../pages) after
the check and before id->lock is taken.
Fix this by rechecking id->areas under lock.
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Commit a2cec68637 ("s390/debug: Add s390dbf kernel parameter") calls
debug_get_param() from both debug_info_create() and debug_set_level().
Since debug_get_param() emits the override notice unconditionally, and
drivers typically call debug_set_level() right after debug_register(),
the same line is printed twice per debug area:
s390dbf: 0.0.1234: override level to 6
s390dbf: 0.0.1234: override level to 6
For areas registered per device this is multiplied by the device count.
With 's390dbf=0.0.*:6' a system with many DASDs emits a large number of
redundant lines during boot.
Add a quiet parameter to debug_get_param() and pass quiet=true from
debug_set_level(), where the override has already been announced during
registration. The remaining callers keep printing the notice.
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Commit a2cec68637 ("s390/debug: Add s390dbf kernel parameter")
incorrectly removed a null-id check from debug_set_level(), introducing
a possible NULL pointer dereference for debug-API users that put
debug_register() results unchecked into debug_set_level().
Fix this by moving the check from the internal _debug_set_level()
variant back to the external debug_set_level() wrapper.
Fixes: a2cec68637 ("s390/debug: Add s390dbf kernel parameter")
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The command 'perf stat -e pai_crypto/CRYPTO_ALL/ -- <command>'
crashes the kernel when CPUs are hotplug added during that run.
Root cause is the missing allocation of per-CPU data structures
for that new CPU. The allocation is dynamic and the first
event that has task context creates such a structure for
each online CPU. This is not sufficient. CPUs may be offline
during event creation and can be set online during the
perf run time. For example commands
# echo 0 > /sys/devices/system/cpu/cpu1/online
# perf stat -e cycles -i -- stress-ng -t10s --matrix X
# sleep 1
# echo 1 > /sys/devices/system/cpu/cpu1/online
Currently without a CPU hotplug handler, that new CPU has no
per-CPU data infrastructure. The scheduler runs PMU call back
function pai_add() to install the PMU support for that CPU before
the task is being scheduled on that new CPU.
In pai_add() instructions
mp = this_cpu_ptr(pai_root[idx].mapptr);
cpump = mp->mapptr;
return a NULL pointer and the result is a kernel panic as variable
cpump is used inside that function.
Add CPU hotplug support for CPU add and delete and create
the necessary per-CPU data infrastructure during CPU hotplug
add processing. Same for CPU hotplug remove.
This is done when the CPU is offline to ensure the data structures
are available when CPU is made online and tasks are scheduled on it.
[hca@linux.ibm.com: fixup error path in pai_init()]
Cc: stable@vger.kernel.org # v6.19
Fixes: 582cc1b28e ("s390/pai_ext: Enable per-task and system-wide sampling event")
Fixes: 9f66572f28 ("s390/pai_crypto: Enable per-task and system-wide sampling event")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Move mutex locking from per CPU allocation to event allocation.
No functional change.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Use PAI PMU index value as function argument instead of pointer
to struct perf_event. Only that index value is used inside
functions pai_alloc_cpu() and pai_event_destroy_cpu().
No functional change.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci
This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.
Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.
Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
When the system has more than 85 secrets, the uv_secret_list struct
array only holds up to 85 items per page, resulting in an out of bounds
read in find_secret_in_page if the targeted secret is in the next page
or not stored at all.
Fix this by looping over the number of stored secrets which is the
per sub-list count of stored secrets and not the overall count.
Fixes: 7c9137af20 ("s390/uv: Retrieve UV secrets support")
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Christoph Schlameuss <schlameuss@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812-uv_secrets_fix-v3-2-a85bd29e0666@linux.ibm.com>
Systems with more than 85 UV secrets got -ENOENT for any secret past the
first page.
Fix this by setting the start index at the beginning of the loop in
uv_find_secret() and not at the end. First test if there are more
secrets left by comparing start_idx with list->next_secret_idx, and then
set the start index to the next secret index.
Fixes: 7c9137af20 ("s390/uv: Retrieve UV secrets support")
Acked-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Reviewed-by: Christoph Schlameuss <schlameuss@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260812-uv_secrets_fix-v3-1-a85bd29e0666@linux.ibm.com>
Use READ_ONCE() in _destroy_pages_crste() to read the crste, avoid
dereferencing the pointer multiple times.
Fixes: a2c17f9270 ("KVM: s390: New gmap code")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-9-imbrenda@linux.ibm.com>
When dat_cond_set_storage_key() finds a large page, it will
conditionally set the storage key in absolute memory using
large_crste_to_phys() to get the absolute address.
There is a race window between dat_entry_walk() and
large_crste_to_phys(): the large page could have been split
concurrently, and large_crste_to_phys() might be called with a crste
that does not designate a large page, leading to crashes.
Similar issues were also present in dat_set_storage_key().
dat_get_storage_key() and dat_reset_reference_bit() did instead check
for a potential concurrent splitting of the large page, but then
handled it incorrectly.
Fix by performing a READ_ONCE on the crste pointer, checking and using
the result, instead of dereferencing the pointer again. In case a race
is detacted, try dat_entry_walk() again.
Fixes: 8e03e8316e ("KVM: s390: KVM page table management functions: storage keys")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-8-imbrenda@linux.ibm.com>
Like kvm_s390_inject_vcpu(), kvm_s390_set_irq_state() also needs the
kvm->srcu or the slots lock when performing the Store status operation.
Fix by taking kvm->srcu in kvm_s390_set_irq_state().
Fixes: ba5c1e9b6c ("KVM: s390: interrupt subsystem, cpu timer, waitpsw")
Fixes: 062e44a931 ("KVM: s390: Use srcu in kvm_arch_vcpu_unlocked_ioctl()")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-7-imbrenda@linux.ibm.com>
Move all code from s390_kvm_mmu_commit_memory_region() into
s390_kvm_mmu_prepare_memory_region(). This allows the function to fail
gracefully if needed. The previous behaviour was to print a warning and
continue execution with page tables inconsistent with the memslots.
Fixes: e38c884df9 ("KVM: s390: Switch to new gmap")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-6-imbrenda@linux.ibm.com>
Refactor dat_set_slot(), _dat_slot_pte(), _dat_slot_crste(). Now they
only take a struct kvm_s390_mmu_cache as priv. For dat_delete_slot(),
mc is NULL, as no allocations should take place.
This is needed as a prerequisite to move gmap DAT table setup from
kvm_arch_commit_memory_region() to kvm_arch_prepare_memory_region().
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-5-imbrenda@linux.ibm.com>
In some circumstances, it is possible that the page of nested guest
memory that is being shadowed is not present at all in the parent guest
gmap. dat_entry_walk() will not find any leaf entry and return with
-ENOENT, which will erroneously be propagated all the way to userspace.
Fix by manually calling gmap_link() on the memory of the nested guest
that is being shadowed if the mapping was not already present.
Fixes: e38c884df9 ("KVM: s390: Switch to new gmap")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-4-imbrenda@linux.ibm.com>
The parameter "old" should be marked as const, to prevent compile-time
warnings.
Fixes: d487a24041 ("KVM: s390: Prepare gmap for a second KVM implementation")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-3-imbrenda@linux.ibm.com>
When the indicator and/or summary bits are set in the guest, the
accessed page was only marked dirty in KVM if the access was performed
using the slow path; accesses through the new kvm_arch_set_irq_inatomic
fast inject path would not mark the page as dirty.
Fix by adding/moving the missing calls to mark_page_dirty(). Note that
for the inatomic path set_page_dirty{,_lock}() is not needed as the
page stays pinned; the unpin path correctly marks it as dirty.
Opportunistically reorder the local variables to be in reverse
Christmas tree order and refactor to use guard().
Fixes: 1e95e3bc6b ("KVM: s390: Enable adapter_indicators_set to use mapped pages")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Message-ID: <20260828115439.145885-2-imbrenda@linux.ibm.com>
In case of an upgrade from four to five level page tables, and a failing
pgd allocation, the exit path of crst_table_upgrade() would incorrectly
dereference the p4d NULL pointer via pagetable_dtor().
Address this by reworking crst_table_upgrade(), which basically is a
revert of [1]. Take into account that GFP_KERNEL order-2 allocation
failures are very unlikely. Therefore keep the code as simple as
possible:
In case of an upgrade from three to five levels, and an allocation
failure of the fifth page table level, keep the upgrade to four levels
instead of reverting back to three levels. This allows to keep error
handling minimal.
[1] commit 31932757c6 ("s390/mm: optimize page table upgrade routine")
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
The facilities and debug-alternative command line parsers iterate over
inclusive numeric ranges. If a range ends at ULONG_MAX, incrementing the
current value wraps to zero and the loop never terminates. Large finite
out-of-range values also cause unnecessary early boot iterations even
though the bitmap helpers ignore them.
Stop each loop at the size of the bitmap it modifies. This preserves all
meaningful range values while guaranteeing termination.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
A command line may occupy all but the terminating byte of
COMMAND_LINE_SIZE. In that case append_ipl_block_parm() passes a zero size
to the IPL parameter conversion helpers and points the destination one
byte past early_command_line. The helpers subtract one from the unsigned
size and write the converted parameter outside the command line buffer.
Convert the IPL parameter in the command line parsing buffer first. A
parameter beginning with '=' can then replace the existing command line
regardless of its length, while other parameters are appended only when
space remains.
Fixes: 5ecb2da660 ("s390: support command lines longer than 896 bytes")
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
