Commit graph

248,460 commits

Author SHA1 Message Date
Linus Torvalds
06ac073129 x86 architecture fixes [with an UM fix included]:
- Don't apply va_align to hugetlb mappings on AMD F15h systems
    that have custom va_align.bits values (Laurent Wandrebeck)
 
  - Fix PMD teardown handling regression flagged by lockdep
    (Mikhail Gavrilov)
 
  - Hide ptrace header register offset macros behind __ASSEMBLER__ or
    __FRAME_OFFSETS, to fix user-space build errors that may trigger
    if they happen to shadow these short and generic macro names
    (Nick Desaulniers)
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmrCBfsRHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1hhvxAAiRvv7gz6K9VIJ7S24PdQOf8VuiuLT4De
 JUxTb6tUJ3VQ9+qk0xKSAg10z603PKn2Ow01znUNV5wTrRcumpkKTk39ut+x9Yl5
 qLDeLVTI/NFAkrYajtjPq3OrwZKnLcAYoZaM9cWtZIoIVCBVIDAsJgS7lIJbUjQ8
 FVvI4UUFaToM79jBMvAA89QOK5FrwO26/ENZLbRyceHzHnPrw/pYObaWpUhUs3S6
 r0OS2NRUacDid3s3SrVJqkJ9siZvS1OdVQZZwSSKz7oNPsk3yH5aqkFrLrCZJNsa
 VG3gCtb7ZLe3omFDmW9RzCYo5afbaUNbdHDOtameVoAT4h71cPoBr3mzAWDJiNEq
 NFu7ICAt7XP9Kvfoa4k0n4HhbYNxH1VAyoXOAiquqpPtsU+ZtG2HhqwZknrqGxM4
 9S127eCrAyGN9CLrYgcnZlALd1HUJ/nGHqdaSVxyxhaJxgxu9e717YmgsxXVl79p
 uCASyox64F4JrKr1n7Yp08UwfPvsquc8ViTyxsfHCuHj59hLnLfxPefr6N86Ztdc
 97yQhb7tDq7z8S8IUQTJuOtcDuqIUdm/VRg6+8SP+hVZquCW2zo0lla24Yi/cS1+
 om89Mfy4l9v+VzkBy2TvNU0cLy0ClK1BQTquHALWFuCDDEmJ4vftc7eWV4VQ+Idh
 lQDDgs0Q66g=
 =j39M
 -----END PGP SIGNATURE-----

Merge tag 'x86-urgent-2026-10-04' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull x86 fixes from Ingo Molnar:

 - Don't apply va_align to hugetlb mappings on AMD F15h systems
   that have custom va_align.bits values (Laurent Wandrebeck)

 - Fix PMD teardown handling regression flagged by lockdep
   (Mikhail Gavrilov)

 - Hide ptrace header register offset macros behind __ASSEMBLER__ or
   __FRAME_OFFSETS, to fix user-space build errors that may trigger
   if they happen to shadow these short and generic macro names
   (Nick Desaulniers)

* tag 'x86-urgent-2026-10-04' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  {x86,um}/uapi/ptrace: Guard register offset macros with __ASSEMBLER__ or __FRAME_OFFSETS
  x86/mm: Drop unnecessary PMD page copy when freeing
  x86/mm: Don't apply va_align to hugetlb mappings on AMD F15h
2026-10-04 08:59:22 -07:00
Linus Torvalds
a27611f899 Perf events fixes:
- Fix race between perf_event_exit_task() and perf_pending_task()
    (Luo Gengkun)
 
  - Fix perf header output management regressions (Ian Rogers)
 
  - Require kernel access for text poke events (Zhengchuan Liang)
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmrCAjwRHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1hTzw/+IJzVOLjNe2TaHJF20uYtuFVp80FGckH3
 qGMQcSSxEsdH2zWgA196UWYCuxcNtq9gRVkBw90u30Tka8F/C57r9fWf2Mx6iUBy
 zRrf70Lp4RsACAnM7kscX5ieJvesflYK5Vdr1je7DtQGE9P40Hq/45zquZt1B0uJ
 IqMVeDnqR46QIdAc/xNkn6G6FmI4CTJ1znwE1ZUm2HoDJ7Ya3zy0Gm9HNvdTaUN5
 IU5HWgNSEyL1id+2hwrv4/f+TPgmFSkJUf/t2hn/Rvzf1eOmRHuB/d8ZEBOEBDxk
 H5qeq/CGStzGMZsJprfQ65BGj/8QlXWNI8a6bzTnpAL2Ss9VKFvLvwmWDXmGo+B6
 n4+MOevA9LWDuZoSxRWzM00nBFp3h8YxyjSLYScZpoA3t0n45ckA+saK/ZDFBjYZ
 meWQX5mVVbktUX39hA/BeR2ps1KuHO8l2hq7BTlyK5nJfC69ptpM3OT2q0Jx/jD2
 kWKlhBLBhSObe3PX2uZiHDwXdEzuiMK0VKSYfz7gr0UDFoBbwe0AETOFIN9qhEXC
 4Lwln4vBpZTPGcGZYmBiWLS0uesy0mUa+FKzD8pZn7DKh8QY210Z9mr3+Tja3gsV
 pVjTs+7cpT9dsfX9z4PuSSnw8VGPfoM6iwGON5byva6Ylss+uL5iCCSQ3+heQFOz
 9aDZYLimUj4=
 =58mT
 -----END PGP SIGNATURE-----

Merge tag 'perf-urgent-2026-10-04' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull perf events fixes from Ingo Molnar:

 - Fix race between perf_event_exit_task() and perf_pending_task()
   (Luo Gengkun)

 - Fix perf header output management regressions (Ian Rogers)

 - Require kernel access for text poke events (Zhengchuan Liang)

* tag 'perf-urgent-2026-10-04' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf: Require kernel access for text poke events
  perf: Replace perf_event_header__init_id with full header init
  perf: Fix race between perf_event_exit_task() and perf_pending_task()
2026-10-04 08:35:29 -07:00
Linus Torvalds
c5adb6c8a2 arm64 fixes for -rc6
- Fix handling of CPU erratum #2645198 when batching pte updates
 
 - Fix truncation of CPU frequency calculation by using 64-bit arithmetic
   in arch_freq_get_on_cpu()
 
 - Work around AMU erratum #3821522 on Cortex-A725
 
 - Fix panic when trying to restore an SVE sigframe on a CPU that only
   supports SME
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEPxTL6PPUbjXGY88ct6xw3ITBYzQFAmq/tpYQHHdpbGxAa2Vy
 bmVsLm9yZwAKCRC3rHDchMFjNJ6gB/9ZmDMU4ED2AbyP7T8Q6FttzvH01X85F2yz
 hlaaXv/2MDzjGSiz0spxDAZOuteJF+WSLj1WjZhlyr95xnQWfumc6D89D+kHeNre
 +twed6hbOQTWJzLB+u8Ef+4nKGcztIZ/FafFvPD6PKjFuhG048al8X/6oNXetBPy
 Ka13ywRYhnccflsWaXyWlvSwfKn2xJHOnpu4Mfbsb9DfaULDLLahz2ug+6apwX23
 tc/dy40Fi0G6wFzUQGF83R3nsyZgLiOv7LFBjjqDmoiu7W86KXn83UXEwHXrp8de
 Zbynk7F4Bxf0lsp6y+bxTmwzm8CwI355/NFU2ccxxu17e7Qv6PnJ
 =TRoC
 -----END PGP SIGNATURE-----

Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Pull arm64 fixes from Will Deacon:
 "Half of this is broken hardware (AMU counters and TLB invalidation)
  and the other half is broken software (frequency scaling and signals).
  So it seems as though we're all as bad as each other.

  The AMU workaround is a little noisy, as it refactors an existing
  workaround so that it can more easily be applied to additional CPUs.

  Summary:

   - Fix handling of CPU erratum #2645198 when batching pte updates

   - Fix truncation of CPU frequency calculation by using 64-bit
     arithmetic in arch_freq_get_on_cpu()

   - Work around AMU erratum #3821522 on Cortex-A725

   - Fix panic when trying to restore an SVE sigframe on a CPU that only
     supports SME"

* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
  arm64/fpsimd: signal: Forbid non-streaming SVE payload on SME-only systems
  arm64: errata: Add Cortex-A725 erratum 3821522 workaround
  arm64: errata: Factor out broken AMU const counter cap
  arm64: topology: fix arch_freq_get_on_cpu() overflow above 4.19 GHz
  arm64: mm: Fix the break-before-make flush range for erratum 2645198
2026-10-03 07:54:58 -07:00
Linus Torvalds
8f150ccedf bpf-fixes
-----BEGIN PGP SIGNATURE-----
 
 iQJRBAABCgA7FiEE+soXsSLHKoYyzcli6rmadz2vbToFAmq/0m8dHGFsZXhlaS5z
 dGFyb3ZvaXRvdkBnbWFpbC5jb20ACgkQ6rmadz2vbTpVWxAAgk0avmO+opg/LpC6
 vQ6kQO743IkD1uc+jnGcRbOSo1YftUu5W1HY7hpj75MAdzSuScN3sflWyFb8Ng7l
 AG6HH9drhOIGzEMzc1847fDqM8BYO4DY72ofM/c6tAqlaO6v5EbT2zW/kXm8PPtS
 sRE60PP0lMh4NCuD2cH9X19GcR44CQXDijioW91hWAg8jxbGKgz9Skw6W+RHqIrH
 Gi5pkg7OV4Qz5+/XuLpJ0Sd7df8/LMtzAi+B6EG8EfldvkuSMT1vNyS04IKXjGN2
 1BWyUhWuIuRs/ReJ7PbkIa+Byc5ToiMmwozLR8X0M2k5iI6lGSIxdjaAUN0W8sIn
 fk0aIIHhXa7L5Kf/6sOpLX3zESx5f/i+/7GdMFLYRkXiQTLsMBi2lqxvz++wxxak
 s/MzJ0luOrbYrU53ufb1tNpwX97mYpvu8YLu7PadfEuQuGf4g1yhrjYAu7wltOCo
 vPOOCm8SSm+WW0BTNkjET7acYP4nXNHxnAx6KI1BPNMVMBa/C/b7PH7BrQhkkIfp
 Fq1cXL2FFTRLr+CxhU9oDHfTekvSrY58OZG+lputgFSy4JMXahoioXImGWw2TaGP
 00ExdJtA5lsTeskNuGoKDT0pZHWpgi0QlbGzdKWq4Qqv2fEOX5PWhgvnrucZ6+Ob
 +aH1vPjhom3qsWPxLKFKW/UZsN8=
 =+6qx
 -----END PGP SIGNATURE-----

Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf

Pull bpf fixes from Alexei Starovoitov:

 - Fix overflow of backward jump offset in constant blinding
   (Alexei Starovoitov)

 - Fix packet range of packet pointers sharing an id when var_off
   tightens umax of one pointer and not the other (Alexei Starovoitov)

 - Fix objects stuck in free_by_rcu_ttrace list of bpf memalloc
   (Alexei Starovoitov)

 - Fix use-after-free of progs detached from busy trampolines: wait for
   an RCU tasks grace period before freeing trampoline progs, and patch
   detached progs out of trampoline images that are still in use
   (Florent Revest)

 - Hold map BTF for the memory allocator destructor record to fix UAF in
   deferred bpf_mem_alloc destruction (Kumar Kartikeya Dwivedi)

 - Fix missing migration protection in resizable hashtab
   lookup_and_delete batch operation (Ömer Mete Kaya)

* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
  bpf: Fix missing migration protection in __rhtab_map_lookup_and_delete_batch()
  selftests/bpf: Add a test for objects stuck in free_by_rcu_ttrace
  bpf: Fix objects stuck in free_by_rcu_ttrace
  bpf: Factor out __do_call_rcu_ttrace()
  selftests/bpf: Test packet range of pointers sharing an id
  bpf: Fix packet range of pointers sharing an id
  selftests/bpf: Detach a trampoline prog while a task sleeps before it
  bpf: Skip detached progs in trampoline images that are still in use
  bpf: Wait for an RCU tasks grace period before freeing trampoline progs
  bpf: Hold map BTF for the memory allocator destructor record
  bpf: Fix overflow of jump offset in constant blinding
2026-10-02 12:59:32 -07:00
Linus Torvalds
ac7445c28e Random number generator fixes for Linux 7.3-rc6.
-----BEGIN PGP SIGNATURE-----
 
 iQJPBAABCAA5FiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmq/uGEbFIAAAAAABAAO
 bWFudTIsMi41KzEuMTIsMiwyAAoJEEn8cBKl3gOumEgQAK0gjA0n40rHWGlmgIJO
 WTqMZGcbTvVXpPRyEXuNE7tTRsDDopen4UnmDqDGafXkBO1vU1LaxN72JonPjp7O
 sg4EqCi0iciuiDvLrjwQ1oPtpkL/feyjcNuNzSH/h2E4hJCUcEnyhTpeaRHCm3t4
 IGDXqqYYqjAQV4N6dWrzwN5K2oM75IOzccr0hyROkN/DBrqNZtupOco725MKxSsr
 FGTgoh0a6GlN3Yp+AIjqaHcPca/aUZApPUKmBzoWeqZmSqAmpzUIJgaHsBZAQS88
 oMcSh71Fzi2tOepiazjaAK9nZC3Nh2xeRU8cwUtvLvluzYA6wnBhcuTBOqThJmp3
 gEYz8yPbvil0aDIhfXB3xW2dxxsHaJrKJDLz0Smhi43D2lLKteQT3BE/2jQQ7OSa
 6FcV905hys3lIxlHeCa2wE/7NgiNAy3qE1aazkIKRfI8dRJxIuhogvW3VpKGCAue
 TwvjfvuoVtkaPAW/gLyDGRsMzrUvGPFa8yVvP5JTPrYQh5P+bEIM8CozWO2ScVcH
 Cdu0/TDe9TTS5mIwh0CrSX3shV/80bvHTINL14sTAKBGtN0N0vJgA8NSdIYt8OaF
 8qDDMg/XV5LY7sbMB3/Kru//wyfUw4LJdYhZSIJ0tZvzaRRIRiZGL1lT+X12PCrq
 uoaJ4AoDCf/ZiDaVTXEVf+28
 =GmSm
 -----END PGP SIGNATURE-----

Merge tag 'random-7.3-rc6-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/crng/random

Pull random number generator fixes from Jason Donenfeld:

 - VMGENID memory needs to be mapped with the decrypted tag, so that
   SEV-SNP machines can boot

 - A fix for an initialization race in VMGENID, followed by a cleanup

 - Trivial kernel doc cleanups in siphash and random.c

 - A fix for a new compilation failure with recent clang on PPC and
   RISC-V, due to generating an out-of-line memset in the vDSO

* tag 'random-7.3-rc6-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/crng/random:
  random: vDSO: avoid call to memset() when zeroing reserved parameter
  random: fix vgetrandom_opaque_params kernel-doc
  random: vDSO: fix repeated word 'to' in comment
  siphash: clean up kernel-doc comments
  virt: vmgenid: move to using dev_set/get_drvdata
  virt: vmgenid: set driver_data before registering notification handlers
  virt: vmgenid: remap memory as decrypted
2026-10-02 11:28:45 -07:00
Nathan Chancellor
eb13a1ff27 random: vDSO: avoid call to memset() when zeroing reserved parameter
After a recent change in LLVM [1], builds with the random vDSO
implementation, such as PowerPC and RISC-V, fail when checking the vDSO:

  arch/powerpc/kernel/vdso/vdso32.so.dbg: dynamic relocations are not supported
  arch/riscv/kernel/vdso/vdso.so.dbg: dynamic relocations are not supported

memset() is now generated when zeroing params->reserved for some builds
because LLVM has an optimization (now run in more instances) that can
recognize at compile time when it is assigning a static value to a
contiguous area of memory and turn that into a call to memset(). Both
clang and GCC assume memset() is always available [2].

Clang has an internal fiddly hook, -max-store-memset, which we can set
to a high number, to disable generating out of line memset calls [3].
Similarly, GCC has -finline-stringops=memset to do the same [4], should
this issue ever hit future version of GCC. While these options wouldn't
make sense for normal kernel code, it is fine for the extremely limited
and intentionally compact vDSO code.

Link: 90cebef141 [1]
Link: https://gcc.gnu.org/onlinedocs/gcc-16.2.0/gcc/Standards.html#index-ffreestanding [2]
Link: b28eeb28be [3]
Link: https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-finline-stringops [4]
Closes: https://github.com/ClangBuiltLinux/linux/issues/2183
Cc: stable@vger.kernel.org # v6.12+
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2026-10-02 12:12:13 +02:00
Linus Torvalds
bd35955b08 Crypto library fixes for v7.3-rc6
- Fix a performance regression in certain AES encryption modes on
   certain architectures, introduced this cycle.
 
 - Fix a small performance regression in the x86_64 optimized AES-GCM
   code, introduced in 6.15.
 -----BEGIN PGP SIGNATURE-----
 
 iIoEABYIADIWIQSacvsUNc7UX4ntmEPzXCl4vpKOKwUCar6ZLxQcZWJpZ2dlcnNA
 a2VybmVsLm9yZwAKCRDzXCl4vpKOKzYvAQDtb68vX+UySUeUo+dGaaZWZv03zKEZ
 QxnsS78p1jvKZgD9FkcqZ6oe2wDPMnpZGV0tg5OpTBfwm+agX08I83tZjQU=
 =gSeD
 -----END PGP SIGNATURE-----

Merge tag 'libcrypto-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux

Pull crypto library fixes from Eric Biggers:

 - Fix a performance regression in certain AES encryption modes on
   certain architectures, introduced this cycle

 - Fix a small performance regression in the x86_64 optimized AES-GCM
   code, introduced in 6.15

* tag 'libcrypto-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux:
  crypto: aes - Fix undesired override of some optimized AES modes
  crypto: x86/aes-gcm - fix always true check for last AAD segment
2026-10-01 11:04:27 -07:00
Linus Torvalds
a940b03cee ARM:
- Take a reference on the last IRQ loaded into an LR to prevent it
   from being freed while running the guest (Marc Zyngier)
 
 - Ensure that the ITS MOVALL command only affects LPIs that were
   previously affined to the source redistributor (Marc Zyngier)
 
 - Fix + test for honoring the host's trap configuration when running
   non-protected VMs while KVM is in protected mode (Fuad Tabba)
 
 - Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
   stage-2 (Mostafa Saleh)
 
 x86:
 
 Various bugfixes where the guest could do stupid things on purpose to
 cause problems in the host:
 
 - Failed VMRUNs can cause pending TLB flushes to be dropped, and in
   general some actions done through VMCB control fields have to be redone
   if VMRUN fails
 
 - Toggling MSR interceptions or eVMCS execution controls can cause the
   host to use a stale MSR permission bitmap
 
 - Bad page tables can cause a WARN.
 
 Also fix issues in last week's pull request (my fault, for changing
 email workflow and thus missing feedback sent to kvm@ but not LKML).
 
 Generic:
 
 - Take kvm_lock when creating vCPUs. For almost two decades everybody
   thought it was not done for some unspecified performance reasons, but
   in reality it was only done because kvm_lock was originally a spinlock.
   This is a better fix than 97d65b544f ("KVM: Check for duplicate vcpu_id
   as early as possible", from the 7.3 merge window), and does not waste
   2K per VM, hence its inclusion here.
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmq+lW0UHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroPUbAf/Vap8HMmHlfQ0/A+25CZtJjWNOuNM
 Zj3YMFLGjiqTHTVjWafWCpvS2kWdV/MyTcgFxEKyx/QJ7cxofLAHCtak5TYoUyJ7
 oUQaHf2eK6VdtA1yz7yBf0zQSpKZhOoF3Ze2kQCUaR1ZEHm3apcFeh7jnjFammdx
 YpeHm1ERKJW9KQNc4KXH9dIx8IGjOW5C6hAcxgZKUeTy+5oz7FMOqPQc/wNowOG5
 ldxzSDa+BoHZ/okOkeBWgTfYFPH0MU3CO/ZX9+eJ+HIIEZ+XkVzo3h8KBpmWPTqj
 dAksp8oZeg9B9XLzxklCAnpJaoNQHhQlQJAwJMivwwNwT99drizphQY7Bw==
 =1X9h
 -----END PGP SIGNATURE-----

Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm

Pull kvm fixes from Paolo Bonzini:
 "The most intrusive change is reverting a commit from 7.3-rc1 that made
  struct kvm a bit too large, and fixing the same issue otherwise.

  There are again a lot of selftests lines; the sheer number of commits
  is not small but I don't expect much more for 7.3 due to people
  travelling to Plumbers next week.

  ARM:

   - Take a reference on the last IRQ loaded into an LR to prevent it
     from being freed while running the guest (Marc Zyngier)

   - Ensure that the ITS MOVALL command only affects LPIs that were
     previously affined to the source redistributor (Marc Zyngier)

   - Fix + test for honoring the host's trap configuration when running
     non-protected VMs while KVM is in protected mode (Fuad Tabba)

   - Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
     stage-2 (Mostafa Saleh)

  x86:

  Various bugfixes where the guest could do stupid things on purpose to
  cause problems in the host:

   - Failed VMRUNs can cause pending TLB flushes to be dropped, and in
     general some actions done through VMCB control fields have to be
     redone if VMRUN fails

   - Toggling MSR interceptions or eVMCS execution controls can cause
     the host to use a stale MSR permission bitmap

   - Bad page tables can cause a WARN.

  Also fix issues in last week's pull request (my fault, for changing
  email workflow and thus missing feedback sent to kvm@ but not LKML).

  Generic:

   - Take kvm_lock when creating vCPUs. For almost two decades everybody
     thought it was not done for some unspecified performance reasons,
     but in reality it was only done because kvm_lock was originally a
     spinlock.

     This is a better fix than 97d65b544f ("KVM: Check for duplicate
     vcpu_id as early as possible", from the 7.3 merge window), and does
     not waste 2K per VM, hence its inclusion here"

* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (29 commits)
  KVM: arm64: Use stage-1 leaf size for VM_PFNMAP
  KVM: arm64: selftests: Check a feature hidden in an ID register is UNDEF
  KVM: arm64: Use the host's HCR_EL2 for non-protected VMs in pKVM
  KVM: arm64: Clear HCR_EL2.RW for 32-bit non-protected vCPUs
  KVM: arm64: Apply the fine-grained UNDEFs without FEAT_FGT
  KVM: arm64: vgic-its: Fix MOVALL handling of source redistributor
  KVM: arm64: vgic: Take a refcount on IRQs referenced by last_lr_irq
  KVM: arm64: vgic: Allow last_lr_irq to be NULL when LRs are not overflowing
  KVM: SEV: Do cache maintenance on the source VM *before* clearing SEV state
  KVM: SEV: Nullify "have run CPUs" mask pointer when freeing it
  KVM: selftests: Extend nested x2APIC test to validate using eVMCS for vmcs12
  KVM: selftests: Extend nested x2APIC test to validate disabling x2APIC virt
  KVM: selftests: Verify that L0's TPR doesn't get clobbered
  KVM: selftests: Run the nested x2APIC with and without APICv being inhibited in L2
  KVM: selftests: Add x2APIC MSR test for inhibiting APICv while nested
  KVM: nVMX: Force MSR bitmap refresh if runtime eVMCS controls are modified
  KVM: SVM: Use the active VMCB's MSR bitmap when checking if MSR is intercepted
  KVM: SVM: Sync guest's PERF_CNTR_GLOBAL_CTL from h/w only on successful VMRUN
  KVM: SVM: Don't mark ASID fields as dirty when setting control.tlb_ctl
  KVM: SVM: Update control fields on #VMEXIT if and only if VMRUN succeeded
  ...
2026-10-01 10:57:19 -07:00
Paolo Bonzini
f9bfc323e7 KVM/arm64 fixes for 7.3, round #2
- Take a reference on the last IRQ loaded into an LR to prevent it
    from being freed while running the guest (Marc Zyngier)
 
  - Ensure that the ITS MOVALL command only affects LPIs that were
    previously affined to the source redistributor (Marc Zyngier)
 
  - Fix + test for honoring the host's trap configuration when running
    non-protected VMs while KVM is in protected mode (Fuad Tabba)
 
  - Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
    stage-2 (Mostafa Saleh)
 -----BEGIN PGP SIGNATURE-----
 
 iI0EABYKADUWIQSNXHjWXuzMZutrKNKivnWIJHzdFgUCar6KFRccb2xpdmVyLnVw
 dG9uQGxpbnV4LmRldgAKCRCivnWIJHzdFoQxAP4lpoLN5+XlRshepgo+t/hZnhqj
 4X6A8t7gtKv1iSp9WgD9FOeekzRH8HunEyjTyez/SmIb4ZZ9yYSMd5wIXX7uqgE=
 =FD18
 -----END PGP SIGNATURE-----

Merge tag 'kvmarm-fixes-7.3-2' of https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD

KVM/arm64 fixes for 7.3, round #2

 - Take a reference on the last IRQ loaded into an LR to prevent it
   from being freed while running the guest (Marc Zyngier)

 - Ensure that the ITS MOVALL command only affects LPIs that were
   previously affined to the source redistributor (Marc Zyngier)

 - Fix + test for honoring the host's trap configuration when running
   non-protected VMs while KVM is in protected mode (Fuad Tabba)

 - Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
   stage-2 (Mostafa Saleh)
2026-10-01 13:13:57 -04:00
Mostafa Saleh
71cc2c67fb KVM: arm64: Use stage-1 leaf size for VM_PFNMAP
When commit 2aa53d68ce ("KVM: arm64: Try stage2 block mapping for
host device MMIO") added VM_PFNMAP support to get_vma_page_shift(),
stage-1 page tables did not support huge PFNMAP (as in VFIO-PCI
vfio_pci_mmap_huge_fault()) and transparent_hugepage_adjust() was
unsafe for MMIO as it dereferenced struct page.

Since commit 6011cf68c8 ("KVM: arm64: Walk userspace page tables to
compute the THP mapping size"), transparent_hugepage_adjust() instead
walks the host stage-1 page tables via get_user_mapping_size() without
touching struct page. Meanwhile, commit 3e509c9b03 ("mm/arm64:
support large pfn mappings") enabled stage-1 huge PFNMAP.

So. we can drop the VMA-based VM_PFNMAP size calculation in
get_vma_page_shift() and let transparent_hugepage_adjust() derive
the stage-2 mapping size directly from the populated stage-1 leaf
for non-cacheable mappings as well.

Assisted-by: LLM
Cc: stable@vger.kernel.org
Fixes: 2aa53d68ce ("KVM: arm64: Try stage2 block mapping for host device MMIO")
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-10-01 09:27:26 -07:00
Mark Rutland
e060d9069b arm64/fpsimd: signal: Forbid non-streaming SVE payload on SME-only systems
When system_supports_sme() is true but system_supports_sve() is false,
restoring a specifically crafted SVE signal context can result in the
task erroneously having non-streaming SVE state. Subsequent attempts to
manipulate the task's FPSIMD/SVE/SME state can result in a variety of
problems, including fatal EL1 UNDEFs.

In such configurations, the kernel always creates an SVE signal context
when delivering a signal, and this can only be in one of two states:

(1) SVE_SIG_FLAG_SM is set, and an SVE payload is present containing
    streaming mode SVE state. The recorded VL is the task's live
    streaming VL.

(2) SVE_SIG_FLAG_SM is clear, and an SVE payload is not present. The
    FPSIMD context contains the non-streaming mode FPSIMD state. The
    recorded VL is 0.

Currently restore_sve_fpsimd_context() correctly rejects cases where
SVE_SIG_FLAG_SM is set and an SVE payload is not present, but fails to
reject cases where SVE_SIG_FLAG_SM is clear and an SVE payload is
present. Consequently, restore_sve_fpsimd_context() can place the task
in a state where it has non-streaming SVE state even when this is not
supported by HW.

For example, this can cause a later EL1 UNDEF when the kernel attempts to
restore the task's ZCR_EL1 value:

| # ./sme-sigcontext-to-sve
| Internal error: Oops - Undefined instruction: 0000000002000000 [#1]  SMP
| Modules linked in:
| CPU: 0 UID: 0 PID: 131 Comm: sme-sigcontext- Not tainted 7.3.0-rc1 #1 PREEMPT
| Hardware name: linux,dummy-virt (DT)
| pstate: 61402009 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
| pc : fpsimd_restore_current_state+0x258/0x458
| lr : exit_to_user_mode_loop+0xb8/0x188
| sp : ffff80008056be40
| x29: ffff80008056be40 x28: fff00000c1670000 x27: 0000000000000000
| x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000
| x23: ffff80008056bec0 x22: 0000000000000008 x21: 0000000000000040
| x20: 0000000000000081 x19: 0000000008800010 x18: 0000000000000000
| x17: 0000fffffd412570 x16: 0000000000001000 x15: 0000fffffd4123b0
| x14: 0000fffffd412780 x13: 0000fffffd412be8 x12: 0000000047435300
| x11: 0000fffffd412570 x10: 0000000000000000 x9 : 0000000045585401
| x8 : fff00000c18a6c44 x7 : 0000000000000000 x6 : 0000000000000002
| x5 : 0000000000000002 x4 : ffff800080568000 x3 : 0000000000000001
| x2 : 0000000008800010 x1 : fff00000c1670000 x0 : 0000000008800000
| Call trace:
|  fpsimd_restore_current_state+0x258/0x458 (P)
|  exit_to_user_mode_loop+0xb8/0x188
|  el0_svc+0x1cc/0x1d0
|  el0t_64_sync_handler+0xa0/0xe4
|  el0t_64_sync+0x198/0x19c
| Code: d5384101 f9400020 53175c03 36b80de0 (d5381202)
| ---[ end trace 0000000000000000 ]---
| Kernel panic - not syncing: Oops - Undefined instruction: Fatal exception in interrupt
| Kernel Offset: 0x291fb1000000 from 0xffff800080000000
| PHYS_OFFSET: 0x40000000
| CPU features: 0x0,00000000,0052802f,ffb88f43,3afcf73f
| Memory Limit: none

Rework restore_sve_fpsimd_context() to reject cases where
SVE_SIG_FLAG_SM is clear and an SVE payload is not present. As
parse_user_sigframe() rejects SVE signal frames when neither SVE nor SME
are supported, it isn't necessary for restore_sve_fpsimd_context() to
handle the case where neither are supported.

Fixes: 7dde62f068  ("arm64/signal: Always accept SVE signal frames on SME only systems")
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: stable@vger.kernel.org
Signed-off-by: Will Deacon <will@kernel.org>
2026-10-01 15:45:23 +00:00
Beata Michalska
45f7304679 arm64: errata: Add Cortex-A725 erratum 3821522 workaround
Cortex-A725 erratum 3821522 affects the CNT_CYCLES event, which can
incur a significant increment error when a CPU enters and subsequently
exits WFE or WFI, and may no longer track the system counter
frequency.

The AMEVCNTR01_EL0 counter is used as the AMU constant counter for
frequency invariance and CPPC FFH feedback counters. Wire the affected
Cortex-A725 range into the shared broken AMU constant-counter capability
so the affected counter is treated as unavailable by returning zero in
the AMU counter paths. This prevents the broken counter from being used
as a reference source.

The erratum can also affect PMUv3 users of the CNT_CYCLES event,
but this workaround intentionally does not change PMU event handling.
Hiding or rejecting the PMU event from the erratum code would change
the perf-visible PMU event interface, including raw event selection,
and would need a separate PMU-specific approach rather than being
folded into the AMU reference-counter workaround.

Cc: stable@vger.kernel.org
Signed-off-by: Beata Michalska <beata.michalska@arm.com>
Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
2026-10-01 12:22:40 +00:00
Beata Michalska
573371caf7 arm64: errata: Factor out broken AMU const counter cap
Move the workaround from the erratum 2457168-specific cpucap to a generic
broken AMU constant-counter one. This keeps the existing Cortex-A510
handling unchanged while allowing other errata with similar AMU constant
counter issue to share the capability bit and call sites.

Signed-off-by: Beata Michalska <beata.michalska@arm.com>
Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
2026-10-01 12:22:39 +00:00
Ian Rogers
b9d1fdc6f4 perf: Replace perf_event_header__init_id with full header init
perf_iterate_sb() invokes its callback for each matching perf_event on
the CPU and task context, passing a shared caller-allocated event
structure.

perf_event_header__init_id() mutated header->size in place by adding
event->id_header_size, requiring sideband output callbacks to save and
restore header fields across iterations. Three sideband callbacks failed
to save and restore header.size around perf_event_header__init_id():
 - perf_event_ksymbol_output()
 - perf_event_bpf_output()
 - perf_event_text_poke_output()

When multiple events with attr.ksymbol, attr.bpf_event, or
attr.text_poke and sample_id_all are active on the same CPU, each
subsequent event receives a record whose header.size is inflated by all
preceding events' id_header_size values while only a single id_sample is
written, leaving uninitialized ring-buffer bytes at the end of the
record and causing userspace perf to fail with -EFAULT ("Bad address")
when parsing the sample_id trailer.

Similarly, perf_event_mmap_output() set PERF_RECORD_MISC_MMAP_BUILD_ID
in mmap_event->event_id.header.misc when event->attr.build_id was
enabled, but only saved and restored header.size and header.type. If an
event with attr.build_id was followed by an event with attr.mmap2 and
!attr.build_id, the second event received PERF_RECORD_MISC_MMAP_BUILD_ID
in header.misc while its payload contained maj/min/ino/ino_generation
instead of a build ID.

Rather than splitting header initialization between callers and output
callbacks and saving/restoring mutated header fields, replace
perf_event_header__init_id() with perf_event_header__init(), which
initializes header->type, header->misc, and header->size alongside the
sample_id fields on each invocation.

Fixes: 76193a9452 ("perf, bpf: Introduce PERF_RECORD_KSYMBOL")
Fixes: 6ee52e2a3f ("perf, bpf: Introduce PERF_RECORD_BPF_EVENT")
Fixes: e17d43b93e ("perf: Add perf text poke event")
Fixes: 88a16a1309 ("perf: Add build id data in mmap2 event")
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260929222332.973435-1-irogers@google.com
Cc: stable@vger.kernel.org
2026-10-01 14:02:06 +02:00
Mohamad Raizudeen
226154ea87 crypto: x86/aes-gcm - fix always true check for last AAD segment
In gcm_process_assoc(), a segment that is not the last one must have its
length rounded down to a multiple of 16 bytes, as required by the
assembly. The check for a non-last segment is `if (unlikely(assoclen))
/* Not the last segment yet? */` where assoclen is the number of AAD
bytes remaining after the current segment.

Since the conversion to the new scatterwalk API, assoclen is decremented
at the end of the loop body rather than the beginning, so it still
includes the current segment when the check executes, making the check
always true.

As a result the last segment was rounded down as well, causing some
avoidable extra work: an additional memcpy into the temporary buffer and
an additional call into the assembly after the loop. The GCM
authentication tag is unaffected either way, so the self-tests pass and
this went unnoticed. It is purely an efficiency issue rather than a
correctness one.

Fix this by moving the assoclen decrement back to the beginning of the
loop body.

Fixes: e9787deff4 ("crypto: x86/aes-gcm - use the new scatterwalk functions")
Cc: stable@vger.kernel.org
Suggested-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mohamad Raizudeen <raizudeen.kerneldev@gmail.com>
Link: https://patch.msgid.link/20260926145903.6061-1-raizudeen.kerneldev@gmail.com
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
2026-09-29 15:28:42 -07:00
Linus Torvalds
b004c4b1c4 ARC fixes for 7.3
- cmpxchg snafu [Bradley] and other miscll fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEOXpuCuR6hedrdLCJadfx3eKKwl4FAmq7+6gACgkQadfx3eKK
 wl46/g//XPlrCeVP5oYddVzn3YdHQ4D0rJpUDdtwmG0epxA+tKnCKIuu81bwvCYT
 n9CpEkdGXgolQrvnCvIE6tVSz+Kld+ftt58DEjQwDi0/hl0oS7OT7ad5V56hlpH8
 yfogX3ulyTcB3hK4lTmIWbr9hdoKYzbnKW+RYUnRkwk9iwy2FHDy1rnv9I77sAF7
 QIf/a0OuKJpbaTu+bCeNc2x9LdiN1PVfoFe0QGj2NyMRgvFDGU3B9Qza5rUlnMEh
 3FWh+bTDOV2AFjW448QDw3mY7e360udufQLm/8e2ZX8zy8uxtgD4vU957XpAuzz7
 gdXUXmjUTwpCICYuMflH4tFdCz1GzbPRoCSVTYY1E0hJ6PB5WXwNXqo2k1apeW2E
 6q8V2tKmSUvNK4NcjhpuLi5SbVdUUP8HnkLtj33WMMFbzZmzXict4EeUPrkxvvow
 8GmUqW9DukBcErJjtf3ynszN7lki32piFPyqUzEdpWn2ryNf+w5m5JB/FWO3FPM+
 yfLuISTdfmRHsO70zHzL2//dZoqXcjfy7UrGhOKugiIh6hwk1NZA4TZ4c1V2f5UP
 dpw+cidCSmKPATSmx1PPpH5A0hQ8p6zbFK2gE4Pcf238wOJg/5RpXz2rl3RjmfaC
 8kpX+rPHw07JBRyN9qsEihK6+x094W/WaekD19J01y4ef0MRE3U=
 =iG8+
 -----END PGP SIGNATURE-----

Merge tag 'arc-fixes-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc

Pull ARC fix from Vineet Gupta

 - cmpxchg snafu spotted by Bradley and other misc fixes

* tag 'arc-fixes-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc:
  ARC: arch_cmpxchg_relaxed to use size of pointed type not pointer
  arc: kernel: Fix clk reference leak in show_cpuinfo()
  arc: remove unused profile.h includes
  ARC: cleanup dead ARC_CANT_LLSC option in Kconfig
2026-09-29 11:30:52 -07:00
Fuad Tabba
04447b95c6 KVM: arm64: Use the host's HCR_EL2 for non-protected VMs in pKVM
For each vCPU, pKVM's EL2 builds its own HCR_EL2 and takes only TWI, TWE
and VSE from the host's value on each entry. For a non-protected VM it
has fallen behind the host's: on a CPU with MTE the guest can read
GMID_EL1, on one with FEAT_EVT2 but no FGT it can execute a TLBI OS its
ID registers hide, an interrupt injected without a vGIC (VI, VF) never
arrives, and set/way emulation loses the TVM trap it relies on.

For a non-protected VM, also take the bits the host varies with the VM's
configuration or at runtime: VI, VF, TVM, TID2, TID4, TID5 and TTLBOS.
They only pend interrupts for, or add traps to, a VM the host controls.
The traps exit to the host, which handles them as it does without pKVM.
The bits that change what EL2 does on entry and exit (E2H, RW, API/APK)
or depend only on the CPU (TEA, TERR, FWB) stay EL2's. A protected VM
still takes only TWI, TWE and VSE.

EL2 now sets TID2 or TID4 only for a protected VM, and never sets ATA,
as the host rejects KVM_CAP_ARM_MTE in pKVM.

Fixes: b56680de9c ("KVM: arm64: Initialize trap register values in hyp in pKVM")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260929090031.3829185-4-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-09-29 09:08:56 -07:00
Fuad Tabba
80ae56184b KVM: arm64: Clear HCR_EL2.RW for 32-bit non-protected vCPUs
pKVM keeps its own copy of each vCPU's HCR_EL2 at EL2, initialised with
RW set unconditionally. Nothing stops userspace from creating a
non-protected AArch32 VM, and with RW set, entering one of its vCPUs is
an illegal exception return: KVM_RUN fails with KVM_EXIT_FAIL_ENTRY.

Clear RW for a vCPU that is AArch32 at EL1, when the CPU has AArch32
EL1. On a CPU without it, RW stays set and the entry still fails, rather
than EL2 switching *32_EL2 registers that are UNDEFINED there. The host
already rejects such a vCPU, but EL2 takes the vCPU's features from the
host and doesn't rely on that.

Fixes: b56680de9c ("KVM: arm64: Initialize trap register values in hyp in pKVM")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260929090031.3829185-3-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-09-29 09:08:56 -07:00
Fuad Tabba
4bdd2b3a70 KVM: arm64: Apply the fine-grained UNDEFs without FEAT_FGT
FGUs don't actually require hardware support, so don't gate fgt trap
config insertion on FGT. This is the only action required to allow FGUs
always, as kvm_calculate_traps() already computes the FGU bits with or
without FEAT_FGT.

This also fixes a spurious WARN when a TLBI OS is run on a guest in vEL1
without FEAT_TLBIOS on non FEAT_FGT hardware. In this configuration the
course-grained HCR_EL2.TTLBOS trap reaches the handler handle_tlbi_el1(),
which expects an EL1 TLBI only from vEL2. With FGUs enabled the UNDEF
will be injected without needing the specific handler.

Fixes: f5a5a406b4 ("KVM: arm64: Propagate and handle Fine-Grained UNDEF bits")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://patch.msgid.link/20260929090031.3829185-2-fuad.tabba@linux.dev
[oliver: Take Wei-Lin's suggested changelog]
Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-09-29 09:07:27 -07:00
Marc Zyngier
751f464156 KVM: arm64: vgic-its: Fix MOVALL handling of source redistributor
The MOVALL command moves the pending state of all LPIs targeting a given
redistributor to another one. However, we seem to have lost the
filtering on the source RD, which means we move all LPIs to the target.

Not quite what the spec mandates.

Hack update_affinity() to take an optional source vcpu that is used as a
filter when non-NULL, restoring the filtering that was performed by
vgic_copy_lpi_list() back in the days.

Fixes: 11f4f8f3e6 ("KVM: arm64: vgic-its: Walk LPI xarray in vgic_its_cmd_handle_movall()")
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260929093548.3598547-7-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-09-29 09:06:19 -07:00
Marc Zyngier
19bc5d79ec KVM: arm64: vgic: Take a refcount on IRQs referenced by last_lr_irq
Referencing the last interrupt inserted in an LR is rather fragile, as
this interrupt can vanish if a concurrently unmapped LPI.

Solve this by bumping up the refcount on the interrupt when populating
last_lr_irq, and drop it at vgic_prune_ap_list() time, when LPIs are
being reclaimed.

Fixes: 6da5e537f5 ("KVM: arm64: vgic: Pick EOIcount deactivations from AP-list tail")
Reported-by: Yuchao Zhang <ndaugoing@gmail.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260929093548.3598547-5-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-09-29 09:06:19 -07:00
Marc Zyngier
e079ef0b1c KVM: arm64: vgic: Allow last_lr_irq to be NULL when LRs are not overflowing
last_lr_irq is always populated when there is any interrupt populated in
the AP list. Not only this is not necessary (it is only useful when we
completely fill the LRs), but this is in the way of further fixes.

Make sure last_lr_irq is kept to NULL when we LRs are not completely
full.

Fixes: 6da5e537f5 ("KVM: arm64: vgic: Pick EOIcount deactivations from AP-list tail")
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260929093548.3598547-4-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
2026-09-29 09:06:19 -07:00
Florent Revest (Anthropic)
17637e1a58
bpf: Skip detached progs in trampoline images that are still in use
bpf_tramp_image_put() makes sure a trampoline image is not freed while
a task may still be running in it, but nothing similar is done for the
progs called by that image. Detach drops the last prog reference right
away and the prog is freed after grace periods, on the basis that a
task still in the traced function skips the fexit progs once the nop at
ip_after_call is patched to a jump.

That leaves out a task sleeping in a sleepable prog that runs before
the detached one, which no grace period waits for:

  CPU 0                               CPU 1
  in image I, sleeping in prog S
                                      detach P from I's trampoline
                                       -> new image, bpf_tramp_image_put(I)
                                      bpf_prog_put(P), last ref
                                      grace periods, P freed
  back from S
  __bpf_prog_enter(P)
  call P->bpf_func

If S and P are fexit progs the task is already past the patched jump,
and fentry only images don't have one. On x86 this is an int3 in
poisoned bpf_prog_pack memory:

  Oops: int3: 0000 [#1] SMP NOPTI
  CPU: 18 UID: 0 PID: 94573 Comm: x169 Not tainted 6.18.44 #1 PREEMPT(lazy)
  RIP: 0010:0xffffffffc0601d8d
  Call Trace:
   <TASK>
   ? bpf_trampoline_6442515411+0x1a4/0x21b
   bpf_lsm_bprm_committed_creds+0x5/0x10
   security_bprm_committed_creds+0x5f/0x70
   begin_new_exec+0x2d6/0x410
   ...

We hit this in production when progs attached through trampolines got
detached while their hooks were busy, and it was independently found
with a fuzzer and KASAN.

Have the JITs emit a patchable nop in front of each prog call sequence
and record it in the image. When a prog is detached, patch its nop to a
jump over the call sequence. Progs that stay attached keep running for
the tasks that are in the image, and ip_after_call isn't needed anymore.

The task can be in any image that isn't freed yet, not only in the
current one. It sleeps in image I1 that calls S, P and Q, then P is
detached and the trampoline moves to image I2, then Q is detached and
its call is still in I1. So the trampoline keeps a list of its images
until they are freed, and detaching a prog patches its nop in all of
them. Images hold a reference on the trampoline for that long.

On riscv and loongarch a jump of any range takes several instructions,
and a task preempted in the middle of them could resume into half of the
new sequence. The nop is a single instruction there, patched to a near
branch through arch_bpf_trampoline_skip().

With the extra nops, BPF_MAX_TRAMP_LINKS progs no longer fit in a page
on x86 and arm64 (and already didn't on powerpc), so lower the limit
there like s390 does.

Fixes: e21aa34178 ("bpf: Fix fexit trampoline.")
Reported-by: Sechang Lim <rhkrqnwk98@gmail.com>
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260926135605.1217928-3-florent.revest@linux.dev
Closes: https://lore.kernel.org/bpf/20260815071927.147049-1-zirajs7@gmail.com/
2026-09-29 05:54:16 +00:00
Sean Christopherson
973ea70393 KVM: SEV: Do cache maintenance on the source VM *before* clearing SEV state
Explicitly flush caches after intra-host migration before clearing "SEV
active" on the source VM, as doing cache maintenance afterwards creates a
tiny window where memory reclaim could return memory to the host without
performing a cache flush, e.g. as pointed out by Sashiko:

  CPU1 in sev_migrate_from():
    src->active = false;

  CPU2 running concurrent unmap:
    Since active is false, the automatic cache flush in
    sev_guest_memory_reclaimed is skipped.
    The host frees and reallocates the page.

  CPU1 in sev_migrate_from():
    sev_writeback_caches(src_kvm);
    Executes a hardware cache flush (wbnoinvd), which writes the guest's old
    dirty ciphertext over the new page owner's data.

Fixes: 93de2a6a4b ("KVM: SEV: Do cache maintenance on the source VM during intra-host migration")
Cc: stable@vger.kernel.org
Reported-by: Sashiko Bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260923165304.1662E1F000FF@smtp.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260928154644.2559454-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-28 17:11:49 -04:00
Sean Christopherson
8abbc76120 KVM: SEV: Nullify "have run CPUs" mask pointer when freeing it
Nullify have_run_cpus when freeing the mask, particularly in the error path
of __sev_guest_init(), so that KVM doesn't have to subtly use sev->active
to track whether or not the mask has been freed.  As pointed out by Sashiko,
blindly freeing the mask in sev_vm_destroy() results in a double-free if
the mask is freed if __sev_guest_init() fails.

Throw the logic in a helper as nullifying the pointer is frustratingly
difficult and weird due to have_run_cpus being a single-entry array when
CPUMASK_OFFSTACK=n.  Deliberately don't use CPUMASK_VAR_NULL, as it's not
directly assignable when the cpumask is on-stack, e.g. requires using a
local variable and a memcpy(), which is beyond ridiculous.  Furthermore,
while clearing the on-stack bitmask is an unnecessary and arguably unwanted
side effect, KVM absolutely relies on '0' being the "null" value given that
the struct is zero-allocated.

Opportunistically add an alloc() helper to pair with free(); there are just
enough call sites to make doing so worthwhile.

Fixes: 12c1f6e03f ("KVM: SEV: Free have_run_cpus during VM destruction even if VM is no longer SEV")
Cc: stable@vger.kernel.org
Reported-by: Sashiko Bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260923165349.CAAF01F000FF@smtp.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260928154644.2559454-2-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-28 17:11:49 -04:00
Nick Desaulniers
a661c34fe6 {x86,um}/uapi/ptrace: Guard register offset macros with __ASSEMBLER__ or __FRAME_OFFSETS
The register offset macros in <asm/ptrace-abi.h> are guarded by
`defined(__ASSEMBLER__) || defined(__FRAME_OFFSETS)` for 64-bit, but
were left unguarded for 32-bit. This causes havoc for userspace that
happens to use identifiers colliding with these short macro names
(e.g., EBX, ECX, EAX, DS, ES, FS, GS, CS, SS). Without this guard,
userspace is forced to be super extra careful with include ordering to
minimize the chance of collision.

Wrap both the 32-bit and 64-bit register definitions under
`#if defined(__ASSEMBLER__) || defined(__FRAME_OFFSETS)`, and ensure
User-Mode Linux (UML) defines `__FRAME_OFFSETS` for 32-bit as well.

Closes: https://github.com/llvm/llvm-project/issues/217413
Assisted-by: LLM
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Johannes Berg <johannes@sipsolutions.net>
Tested-by: Elliott Hughes <enh@google.com>
Link: https://patch.msgid.link/20260821-ptrace_uapi-v1-1-3de8638a29f2@google.com
2026-09-28 12:11:19 -07:00
Mikhail Gavrilov
e3ee38c1bc x86/mm: Drop unnecessary PMD page copy when freeing
On a box with a discrete GPU, lockdep reports a possible deadlock as
soon as kswapd shrinks the TTM page pool. The immediate cause is an
x86 commit that added an mmap_read_lock() to kernel page protection
munging code.

The huge vmap code holds the same lock over a GFP_KERNEL allocation,
which is a no-no now that reclaim can take it. That allocation is in a
page table *free* path and ends up being for dubious purposes[1].
Basically, it tries to avoid hardware setting Accessed=1 in page table
entries that are unreachable by the hardware, a non-issue.

Remove the PMD copy. Detach the original PMD page at the PUD, flush
the mid-level caches, and free the PTE tables straight from the
detached PMD page. With no allocation left, the locking issue is gone.

Lockdep splat/analysis:

  WARNING: possible circular locking dependency detected
  7.3.0-rc3-f6e7b42bf05b+ #183 Tainted: G     U
  ------------------------------------------------------
  kswapd0/269 is trying to acquire lock:
  ((init_mm).mmap_lock){++++}-{4:4}, at: change_page_attr_set_clr+0x29a/0x4a0
  but task is already holding lock:
  (pool_shrink_rwsem){.+.+}-{4:4}, at: ttm_pool_shrink+0xb2/0x330 [ttm]
  Chain exists of:
    (init_mm).mmap_lock --> fs_reclaim --> pool_shrink_rwsem

The cycle is built from three edges:

  1) pool_shrink_rwsem -> (init_mm).mmap_lock

     The TTM shrinker restores the caching attribute of every page it
     frees, while holding pool_shrink_rwsem:

       ttm_pool_shrink()
       -> ttm_pool_dispose_list()
       -> ttm_pool_free_page()
       -> set_pages_wb()
       -> change_page_attr_set_clr()   [ init_mm mmap read lock ]

  2) fs_reclaim -> pool_shrink_rwsem

     The same shrinker, called from reclaim.

  3) (init_mm).mmap_lock -> fs_reclaim

     ioremap() installing a huge PUD mapping over an existing PMD table:

       ioremap_page_range()
       -> vmap_range_noflush()
       -> vmap_try_huge_pud()          [ init_mm mmap read lock ]
       -> pud_free_pmd_page()
       -> __get_free_page(GFP_KERNEL)  [ enters reclaim ]

[ dhansen: Lots of changelog munging/trimming and merged comments from my
	   version of the fix. ]

Fixes: d5d8b8662e ("x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF")
Suggested-by: Pedro Falcato <pfalcato@suse.de>
Signed-off-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Link: https://lore.kernel.org/20260916062222.27347-1-mikhail.v.gavrilov@gmail.com
Link: https://lore.kernel.org/all/e11449f0-d9ad-4d1b-ab21-2be7d71fe335@intel.com/ [1]
Link: https://patch.msgid.link/20260923223116.20090-1-mikhail.v.gavrilov@gmail.com
Cc: stable@vger.kernel.org
2026-09-28 08:08:16 -07:00
Laurent Wandrebeck
d2457a7e27 x86/mm: Don't apply va_align to hugetlb mappings on AMD F15h
get_align_mask() returns huge_page_mask_align() for hugetlbfs, but get_align_bits()
adds va_align.bits regardless, so vm_unmapped_area() returns an address off the huge
page boundary and __unmap_hugepage_range() hits BUG_ON(start & ~huge_page_mask(h)) at
teardown.

This can be triggered on Carrizo and FX-8370E, both hstates.

Pass the file to get_align_bits() and skip the randomisation for hugetlbfs.

  [ bp: Massage commit message. ]

Fixes: 1317a5e7f7 ("arch/x86: teach arch_get_unmapped_area_vmflags to handle hugetlb mappings")
Suggested-by: Dave Hansen <dave.hansen@intel.com>
Acked-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Laurent Wandrebeck <l.wandrebeck@quelquesmots.fr>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Cc: stable@vger.kernel.org # 6.13+
Link: https://patch.msgid.link/20260922085032.46144-1-l.wandrebeck@quelquesmots.fr
2026-09-27 19:46:45 -07:00
Linus Torvalds
efb44d93a6 x86 fixes:
- Fix preemption bugs in the SVSM vTPM guest implementation
    (Melody Wang)
 
  - Fix MCE-triggered hardware debug register corruption on
    task migration (Masami Hiramatsu)
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmq4zuYRHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1iXThAAsqCXcDx+RNrPihj8j4G7qLBx5P7q+8I0
 HQ+LWhdIp/wEOSYVUk4GOX4r13bPg1S6BZl9YZlcV8A5uspQdK9hBvVjH+5g21ZH
 0GTepgxQs+OawPHWRPrRAmV8HvKKFvnqdDqCor7/HZgiW5aLozfA5+9EmzHcjUHZ
 emAWP6gnbliqJLyOHwbMCHSXnPywYaIBwGFAzWiXdkQzCu/SHvkeoQ0xyvCM+aoE
 cWGJD6S3MgvUOEtEQGpPNISpyXHRrorfNd3Km2ypzpoNbkYoompVW64+RkGUlHJH
 CmRyP/dK5h4Y0NhE8dZH3gn1FjvjPXT399vwkShYpc4LRLcGnLwk6cpVb4DHKwDG
 PUQSkV3akXuL/CndoMMcC7HYsYLriC02wiAYKLQGBM+q9O61iLhJin4f/5tvY219
 olyU40sPienXzFS+stenFoN46NuVq5tx7+1qURcO1kE8c2dohAixnJ/PVvl/zJCx
 +RSKQcW8rONn0aptg9RkKqlKxERpf0hIUCuNfOUmlcTG9q8zpCOkkfwGZ8LHoOFE
 P/FQ5uRAy9IF3egl+BUGOPLVlekJ3VxY6Vh6GPP1DwbUQqFoiEMYsfsW5jSgJcfN
 zP72bSSx9dKoPfI6dpDN1tUHFDFEsYU0hFWhfn5+Bo1LJdp1s6LZqRzDc4PSXtla
 RNrDnMS31sc=
 =1sLJ
 -----END PGP SIGNATURE-----

Merge tag 'x86-urgent-2026-09-27' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull x86 fixes from Ingo Molnar:

 - Fix preemption bugs in the SVSM vTPM guest implementation
   (Melody Wang)

 - Fix MCE-triggered hardware debug register corruption on
   task migration (Masami Hiramatsu)

* tag 'x86-urgent-2026-09-27' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/mce: Fix hardware debug register corruption on task migration
  x86/sev: Make vTPM SVSM calls preemption-safe
2026-09-27 08:44:12 -07:00
Linus Torvalds
5ccda18d1b Perf events fixes:
- Fixes for KVM guest PEBS virtualization (Sean Christopherson)
 
  - Fixes for various Intel PMUs related to PEBS data-source
    (Dapeng Mi)
 
  - Fix Intel Panther Cove event scheduling constraints (Dapeng Mi)
 
  - Fix Intel DMR/NVL OMR extra registers event scheduling (Dapeng Mi)
 
  - Rename two confusingly named PMU attributes (Dapeng Mi)
 
  - Fix a refcount leak in attach_perf_ctx_data() (Namhyung Kim)
 
  - Fix NULL pointer dereference crash in __perf_pmu_sched_task()
    (Puranjay Mohan)
 
  - Fix CPU-wide event scheduling (Puranjay Mohan)
 
  - Fix x86 LBR branch entry generation (Puranjay Mohan)
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmq4zGoRHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1guAw//U+WiZoWkWv6oFgAg+KrQFPr/OAzyIYOe
 Q355hrxdb2yh5+YGkLl/OCte/wIg1JiUq54gUBHPjEBUK8lxenNK2kOo6/Zd8tDx
 oGPyX6t0tpiP1O/PMII3yyd6Q7DYVC08TqOYW68r7Jv/fO2qCtBWZY3hxe0otU2/
 z+jqcQno3o8DbzfJnlDJb5honJos8CaT5FA+FkZkvjBF4dMWVErQowefdz2Zd78H
 Bu/wxmPG+Hej7ownQalps+ZA52KuQboRwJ+NCmas+fbFCcaWRbdDKTPbvEvzBsHT
 nybzlmBdneikrM0aNAXtBxJCLNzVrB6ffmKNFnd3CyfXmNtd/zleoqcKZpII2xO3
 UqlnwWR+u2JMGscvWiAAtlkdI0K5P4U1sHObhvzyXwgi3LgRZWJoLoIyiYWGcfnW
 Mxe3MCTy0wS+AEKRDvzg+rRr4jqTfl0BsHyzaTyTmWwx0LEg+7zwdryeixpXBbwH
 qC/SWkr4sfQlO/kVZHGa1sdHTZocRLcLl3YHsLiwQygu3rjNes+olKyPTtbvt9/a
 ZePBdcP+84K146VUczNJRAI4qt5b/X2aMvIYYxVvfDF6kiL0GbXgAfhdNVm5BIZF
 u5e2Sb6AX3w78V+YaMX1DuF+ZaFrpQd2UKOkBJPLvFnHTQr2eP2Fa6XfVMXnMcDy
 U7APgunF6D8=
 =Ggeg
 -----END PGP SIGNATURE-----

Merge tag 'perf-urgent-2026-09-27' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull perf events fixes from Ingo Molnar:

 - Fixes for KVM guest PEBS virtualization (Sean Christopherson)

 - Fixes for various Intel PMUs related to PEBS data-source (Dapeng Mi)

 - Fix Intel Panther Cove event scheduling constraints (Dapeng Mi)

 - Fix Intel DMR/NVL OMR extra registers event scheduling (Dapeng Mi)

 - Rename two confusingly named PMU attributes (Dapeng Mi)

 - Fix a refcount leak in attach_perf_ctx_data() (Namhyung Kim)

 - Fix NULL pointer dereference crash in __perf_pmu_sched_task()
   (Puranjay Mohan)

 - Fix CPU-wide event scheduling (Puranjay Mohan)

 - Fix x86 LBR branch entry generation (Puranjay Mohan)

* tag 'perf-urgent-2026-09-27' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf/core: Fill branch entries with a single assignment
  perf/core: Run sched_task() for PMUs with only CPU-wide events
  perf/core: Fix NULL pmu_ctx passed to pmu->sched_task()
  perf/core: Fix a refcount leak in attach_perf_ctx_data()
  perf/x86/intel: Rename NVL offcore_rsp attribute to offmodule_rsp
  perf/x86/intel: Rename DMR offcore_rsp attribute to offmodule_rsp
  perf/x86/intel: Fix precise OMR event scheduling for DMR/NVL
  perf/x86/intel: Constrain Panther Cove UOPS_DISPATCHED events to PMCs 0-3
  perf/x86/intel: Delete dead NVL PEBS data-source initcall
  perf/x86/intel: Fix Panther Cove PEBS data-source snoop states
  perf/x86/intel: Remove incorrect Panther Cove PEBS data-source constraints
  perf/x86/intel: Remove incorrect LionCove PEBS data-source constraints
  perf/x86/intel: Update arw_latency_data() mem-op direction handling
  perf/x86/intel: Fix DKT PEBS load/store direction for latency events, to fix sample classification
  perf/x86/intel: Fix CMT PEBS load/store direction for latency events, to fix sample classification
  perf/x86/intel: Fix GRT PEBS load/store direction for latency events, to fix sample classification
  perf/x86/intel: Make @data a mandatory param for intel_guest_get_msrs()
  perf/x86/intel: Don't pointlessly context switch DS_AREA (and PEBS config) if PEBS is unused
  perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has PEBS isolation, to fix stuck PEBS_ENABLED
  perf/x86/intel: Ensure KVM guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits
2026-09-27 08:15:58 -07:00
Linus Torvalds
fddfc3ec31 pci-v7.3-fixes-2
-----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAmq38q4UHGJoZWxnYWFz
 QGdvb2dsZS5jb20ACgkQWYigwDrT+vwVFhAAhO4zr6WHzB/Tx87FGSxYnjrwzOmH
 nS/w0MD0FxPkAzhbkPaXkdgAks10dBNF/umQUXvWBwgZYbR9v0CoZBSm8viuq/8C
 tYOdOm4u4whsHVPU1QJPV5nUSGb2Eiq7vza+2kS1Ll0xy/3/24yVn+IU8sy4EcgX
 IUTwApHfBGq6se88LxtdT5ctAUCRSP8XAeFlKXqfPig9yrrQOZ0cJrA5nhbe4jCL
 jJMFjgH12OvR55ZeKyZianz/lCZ8NcyATblM0EUCVKpx1gv4BcPfbprcycgnyLbw
 R+SoPl0wJDWKGqwuCFYkLMaT4XsLYnJv+6C/aQQ8CJjovAVUzaeDXwlJzPP3hH+N
 iIZp3b5tULHK/c/AykjoRQPq/rVy9ywMvlqmafIeR4+BEJBVYlUGLhum14pnjAQo
 B/wLCSmxrP7K7E+71ODunUdpts6HsH+GbKqSjBS46BAY/Vw+icq0v5RxaIaa6WCv
 BEkVEIfcJ/pnlmxMCoQEVu0bjCrIIaTe7EflO4BDhEIrWTxSq19vCVlXB+XgsHUI
 DWPyCFnAWFG5bZi50PKkrwTOhmcRMRgAjc69DOZ400chbuMAj9PxpomYBEeWvnss
 N+DLTLL2Yp4Y8/tMJIxjNB2eAbiSKql2EoGIpyjQruDRmg1Bcug5gBY90RjRr204
 afoNJYCbtPnd0mY=
 =hhCg
 -----END PGP SIGNATURE-----

Merge tag 'pci-v7.3-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci

Pull PCI fixes from Bjorn Helgaas:

 - Make BAR resize work even for devices where no upstream bridge is
   visible to the OS, which fixes an amdgpu regression on SolidRun
   HoneyComb, which doesn't expose Root Ports to the OS (Liz Fong-Jones)

 - Omit bus properties in dynamic OF nodes when a bridge has no
   subordinate bus, which fixes early boot hangs caused by NULL pointer
   dereferences with CONFIG_PCI_DYNAMIC_OF_NODES enabled (Angel J)

 - Disable enhanced atomics on AMD NBIO 7.7 and 7.11 to avoid silent
   data corruption on 64-bit DMAs (Mario Limonciello)

* tag 'pci-v7.3-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci:
  x86/PCI: Disable enhanced atomics on AMD NBIO 7.7 and 7.11
  PCI: of_property: Omit bus properties without a subordinate bus
  PCI: Fix BAR resize for devices on a root bus
2026-09-26 09:59:59 -07:00
Linus Torvalds
eff8d2791c Arm:
- Invalidate the ITS translation cache when the guest changes the
   base address of the ITS tables (Fuad Tabba)
 
 - Skip saving ITS devices with device IDs that are out-of-bounds
   rather than failing the entire ITS save ioctl (Fuad Tabba)
 
 - Close race between VM teardown and invalidations of nested MMUs
   when handling MMU operations that are allowed to block
   (Lorenzo Stoakes)
 
 - Various fixes for the handling of the host's untrusted SVE
   configuration in pKVM (Fuad Tabba)
 
 - Make sure that empty SMCCC ranges based at 0 are rejected by the
   kvm_smccc_set_filter() (Karl Mehltretter)
 
 - Revoke the host mapping for pKVM's private stack pages, along
   with a new sanity check that all mappings in the hyp's private
   VA range have been correctly marked as hyp-owned (Fuad Tabba)
 
 - Lifetime fixes for the array of shadow stage-2 MMUs, ensuring that
   concurrent vCPU initialization cannot relocate in-use MMUs. Defer
   the freeing of shadow stage-2 MMUs to the point that no other users
   (e.g. MMU notifier) could reference them (Marc Zyngier)
 
 - Drop useless WARN when rejecting an unsupported ioctl for pKVM
   (Fuad Tabba)
 
 - Fix the steal_time selftest to install correctly-sized mappings for
   non-4K hosts (Sebastian Ott)
 
 - Correct mapping of fine-grained trap for GCSPOPX instruction
   (Mark Brown)
 
 - Fix KVM_BUG_ON() due to missing handling of DBGBXVR<n> from 32-bit
   guests (Karl Mehltretter)
 
 RISC-V:
 
 - Synchronize hrtimer during VCPU teardown.
 
 - Fix the conversion between vsip and hvip values.
 
 - Serialize IMSIC attributes with vCPU migration.
 
 - Release unused page after MMU invalidation.
 
 - Propagate interrupted G-stage faults to KVM user-space as EINTR.
 
 - Fix nested acceleration hfence entry update order.
 
 - Fix sdata leak and stale snapshot_addr in snapshot_set_shmem.
 
 - Preserve firmware counter value across PMU counter stop/start.
 
 - Report PMU snapshot write failure to the guest.
 
 - Fix perf-backed counter accounting across PMU stop and read.
 
 - Correctly propagate error of a hart status SBI call.
 
 s390:
 
 - Ensure that accesses through kvm_arch_set_irq_inatomic mark as dirty
   the pages that contain indicator and summary bits.
 
 - Fix compile warning for kvm_s390_update_cmma_dirty().
 
 - Fix incorrect propagation of ENOENT from _gaccess_shadow_fault() to
   userspace.
 
 - Move s390_kvm_mmu_commit_memory_region() into
   s390_kvm_mmu_prepare_memory_region() so that it can fail instead of WARN.
 
 - Add missing srcu in kvm_s390_set_irq_state().
 
 - Fix potential races in storage functions.
 
 - Fix race in _destroy_pages_crste().
 
 - Fix issues in the handling of KVM interrupt and page resources, when a
   queue that is assigned to a mediated device (mdev) is removed from the
   host's AP configuration.
 
 - Fix loop condition in uv_find_secrets.
 
 - Prevent potential out-of-bounds read.
 
 x86:
 
 - Fix a brown paper bag bug where KVM would incorrectly treat Intel PMU MSRs
   as valid on AMD.
 
 - Fix a regression in the hardware disable selftest where it checked the wrong
   macro when detecting glibc support (breaks at least musl).
 
 - Never clear KVM_REQ_VM_DEAD so that dead VMs stay dead, which is especially
   important for KVM_BUG_ON() flows, which often guard more dangerous bugs.
 
 - Re-pend GET_NESTED_STATE_PAGES if getting the pages fails, to fix a bug
   where KVM would let userspace run a broken setup with stale vmcs12 pages.
 
 - Fix a class of bugs where KVM would fail to fill kvm_run exit fields if
   getting nested pages failed.
 
 - Treat reserved entries in the memory attributes xarray as "no attributes",
   to fix false positives when checking for mixed attributes.
 
 - Fix memcg accounting for the memory attributes xarray (the xarray library
   subtly requires the xarray to be configured for accounting upfront; the gfp
   flags taken at runtime are used only rarely).
 
 - Don't pre-reserve xarray entries when storing empty attributes, as storing
   NULL must not require memory allocation (KVM and other subsystems heavily
   rely on this behavior).
 
 - Fix a memory leak and a cache maintenance issue related to doing intra-host
   migration on an SEV guest.
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmq3Ta0UHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroMcygf/Z8GovgUACbWs/PSgVxZr5++MsbnM
 /N+vUP0VhsBxWCt191fBmKSBMNyTL734CKe0fkmisIAIPxf63uA3lttfMshNXKct
 VJ3TQkbA7qiSEGf2Um1dp4k6egqDDbTS1a4NF7CM7AEL4Wm3YDfqv2h4rZicEBsd
 Spy4yV8sKoiEHMnduhF2m5r0cWkrj194e2dolJy0p1WiNeEykf/nMTmNq5JWusBh
 TDDzZf1p2BianOmQ6+fkzAfWDSIdv0OPmoH7iqvQ/xhvm4dLWq2oeOz5Q2/uWAp0
 BqU4Z5zBrGRUWyXM0cDZEqBfmNH4v7oR9PfLgvD4qpMoqONuMj1+mIO68A==
 =XtLD
 -----END PGP SIGNATURE-----

Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm

Pull kvm fixes from Paolo Bonzini:
 "Arm:

   - Invalidate the ITS translation cache when the guest changes the
     base address of the ITS tables (Fuad Tabba)

   - Skip saving ITS devices with device IDs that are out-of-bounds
     rather than failing the entire ITS save ioctl (Fuad Tabba)

   - Close race between VM teardown and invalidations of nested MMUs
     when handling MMU operations that are allowed to block (Lorenzo
     Stoakes)

   - Various fixes for the handling of the host's untrusted SVE
     configuration in pKVM (Fuad Tabba)

   - Make sure that empty SMCCC ranges based at 0 are rejected by the
     kvm_smccc_set_filter() (Karl Mehltretter)

   - Revoke the host mapping for pKVM's private stack pages, along with
     a new sanity check that all mappings in the hyp's private VA range
     have been correctly marked as hyp-owned (Fuad Tabba)

   - Lifetime fixes for the array of shadow stage-2 MMUs, ensuring that
     concurrent vCPU initialization cannot relocate in-use MMUs. Defer
     the freeing of shadow stage-2 MMUs to the point that no other users
     (e.g. MMU notifier) could reference them (Marc Zyngier)

   - Drop useless WARN when rejecting an unsupported ioctl for pKVM
     (Fuad Tabba)

   - Fix the steal_time selftest to install correctly-sized mappings for
     non-4K hosts (Sebastian Ott)

   - Correct mapping of fine-grained trap for GCSPOPX instruction (Mark
     Brown)

   - Fix KVM_BUG_ON() due to missing handling of DBGBXVR<n> from 32-bit
     guests (Karl Mehltretter)

  RISC-V:

   - Synchronize hrtimer during VCPU teardown

   - Fix the conversion between vsip and hvip values

   - Serialize IMSIC attributes with vCPU migration

   - Release unused page after MMU invalidation

   - Propagate interrupted G-stage faults to KVM user-space as EINTR

   - Fix nested acceleration hfence entry update order

   - Fix sdata leak and stale snapshot_addr in snapshot_set_shmem

   - Preserve firmware counter value across PMU counter stop/start

   - Report PMU snapshot write failure to the guest

   - Fix perf-backed counter accounting across PMU stop and read

   - Correctly propagate error of a hart status SBI call

  s390:

   - Ensure that accesses through kvm_arch_set_irq_inatomic mark as
     dirty the pages that contain indicator and summary bits

   - Fix compile warning for kvm_s390_update_cmma_dirty()

   - Fix incorrect propagation of ENOENT from _gaccess_shadow_fault() to
     userspace

   - Move s390_kvm_mmu_commit_memory_region() into
     s390_kvm_mmu_prepare_memory_region() so that it can fail instead of
     WARN

   - Add missing srcu in kvm_s390_set_irq_state()

   - Fix potential races in storage functions

   - Fix race in _destroy_pages_crste()

   - Fix issues in the handling of KVM interrupt and page resources,
     when a queue that is assigned to a mediated device (mdev) is
     removed from the host's AP configuration

   - Fix loop condition in uv_find_secrets

   - Prevent potential out-of-bounds read

  x86:

   - Fix a brown paper bag bug where KVM would incorrectly treat Intel
     PMU MSRs as valid on AMD

   - Fix a regression in the hardware disable selftest where it checked
     the wrong macro when detecting glibc support (breaks at least musl)

   - Never clear KVM_REQ_VM_DEAD so that dead VMs stay dead, which is
     especially important for KVM_BUG_ON() flows, which often guard more
     dangerous bugs

   - Re-pend GET_NESTED_STATE_PAGES if getting the pages fails, to fix a
     bug where KVM would let userspace run a broken setup with stale
     vmcs12 pages

   - Fix a class of bugs where KVM would fail to fill kvm_run exit
     fields if getting nested pages failed

   - Treat reserved entries in the memory attributes xarray as "no
     attributes", to fix false positives when checking for mixed
     attributes

   - Fix memcg accounting for the memory attributes xarray (the xarray
     library subtly requires the xarray to be configured for accounting
     upfront; the gfp flags taken at runtime are used only rarely)

   - Don't pre-reserve xarray entries when storing empty attributes, as
     storing NULL must not require memory allocation (KVM and other
     subsystems heavily rely on this behavior)

   - Fix a memory leak and a cache maintenance issue related to doing
     intra-host migration on an SEV guest"

* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (54 commits)
  KVM: SEV: Do cache maintenance on the source VM during intra-host migration
  KVM: SEV: Free have_run_cpus during VM destruction even if VM is no longer SEV
  KVM: Don't pre-reserve xarray entries when storing empty/NULL attributes
  KVM: Ensure memory attributes xarray nodes are accounted to the caller's memcg
  KVM: Don't treat reserved xarray entries as having memory attributes
  KVM: x86: Fill kvm_run exit fields in common get_nested_state_pages() error paths
  KVM: x86: Re-pend GET_NESTED_STATE_PAGES if getting said pages fails
  KVM: arm64: Fix AArch32 DBGBXVR<n> handling
  KVM: arm64: Fix FGT mapping for HFGITR_EL2.nGCSEPP
  KVM: selftests: fix steal_time for arm64 with host page size > 4K
  KVM: arm64: Don't WARN on an unknown VM ioctl in protected mode
  KVM: arm64: nv: Delay freeing of shadow S2 structures until VM destruction
  KVM: arm64: nv: Fix life cycle of the nested_mmus array
  KVM: arm64: Check every private mapping is hyp-owned at pKVM init
  KVM: arm64: Move the private VA allocation cursor to __io_map_next
  KVM: arm64: Match hyp text by physical address in fix_host_ownership()
  KVM: arm64: Transfer the hyp stack pages out of the host stage-2
  KVM: arm64: selftests: Test empty SMCCC filter range at base 0
  KVM: arm64: Return -EINVAL for an empty SMCCC filter range at base 0
  KVM: arm64: Derive GUEST_HAS_SVE from the SVE feature bit at EL2
  ...
2026-09-26 08:26:12 -07:00
Sean Christopherson
2abb25273c KVM: nVMX: Force MSR bitmap refresh if runtime eVMCS controls are modified
Force a refresh of the vmcs02 MSR bitmap during nested VM-Enter if the
runtime eVMCS controls (pin, primary, secondary, etc.) are being updated.
If L1 isn't intercepting TPR writes, runs L2 with TPR virtualization, and
then runs the same L2 with TPR virtualization disabled, KVM will fail to
refresh msr_bitmap02 and leave TPR in passthrough mode even though TPR
virtualization is disabled.  I.e. failure to refresh the bitmap lets L2 (or
L1 by proxy) read and write L0's TPR.

Fixes: 502d2bf5f2 ("KVM: nVMX: Implement Enlightened MSR Bitmap feature")
Cc: stable@vger.kernel.org
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 02:22:15 -04:00
Sean Christopherson
9e06bbb9ad KVM: SVM: Use the active VMCB's MSR bitmap when checking if MSR is intercepted
Use the MSR permission bitmap of the active VMCB instead of assuming that
KVM is always using vmcb02's bitmap when L2 is active, as KVM uses msrpm02
if and only if L1 wants to intercept MSR accesses, i.e. if and only if KVM
needs to merge msprm01 with msrpm12.

Don't bother tracking the virtual address of the bitmap that's being used,
as __va() is cheap on x86, and caching the virtual address would introduce
yet another source of potentially stale information.

Fixes: b2ac58f905 ("KVM/SVM: Allow direct access to MSR_IA32_SPEC_CTRL")
Cc: stable@vger.kernel.org
Reported-by: Stefan Teodorescu <fane@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260826195833.844526-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 02:22:12 -04:00
Sean Christopherson
73392706d0 KVM: SVM: Sync guest's PERF_CNTR_GLOBAL_CTL from h/w only on successful VMRUN
Don't (re)read PERF_CNTR_GLOBAL_CTL from hardware on a failed VMRUN, as the
purpose of the read is to synchronize KVM's cache with any writes done by
the guest, and the guest can't possibly have modified the MSR if it never
got a chance to run.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260904170642.3291466-5-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 01:15:43 -04:00
Sean Christopherson
64c5b15af7 KVM: SVM: Don't mark ASID fields as dirty when setting control.tlb_ctl
Don't mark the ASID as dirty in the VMCB when requesting a TLB flush via
control.tlb_ctl.  Per "15.15.3 VMCB Clean Field" of the July 2026, Revision
3.45 version of the APM:

  The following are explicitly not cached and not represented by Clean bits:

    * TLB_Control

Fixes: 7e8e6eed75 ("KVM: SVM: Move asid to vcpu_svm")
Suggested-by: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260904170642.3291466-4-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 01:15:43 -04:00
Sean Christopherson
e27fc915c8 KVM: SVM: Update control fields on #VMEXIT if and only if VMRUN succeeded
Leave control.erap_ctl and control.clean as-is in the VMCS if VMRUN fails,
because as per AMD:

  there's no explicit architectural guarantee about the behavior in the
  presence of VMRUN failures. So the best thing to do would be to assume
  that if VMRUN fails, the actions requested in the control fields may not
  have been performed.

Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260904170642.3291466-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 01:15:43 -04:00
Sean Christopherson
60d93c2785 KVM: SVM: Preserve TLB control (i.e. pending TLB flush) on failed VMRUN
Don't reset the VMCB's TLB control back to "do nothing" on a failed VMRUN,
as empirical testing shows that the CPU performs the requested TLB flush if
and only if VMRUN is successful, i.e. clearing TLB control on a failed
VMRUN effectively drops a TLB flush.

Explicitly track the need to flush all ASIDs on a per-CPU basis, as the
ASID reuse condition is tied to the pCPU, not to the vCPU.  As a bonus,
this also obviates the need to avoid clobbering FLUSH_ALL_ASID with
TLB_CONTROL_FLUSH_ASID, e.g. in svm_flush_tlb_asid().

Deliberately don't bother saving/restoring the "old" tlb_ctl on failure,
in quotes because it's not exactly the old tlb_ctl, it's the tlb_ctl from
after pre_svm_run(), but before updating tlb_ctl for flush_all_asids.  If
VMRUN fails and TLB_CONTROL_FLUSH_ALL_ASID is forced, then the next
successful run of the VMCB *may* unnecessarily flush all ASIDs, which
strictly speaking could result in noisy neighbor issues.  However, the
fact that new_asid() is already guest-triggerable, because of KVM's flawed
behavior of clearing the ASID on emulated INIT, means that a guest can
already trigger a flush of all ASIDs at roughly the same rate.  And once
KVM stops clobbering the ASID on emulated INIT, *or* assigns a static ASID
to each vCPU, this flaw goes away.

Fixes: 38e5e92fe8 ("KVM: SVM: Implement Flush-By-Asid feature")
Cc: stable@vger.kernel.org
Reported-by: Stefan Teodorescu <fane@google.com>
Suggested-by: Yosry Ahmed <yosry@kernel.org>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jim Mattson <jmattson@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260904170642.3291466-2-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 01:15:43 -04:00
Sean Christopherson
b6249c1d56 KVM: x86/mmu: Bail from shadow walks if the root is invalid or a dummy
When walking shadow page tables, immediately terminate the walk if the root
is a "dummy" root, i.e. a root whose top-level page table is backed by the
zero page, but otherwise doesn't exist.  If memslot creation races with a
stage-2 page fault (EPT violation or #NPF) from L2, then if the stars align,
KVM will attempt to walk shadow page tables using the zero page and hit a
NULL pointer deref.

  BUG: kernel NULL pointer dereference, address: 0000000000000021
  #PF: supervisor read access in kernel mode
  #PF: error_code(0x0000) - not-present page
  PGD 0 P4D 0
  Oops: Oops: 0000 [#1] SMP
  CPU: 30 UID: 1000 PID: 941 Comm: qemu Not tainted 7.2.0-rc2-1b731e5ded48-next-vm #1741 PREEMPTLAZY
  Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
  RIP: 0010:__kvm_mmu_invalidate_addr+0xea/0x210 [kvm]
  Call Trace:
   <TASK>
   kvm_mmu_invalidate_addr+0x92/0xe0 [kvm]
   __kvm_inject_emulated_page_fault+0x67/0x80 [kvm]
   ept_page_fault+0x160/0x850 [kvm]
   kvm_mmu_do_page_fault+0x102/0x1f0 [kvm]
   kvm_mmu_page_fault+0x8e/0x6b0 [kvm]
   vmx_handle_exit+0x163/0x640 [kvm_intel]
   kvm_arch_vcpu_ioctl_run+0x960/0x2120 [kvm]
   kvm_vcpu_ioctl+0x2c7/0x970 [kvm]
   __x64_sys_ioctl+0x90/0xd0
   do_syscall_64+0x67/0x5f0
   entry_SYSCALL_64_after_hwframe+0x4b/0x53
  RIP: 0033:0x7f606d0b53bb

Opportunistically harden the shadow walks against fully invalid roots, but
WARN, as all callers are expected/required to pre-check for a valid root.
Don't WARN in the dummy root case as the whole point of using a dummy root
is to provide a root that's valid enough to enter the guest, i.e. it should
Just Work for all flows except those that *need* to know about dummy roots.

Alternatively, KVM could allocate a dedicated page and associated shadow
page structure for the dummy root, which is very tempting as it such an
approach should be more resilient against unexpected behavior.  But that
would be a much larger and thus riskier change than simply terminating
walks of dummy roots.

Fixes: 0e3223d8d0 ("KVM: x86/mmu: Use dummy root, backed by zero page, for !visible guest roots")
Reported-by: Gabriel Schneider <gbrls@osec.io>
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260925234256.2384816-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 01:15:43 -04:00
Sean Christopherson
6418b715d5 KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
When creating a vCPU, don't drop kvm->lock to when doing the bulk of actual
vCPU creation, as allowing multiple vCPUs to be created in parallel adds
significant complexity in KVM (as evidenced by the many related bugs), and
all known VMMs fully serialize vCPU creation.  Remove all manually locking
of kvm->lock from kvm_arch_vcpu_{post,}create() for obvious reasons.

For many years, "everyone" has assumed that dropping kvm->lock was done for
performance reasons optimization, e.g. to allow userspace to create all
vCPUs concurrently for latency purposes.  But as above, no known VMM does
that.  Looking at the history of this code, before commit 11ec280471
("KVM: Convert vm lock to a mutex"), kvm->lock was a spinlock.  I.e. KVM
*had* to drop kvm->lock when doing the bulk of vCPU creation, otherwise KVM
couldn't do normal memory allocations.  When kvm->lock got turned into a
mutex for unrelated reasons, no one took advantage updated of the change to
simplify vCPU creation.  And 19 years later, everyone just assumed that KVM
continued to deal with the complexity for performance reasons.

Furthermore, naively parallelizing vCPU creation in userspace is likely a
net negative due to the overheads of task creation.  Unless a VMM carefully
avoids the extra overhead related to parallelization, e.g. spawns each
vCPU's thread before creating the vCPU, creating vCPUs concurrently is a
net *negative* up until about ~64 vCPUs, after which the times are a wash.

The absolute speed of light _is_ faster if KVM doesn't hold kvm-lock, but
at vCPU counts of ~16 or less, it's probably in the noise when considering
total VM creation time, as the added latency is less than 1ms up until 16
or so vCPUs.

On top of all that, KVM has had a *lot* of fatal bugs (most often found by
syzkaller) related to vCPUs being created while trying to do per-VM
operations (basically, see every flow that locks all vCPUs).  I.e. the
parallel vCPU creation "support" is actively harmful as the only "use case"
is for misbehaving userspace to exploit KVM bugs.

Serializing vCPU creation will allow reverting commit 97d65b544f ("KVM:
Check for duplicate vcpu_id as early as possible"), which had "minor" math
error: the worst case scenario isn't "256 bytes per VM", it's "256 unsigned
longs per VM", i.e. 2048 bytes per VM, which doubles the size of each VM
and pushes several architectures into order-1 allocations.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Tested-by: Jean-Christophe Guillain <jean-christophe@guillain.net>
Tested-by: Naveen N Rao (AMD) <naveen@kernel.org>
Message-ID: <20260921174445.911676-5-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 00:49:49 -04:00
Sean Christopherson
10f4082b75 KVM: RISC-V: Use kvm_is_vcpu_creation_in_progress() instead of open-coded equivalent
Use kvm_is_vcpu_creation_in_progress() instead of an open-coded equivalent
during AIA initialization.  Unlike similar vGIC code in arm64, the relevant
RISC-V code runs under kvm->lock, i.e. can use the standard API without
hitting lockdep false positive.

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Tested-by: Jean-Christophe Guillain <jean-christophe@guillain.net>
Tested-by: Naveen N Rao (AMD) <naveen@kernel.org>
Message-ID: <20260921174445.911676-4-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 00:49:49 -04:00
Sean Christopherson
e3995b05e9 KVM: arm64: vgic: Rely on vCPU creation check in "trylock all vCPUs"
Now that KVM's APIs for locking all vCPUs return -EBUSY if vCPU creation is
in-progress, drop the manual check for the same from vGIC creation, and
update the comments accordingly.

Note, while KVM arm64 guards many vGIC operations with its arch-specific
config_lock, holding kvm->lock is sufficient to guarantee a stable result
for "is vCPU creation in-progress".  So, no functional change intended.

Note #2, the open coded check in vgic_init() is racy when called without
kvm->lock held, e.g. via vgic_lazy_init().  I.e. that check needs to stay
open coded to avoid triggering a lockdep assert.  Whether or not the race
is "fine" is a problem for a different day.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Tested-by: Jean-Christophe Guillain <jean-christophe@guillain.net>
Tested-by: Naveen N Rao (AMD) <naveen@kernel.org>
Message-ID: <20260921174445.911676-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 00:49:49 -04:00
Sean Christopherson
8264c0d73f KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
Reject locking of all vCPUs if vCPU creation is in-progress, i.e. if the
number of "created" vCPUs doesn't match the number of "onlined" vCPUs.
It's simply not possible to guarantee that KVM has truly locked all vCPUs
if one or more vCPUs are actively being created.  Holding kvm->lock does
prevent in-flight vCPUs from being fully onlined, but it's infeasible for
common KVM to know whether or not that provides sufficient protection.

In practice, this is likely a minor bug fix for the ARM and RISC-V usage of
kvm_trylock_all_vcpus(), and a glorified nop for everything else.  E.g.
ARM's kvm_timer_vcpu_init() can race kvm_vm_ioctl_set_counter_offset() with
respect to observing KVM_ARCH_FLAG_VM_COUNTER_OFFSET.

Opportunistically drop x86's existing manual checks on vCPU creation being
in-progress as all of x86's checks immediately precede or follow locking of
all vCPUs.  Leave arm64 and RISC-V alone for the moment, as their checks
aren't as obviously redundant/equivalent.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Tested-by: Jean-Christophe Guillain <jean-christophe@guillain.net>
Tested-by: Naveen N Rao (AMD) <naveen@kernel.org>
Message-ID: <20260921174445.911676-2-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 00:49:49 -04:00
Paolo Bonzini
c2f24f140c KVM fixes for 7.3-rcN
- Fix a brown paper bag bug where KVM would incorrectly treat Intel PMU MSRs
    as valid on AMD.
 
  - Fix a regression in the hardware disable selftest where it checked the wrong
    macro when detecting glibc support (breaks at least musl).
 
  - Never clear KVM_REQ_VM_DEAD so that dead VMs stay dead, which is especially
    important for KVM_BUG_ON() flows, which often guard more dangerous bugs.
 
  - Re-pend GET_NESTED_STATE_PAGES if getting the pages fails, to fix a bug
    where KVM would let userspace run a broken setup with stale vmcs12 pages.
 
  - Fix a class of bugs where KVM would fail to fill kvm_run exit fields if
    getting nested pages failed.
 
  - Treat reserved entries in the memory attributes xarray as "no attributes",
    to fix false positives when checking for mixed attributes.
 
  - Fix memcg accounting for the memory attributes xarray (the xarray library
    subtly requires the xarray to be configured for accounting upfront; the gfp
    flags taken at runtime are used only rarely).
 
  - Don't pre-reserve xarray entries when storing empty attributes, as storing
    NULL must not require memory allocation (KVM and other subsystems heavily
    rely on this behavior).
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmq3A78ACgkQOlYIJqCj
 N/1uVA//TVDznH49htPNV4dh7vfiE8oF9ka4e9KTPA2eMdDeNVInNXBz0ZbJCdNN
 Hm7iZZfk8yx0JCCGXdp8EoYg66iwulBr0KZXRP2U2LoyiRELA1VtGGOLUVP3PXjF
 hElXD56+RrkJJSucINlFU7kSfl2hY/Jbaq87HVBLpXp8iY3stBbLdgMT1ReLdorO
 s9i2uJRqHKjtSG330qDYEG1TtRAdUehgRbxbbJVuOEYtU37tCvX6ai8dEjZAvxYo
 1VZOD05ITo12HUXPiZjIMoNTujQnZKmGYlBYczWlfOmkY/tN6SV8M41srV3ZcK0k
 XGOQM6tOzbVTWaSH/pW1krklnOFAbnL6QzHzb3enHZfUN8AikHPDyfsJWzt/s0f/
 ATTQk03OYL4fJMQSQ778EI6S4PcOzY9HiBxDXM5ySpUv1uyXw8ZFsQSdH/TYTMhi
 Dce42r5rlGa1vAKHwI1Xya2su3REhWzxYxGpUOg/PtRRqWvL2RMrHArzEd6zu+Ti
 jPinuQuXYcT1FTCa5S39NWhU4gWpjw8Pvs9VuUYSCDYxZCaqQ2YheYgsAcgPqgfE
 hS43XVmmOB/tXGsvu+/YuEfbzNV/mHOuJBcjBnnsBP3d6EFB02FPKg4/LYiXujYo
 OocUbSB7bKd87W3B+Ys3QmgOtZh4qX/aLYusa2vQ4Dp9ujXqcWs=
 =LG0B
 -----END PGP SIGNATURE-----

Merge tag 'kvm-x86-fixes-7.3-rc5' of https://github.com/kvm-x86/linux into HEAD

KVM fixes for 7.3-rcN

 - Fix a brown paper bag bug where KVM would incorrectly treat Intel PMU MSRs
   as valid on AMD.

 - Fix a regression in the hardware disable selftest where it checked the wrong
   macro when detecting glibc support (breaks at least musl).

 - Never clear KVM_REQ_VM_DEAD so that dead VMs stay dead, which is especially
   important for KVM_BUG_ON() flows, which often guard more dangerous bugs.

 - Re-pend GET_NESTED_STATE_PAGES if getting the pages fails, to fix a bug
   where KVM would let userspace run a broken setup with stale vmcs12 pages.

 - Fix a class of bugs where KVM would fail to fill kvm_run exit fields if
   getting nested pages failed.

 - Treat reserved entries in the memory attributes xarray as "no attributes",
   to fix false positives when checking for mixed attributes.

 - Fix memcg accounting for the memory attributes xarray (the xarray library
   subtly requires the xarray to be configured for accounting upfront; the gfp
   flags taken at runtime are used only rarely).

 - Don't pre-reserve xarray entries when storing empty attributes, as storing
   NULL must not require memory allocation (KVM and other subsystems heavily
   rely on this behavior).
2026-09-26 00:40:07 -04:00
Sean Christopherson
93de2a6a4b KVM: SEV: Do cache maintenance on the source VM during intra-host migration
Manually perform cache maintenance on the source VM during intra-host
migration to ensure no stale data is left in CPU caches after the VM is
destroyed.  Because the source VM is "converted" to a non-SEV VM, KVM's
memory reclaim flows won't trigger cache maintenance, e.g. when all guest
memory is reclaimed in response to detaching from the mmu_notifier.

Note, relying on the destination VM to do cache maintenance isn't an option
as KVM doesn't require identical guest memory configurations, i.e. the
source VM may have access to memory that the destination VM does not.
Enforcing equivalent memory configurations is infeasible, as it would
require a *deep* comparison of memslots, e.g. to verify that not only are
the memslot identical, but what the memslots point at is also identical.

Fixes: b56639318b ("KVM: SEV: Add support for SEV intra host migration")
Cc: stable@vger.kernel.org
Reported-by: Stefan Teodorescu <fane@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260923163721.1584779-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 00:39:55 -04:00
Sean Christopherson
12c1f6e03f KVM: SEV: Free have_run_cpus during VM destruction even if VM is no longer SEV
Unconditionally free SEV's "have run CPUs" cpumask in the VM destroy path,
i.e. even for what appear to be non-SEV VMs, as an SEV VM becomes a non-SEV
VM if its state is intra-host migrated.  Alternatively, the mask could be
freed in sev_migrate_from() when "converting" the source VM, but that gets
annoying because ideally KVM would nullify the mask to guard against UAF,
and nullifying the mask would need be conditioned on CPUMASK_OFFSTACK=y.

Freeing the mask during sev_migrate_from() is also not robust against other
KVM bugs, though that's kind of a moot point since any such bugs would show
up even if sev->active is never set.  I.e. KVM must get that side of things
correct.  But, that's not a great reason to add more code just to make
things marginally less robust.

Fixes: 6f38f8c574 ("KVM: SVM: Flush cache only on CPUs running SEV guest")
Cc: stable@vger.kernel.org
Reported-by: Stefan Teodorescu <fane@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260923163721.1584779-2-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2026-09-26 00:39:55 -04:00
Linus Torvalds
547463efb9 s390 fixes for 7.3-rc5
- Fix several bugs in PCI error recovery SCLP reporting: don't report
   success on skipped recovery, report errors when no pdev is associated,
   add missing device lock, and fix struct pci_dev reference leak in
   zpci_report_status()
 
 - Fix several bugs in CIO code: fix use of invalid SCHIB data, guard PMCW
   field accesses, check device number valid bit in PMWC before accessing
   other fields, and fix NULL pointer dereference in
   ccw_device_get_util_str()
 
 - Fix virtual vs physical address confusion in channel measurement
   facility code on kernels with CONFIG_RANDOMIZE_IDENTITY_BASE=y
 
 - Fix couple of bugs in s390dbf: fix copy of failed static debug areas,
   skip view registration on failure, and reject NULL pointer in
   debug_dump()
 
 - Fix sriov_numvfs attribute name in zPCI documentation
 
 - Fix typos in comments
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEECMNfWEw3SLnmiLkZIg7DeRspbsIFAmq2V1AACgkQIg7DeRsp
 bsJ7zg/7BrnPs0WyKv9NZvVUudYbz3nUIAleJo6HcDK2SUDr+EX3W4U7OjsXL6Fv
 H76hx5AqF6ddtUOJ2uevCAb3PKm0eA7vmK9YiERAclrBA+tWGW0xBwCqqyc84k38
 CPxOBr87Ae1jIIcfDxfBWujtKTFi/4BpSvKvKgg3mnd2egny4UnHnABx5ZcClEee
 p6aVjCKycQFHwNLMoJA1BxvGSpcXdiKXbm6VRJ4gCaBnVuzkLUv1ZMp/Pi1fWVbV
 E1AEztZ/03Tfry51q7wQLv6wD5lR3Z/Py2vZ4YIbelQwVMAMI+rFtgd5oFfo9qkF
 gcFJ+faE9m0E/nsoaaVCaa2Ja4+hV/vZ5ik0bq1ljEqEvInaa2XTuxNAljId351Y
 JrLvCdaip0FkMNMqEym0iydMAlpc491GAhzNrT3yvq6t03VVpxprPPA6VIwHg+zk
 UBMP+QdxtnaV8AAUzDD9E255d88N4DECBICUgf1XjFm/I7xLJFWyUvbzcqNn9VnY
 EGVBOeQmoByhCL9lN0+nzEM1sgT+hQB/C3WmLqiuLxrxjnc8dVr8rVHcPXFgARPw
 Ht3W9jxdTdl4VBwckOWmbvG4L6spXFLdEzI51cP46L1RKWhFjLl/6RoY8fUHf4Vz
 YnHutQevSoPI2D6G4v8K1FBz9wclxCojBPbXCmevfbL9B4ChPoQ=
 =r3uP
 -----END PGP SIGNATURE-----

Merge tag 's390-7.3-4' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux

Pull s390 fixes from Heiko Carstens:

 - Fix several bugs in PCI error recovery SCLP reporting: don't report
   success on skipped recovery, report errors when no pdev is
   associated, add missing device lock, and fix struct pci_dev reference
   leak in zpci_report_status()

 - Fix several bugs in CIO code: fix use of invalid SCHIB data, guard
   PMCW field accesses, check device number valid bit in PMWC before
   accessing other fields, and fix NULL pointer dereference in
   ccw_device_get_util_str()

 - Fix virtual vs physical address confusion in channel measurement
   facility code on kernels with CONFIG_RANDOMIZE_IDENTITY_BASE=y

 - Fix couple of bugs in s390dbf: fix copy of failed static debug areas,
   skip view registration on failure, and reject NULL pointer in
   debug_dump()

 - Fix sriov_numvfs attribute name in zPCI documentation

 - Fix typos in comments

* tag 's390-7.3-4' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
  s390/cio: Fix NULL pointer dereference in ccw_device_get_util_str()
  s390/debug: Fix NULL pointer dereference in debug_info_copy()
  s390/debug: Do not register views for failed static debug areas
  s390/debug: Reject NULL debug info in debug_dump()
  s390/cmf: Fix virtual vs physical address confusion
  s390/pci: Don't report recovery success on skipped recovery
  s390/pci: Report SCLP status on error events when no pdev is associated
  s390/pci: Fix missing device lock in zpci_report_status()
  s390/pci: Fix leak of struct pci_dev reference in zpci_report_status()
  s390/cio: Guard PMCW field accesses with dnv check
  s390/cio: Check pmcw.dnv before pmcw.ena in I/O entry points
  s390/cio: Fix cio_update_schib() to not cache invalid schib
  s390/pci/docs: Fix sriov_numvfs attribute name
  s390: Fix typos in comments
2026-09-25 09:39:33 -07:00
Vineet Gupta
f050c3e61d ARC: arch_cmpxchg_relaxed to use size of pointed type not pointer
Bradley spotted this snafu when working on two-byte cmpxchg emulation.
He originally folded this fix into his series but the fix itself
uncovered a pre-existing issue, missing arch_atomic64_cmpxchg_relaxed
which just needed to be wired up.

Fixes: e188f3330a ("ARC: cmpxchg/xchg: rewrite as macros to make type safe")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202609240539.AkulvjVe-lkp@intel.com/
Suggested-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
2026-09-24 21:17:09 -07:00
Linus Torvalds
5fc5768c7c bpf-fixes
-----BEGIN PGP SIGNATURE-----
 
 iQJRBAABCgA7FiEE+soXsSLHKoYyzcli6rmadz2vbToFAmq1NvAdHGFsZXhlaS5z
 dGFyb3ZvaXRvdkBnbWFpbC5jb20ACgkQ6rmadz2vbTpmpA//fqEoC6Sq1zxo3ADH
 hV0Z9ewkNTjjH85QnispjcRkAhSHG3JscNXKRXm1NmNkwJsHJ4TDIKcjABYDjquD
 wqfvL9hLXPsvud0M/PR6/CZeBAWXpukkaxeYedY+83ttTHjzR0tDq0Ne9yvIV+Nc
 hS0qFIIHs8C6l2nyuNSxvgrv216orG8qd0Bi3tpDfsLqCsLLEmDyQ1H+ZzpJF2xW
 RV3oMcggCeo305m8+uiofQGf8hmmRrmA7SEfF+Qe08ab2GOn9glINfVTbTE3AdQW
 fkvAk8Zuio3hwwMBHDWWYXrKO0N3ykzcDk4V6JPUWiH1dOANf1tS3G1uJyxb4tsv
 dHVdA0xL3sg7YnuSywfb82vTXvQz5QEFeDYxxLB2fMJe8LcCfgF1lkIr1DbdrMjI
 hlozGCs3p/GVIVNhjGVPezWsUvzK4PIuKVM6U1qWojtAhXU/bJCvP0iBU83RlBL7
 S0GGSC/qvkuPed9hAp2pnrrRo/9GEp1PZq8AXsp3nti0OaRxgxpjjQQFzZEWlOrR
 bErtbKziHzl2xARvCRycNZ+QWT4ZdjmK++8pba7hOuFkRclOV4KRWk4OthBvcADu
 NfNsVXdXpOnesg7TNIUJ8heVAYPjKaBHJBuG2Prghrnc95uqEQwOfbYT15zAbe6I
 l3HqGerSa7WtbrnCwsf1DvyPPII=
 =ZxKg
 -----END PGP SIGNATURE-----

Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf

Pull bpf fixes from Alexei Starovoitov:

 - Fix bpf_skb_change_tail() to drop the checksum offload instead of
   rejecting the trim of CHECKSUM_PARTIAL skbs (Daniel Borkmann)

 - Add KF_PERFMON kfunc flag and require CAP_PERFMON for kfuncs that
   read arbitrary memory and for untrusted read-only memory reads
   (Daniel Borkmann)

 - Clear scalar delta on narrowing stack spill (Daniel Borkmann)

 - Set up the frame pointer for the exception callback in arm64 JIT, and
   zero-fill other CPUs when BPF_F_CPU update creates a per-cpu hash
   element (Donggeun Yoo)

 - Various fixes (Emil Tsalapatis):
     - Fix bounds check underflow for skb-backed dynptrs
     - Fix rx_queue_mapping context access code generation in bpf_sock
     - Reject packet pointer arguments to subprogs that may mutate the
       packet
     - Reject ALU instructions that see arena and non-arena operands on
       different code paths

 - Fix copied_seq double-counting on sockmap self-redirect
   (Geliang Tang)

 - Fix divide-by-zero in btf_struct_walk() on a flexible array of
   zero-sized elements, fix out-of-bounds read of rtt_min in sock_ops
   (Jiayuan Chen)

 - Fix bpf_sock_destroy() out-of-bounds read of sk_protocol on TIME_WAIT
   and request socks, and sleeping under RCU when destroying a listener
   with pending children (Jiayuan Chen)

 - Fix JEQ/JNE with immediate operand in MIPS32 JIT and missing zero
   extension of BSWAP 16/32 in MIPS64 JIT (Johan Almbladh)

 - Avoid soft lockup in htab lookup[_and_delete] batch operations on
   large maps (Jose Fernandez)

 - Various fixes (Kumar Kartikeya Dwivedi):
     - Verify global subprogs in each sleepability context they are
       called from
     - Make post-verification instruction rewrites killable
     - Preserve packet pointer displacement in regsafe()
     - Apply CO-RE relocations before subprogram validation, restrict
       CO-RE poisoning to relocatable instructions, and reject truncated
       ldimm64 CO-RE relocations in libbpf
     - Assign lock identity to callback map values
     - Compare stack frames in regs_exact()
     - Bound ownership depth through local kptrs and graph roots

 - Fix u32 overflow in map batch operations when the map size exceeds
   4GB (Masoud Aghasi)

 - Fix UAF in bpf memalloc due to concurrent consumption of ttrace lists
   in alloc_bulk() (Pu Lehui)

 - Allow gotox as the terminal instruction of a program or a subprogram
   (Siddharth Chintamaneni)

 - Disallow bpf_skb_pull_data() for LWT_SEG6LOCAL, skip unsettled links
   in link iterator, and reject dev-bound-only programs on other devices
   (Weiming Shi)

 - Reject non-negative stack offsets in stack_slot_obj_get_spi()
   (Xu Yunxiang)

 - Check params size before reading reserved fields in
   bpf_crypto_ctx_create() (Yuqi Xu)

 - Reject max_entries > INT_MAX in sock_map_alloc() (Zhao Gongyi)

 - Use a 32-bit compare in xsk_map_gen_lookup() (Zhiling Zou)

 - Use kvfree() in xdp_test_run_teardown() (Zhixing Chen)

* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (58 commits)
  selftests/bpf: Test per-cpu initialization of a BPF_F_CPU created element
  bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element
  bpf: Fix BSWAP 32 and 16 on MIPS64
  bpf: Fix immediate JMP JEQ/JNE on MIPS32
  bpf: Reject dev-bound-only programs on other devices
  bpf, sockmap: Reject max_entries > INT_MAX in sock_map_alloc
  selftests/bpf: Test for mixed arena/nonarena code paths
  bpf: Prevent variable arena/non-arena register contents
  selftests/bpf: Test rejection of pkt args to mutating subprogs
  bpf: Reject pkt arguments in mutating subprogs
  selftests/bpf: Add selftests for rx_queue_mapping context access
  bpf: Fix bpf_sock context code generation
  selftests/bpf: Test dynptr slices past end of skb
  bpf: Fix bounds check for skb-backed dynptrs
  selftests/bpf: Reject iterator destruction through fp+0
  bpf: Reject non-negative offsets in stack_slot_obj_get_spi()
  bpf: Check params size before reading reserved fields
  selftests/bpf: Check local object ownership depth
  bpf: Bound ownership depth through local kptrs and graph roots
  selftests/bpf: Cover frame changes in bounded loops
  ...
2026-09-24 08:25:26 -07:00
Linus Torvalds
ef33dd4e0a arm64 fixes for -rc5
- Fix the recently merged user ABI for the Arm CMN PMU driver filtering
   logic so that the ordering matches the hardware spec
 
 - Fix spurious warning when attempting to read PROT_NONE mappings of
   /dev/mem
 
 - Allow WFxT to be disabled on the command line, which is necessary for
   some configurations of recent Apple SoCs
 
 - Fix EL2 fine-grained trap configuration for the CPU PMU
 
 - Fix MIDR matching when applying CPU errata workarounds in a VM
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEPxTL6PPUbjXGY88ct6xw3ITBYzQFAmq06TIQHHdpbGxAa2Vy
 bmVsLm9yZwAKCRC3rHDchMFjNPGQCACK7VeQk7ER46znaeMNyHaQARWkuF7D3wdU
 wCvt/hppo6DzHpsiWw5rDrBIHyeVpWkilDVROuaM1q8Qf6HaFe2h+sbuK96OVo1Y
 OLP1aZUBj3ZGLUNBIxDANP4A200Avd6gFLLY2jPvZ2YyJfEB9Ok3D0KahaRdtMh9
 PGX4qo3j7dZaTLChMrVhXApAidkZ079w5+Di5wXilU5QNyCWsfA+SKBY7jZWP7jw
 yPQ5PPDeYDYUziCUT4ooAegY9vODgmtbikXPFQQzF0zPeTtqKuFwSekdXnkJQS3E
 R3+bBmV5JrDuitjMqUvrxreTGZqWj6R1mHL4tsActCnhTFEc2mar
 =dNHO
 -----END PGP SIGNATURE-----

Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Pull arm64 fixes from Will Deacon:

 - Fix the recently merged user ABI for the Arm CMN PMU driver filtering
   logic so that the ordering matches the hardware spec

 - Fix spurious warning when attempting to read PROT_NONE mappings of
  /dev/mem

 - Allow WFxT to be disabled on the command line, which is necessary for
   some configurations of recent Apple SoCs

 - Fix EL2 fine-grained trap configuration for the CPU PMU

 - Fix MIDR matching when applying CPU errata workarounds in a VM

* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
  arm64/boot: Disable trapping of PMZR_EL0 writes to EL2
  arm64: errata: match the target implementation CPU's own MIDR
  arm64: Add override for WFxT
  arm64: io: Reject non-user protection in ioremap_prot()
  perf/arm-cmn: Fix multi-filter encoding
2026-09-24 06:08:15 -07:00