Synthesize SHUTDOWN (for L1) if L2 requires unhandleable emulation after
loading guest state from SMRAM during RSM to prevent a misbehaving L1 (or
userspace via L1) from tripping the sanity check that KVM doesn't try to
cancel a pending nested VM-Enter. If SMRAM is modified such that RSM will
load what should be impossible state for L2, then KVM will detect that it
needs to emulate the current code stream and will abort VM-Entry to L2.
And because KVM (rightly) expects such a scenario to be impossible, KVM
WARNs and bugs the VM.
__ret && !(vcpu->kvm)->vm_bugged
WARNING: arch/x86/kvm/vmx/vmx.c:6741 at vmx_handle_exit+0x65/0x790 [kvm_intel], CPU#13: vmx_invalid_nes/2902
Modules linked in: kvm_intel kvm irqbypass [last unloaded: kvm]
CPU: 13 UID: 1000 PID: 2902 Comm: vmx_invalid_nes Tainted: G W 7.2.0-rc2 #124 PREEMPT
Tainted: [W]=WARN
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
RIP: 0010:vmx_handle_exit+0x65/0x790 [kvm_intel]
Call Trace:
<TASK>
kvm_arch_vcpu_ioctl_run+0xdf8/0x1d00 [kvm]
kvm_vcpu_ioctl+0x2d5/0x960 [kvm]
__x64_sys_ioctl+0x8a/0xd0
do_syscall_64+0xb7/0x570
entry_SYSCALL_64_after_hwframe+0x4b/0x53
</TASK>
Alternatively, KVM could suppress the WARN for the RSM case, but that would
still leave the vCPU in a "bad" state that KVM doesn't know how to handle
(which is also why KVM rejects attempts to do KVM_RUN when the vCPU is
loaded with invalid state). And architecturally, the Intel SDM explicitly
states that RSM leads to shutdown if the CPU detects invalid state.
Fixes: 2bb8cafea8 ("KVM: vVMX: signal failure for nested VMEntry if emulation_required")
Reported-by: Hao Zhang <zhanghao1@kylinos.cn>
Link: https://patch.msgid.link/20260731173340.2644656-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Expose VMX's check for unhandleable emulation as its own kvm_x86_ops hook,
and move the actual pre-KVM_RUN check into common x86. This will allow
sharing the core logic with KVM's RSM emulation without needed to add a
post-RSM hook, and is a step towards removing the .vcpu_pre_run() hook
entirely.
Alternatively, KVM could provide a post-RSM hook as mentioned, but pre/post
hooks tend to be unwieldy as the exact "timing" of the call often matters
greatly. E.g. in this case, the call must slot in exactly between loading
guest state from SMRAM and the hack to force the vCPU out of L2 on SHUTDOWN.
Link: https://patch.msgid.link/20260731173340.2644656-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Now that all on-stack maps use CLASS(kvm_vcpu_map_local), i.e. now that all
maps are zero-allocated, explicitly put any existing mappings/references
when establishing a new mapping to harden against KVM bugs leaking memory,
but yell loudly as the owner of the map is still ultimately responsible for
the lifecycle of the mapping.
Suggested-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260724004757.131420-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Convert the kvm_vcpu_map() usage in PPC dcbz patching to the new
CLASS(kvm_vcpu_map_local) implementation, to eliminate the very last of the
the open-coded on-stack "struct kvm_host_map" declarations. This will
allow adding hardening kvm_vcpu_map() against memory leaks (due to
clobbering the existing mapping).
No functional change intended.
Link: https://patch.msgid.link/20260724004757.131420-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Convert the kvm_vcpu_map_readonly() usage in nVMX's MSR bitmap merging to
the new CLASS(kvm_vcpu_map_local_readonly) implementation, to eliminate the
last of the open-coded on-stack "struct kvm_host_map" declarations (in x86,
PPC still has one more to convert).
No functional change intended.
Link: https://patch.msgid.link/20260724004757.131420-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Convert the kvm_vcpu_map() usage in the enter/leave SMM flows to the new
CLASS(kvm_vcpu_map_local) implementations, to eliminate the need to
manually do unmap() in error paths, and more importantly to eliminate more
of the open-coded on-stack "struct kvm_host_map" declarations.
No functional change intended.
Link: https://patch.msgid.link/20260724004757.131420-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add CLASS() definitions for locally mapping a PFN using kvm_vcpu_map()
given a vCPU+gfn pair. In addition to eliminating the need to manually
do unmap(), e.g. in error paths, this will allow hardening KVM against
double-mapping without having to manually ensure every on-stack declaration
is zero-initialized.
Use "map local" as the primary terminology as the basic concept is more or
less the same as kmap_local(): ensure the current context has a kernel
mapping to the underlying memory.
Immediately convert the relatively straightforward nested SVM flows, and
defer converting the more involved SMM flows to a separate change.
No functional change intended.
Cc: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260724004757.131420-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Always check and clear KVM_REQ_GET_NESTED_STATE_PAGES when emulating a
nested VM-Exit to ensure the request is cleared, even when KVM was built
with CONFIG_KVM_HYPERV=n, as KVM subtly relies on the "check" to clear
the flag and thus avoid double-mapping the vmcs12 pages, e.g. if KVM
manages to bail from VM-Enter without processing the request, and then
emulates VMLAUNCH or VMRESUME.
Fixes: b4f69df0f6 ("KVM: x86: Make Hyper-V emulation optional")
Cc: stable@vger.kernel.org
Reported-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260724004757.131420-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When getting a to-be-injected IRQ, don't WARN if the IRQ disappeared and
Xen emulation is supported, as a guest could concurrently toggle its
evtchn_upcall_pending flag in shared memory and deassert the IRQ. Even
more annoyingly, userspace could disable Xen emulation for the entire VM
KVM_XEN_HVM_CONFIG. So, suppress WARNs on lost IRQs if Xen emulation is
supported to prevent false positives.
Alternatively, KVM could track if the VM has ever used Xen emulation, but
the added complexity isn't worth carrying given that the vast majority of
deployments can and should disable Xen emulation.
Fixes: bf672720e8 ("KVM: x86: check the kvm_cpu_get_interrupt result before using it")
Reported-by: Sashiko Bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260625212001.3B6561F000E9@smtp.kernel.org
Link: https://patch.msgid.link/20260724173425.278753-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When getting a to-be-injected IRQ, don't WARN if the IRQ disappeared and
the VM has an in-kernel PIC, as the ExtINT handling that's routed through
KVM's virtual PIC is tracked per-VM, not per-vCPU. If another vCPU grabs
the IRQ, or deasserts the interrupt (which is level-triggered), then it's
both expected and "fine" for a
Keep the assert for split IRQCHIP VMs to help detect KVM bugs, as userspace
is responsible for routing ExtINT to the intended vCPU, i.e. once an ExtINT
is pending, it can't be cleared without holding the vCPU's mutex, and thus
false positives are impossible.
Fixes: bf672720e8 ("KVM: x86: check the kvm_cpu_get_interrupt result before using it")
Debugged-by: Alexander Potapenko <glider@google.com>
Reported-by: syzbot+dd769db18693736eee89@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=dd769db18693736eee89
Closes: https://lore.kernel.org/all/6a360fdf.871e809a.2d6dda.0000.GAE@google.com
Link: https://patch.msgid.link/20260724173425.278753-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
The default case in __apic_accept_irq() has carried a printk("TODO:
unsupported delivery mode") since the original LAPIC emulation was
introduced in commit 97222cc831 ("KVM: Emulate local APIC in
kernel").
The switch now handles all eight delivery modes defined by the x86
architecture and is always constrained to the three-bit field
defined by the architecture: Either by masking with APIC_MODE_MASK,
by three-bit bitfield widths in the IOAPIC and MSI structs, or by
using APIC_DM_* constants directly.
Replace the unreachable printk with WARN_ON_ONCE(1) to match the
existing pattern for impossible defaults elsewhere in the same file.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Link: https://patch.msgid.link/20260715120341.2661873-1-twiederh@redhat.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Extend the set_sregs test to cover various bits in EFER. Update
TEST_INVALID_CR_BIT() to operate on EFER as well as CRx (and rename it
accordingly). Add test cases to check that EFER bits are disallowed
without the relevant CPUID enablement.
Assisted-by: Gemini:unknown-version
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260713180153.2728382-3-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
When handling userspace SREGS writes, check the validity of EFER (i.e.
allowed bits) before writing the new value of EFER through the
per-vendor set_efer callbacks. This prevents userspace from writing
bogus values (e.g. EFER.SVME=1 with nested=0).
Note: on KVM_SET_MSRS, KVM only checks EFER validity in terms of KVM
caps, not guest caps, so it is possible to set EFER bits that are
supported by KVM but not by the guest CPUID. Potentially allowing
userspace to set msrs before CPUID.
However, for KVM_SET_SREGS*, check the validity of the set bits against
both KVM and guest caps. This is consistent with other validity checks
(e.g. for CR4) that check validity against guest caps, which already
imposes the need to set CPUID before SREGS.
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260713180153.2728382-2-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Supported EFER bits naturally fits into kvm_caps because it gets
recomputed during vendor initialization (e.g. to account for EFER.SVME
being allowed/disallowed based on nested being enabled/disabled). Move
efer_supported_bits into kvm_caps as supported_efer_bits (for naming
consistency).
As the bitmask is now globally visible as part of kvm_caps, there's
little use for helpers to enable/disable specific bits, so drop them and
open-code updates to kvm_caps.supported_efer_bits.
No functional change intended.
Suggested-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260713181020.2735367-6-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
In preparation for moving efer_reserved_bits into kvm_caps, reverse its
polarity and make it efer_supported_bits, to be more consistent with
other fields in kvm_caps.
No functional change intended.
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260713181020.2735367-5-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
EFER reserved bits are statically initialized, and do not reset if a
vendor module is re-loaded. For example, loading kvm_amd with nested=1
removes EFER.SVME (and potentially EFER.LMSLE) from the reserved bits.
Reloading kvm_amd with nested=0 does not add them back, allowing
userspace to set EFER.SVME with nested=0.
Re-initializing EFER reserved bits before configuring them on vendor
initialization.
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260713181020.2735367-4-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Remove EFER.LME and EFER.LMA from EFER reserved bits only if long mode
is actually supported. KVM does check long-mode support before allowing
the bits for guest writes and userspace writes through KVM_SET_SREGS*
(in __kvm_valid_efer()), but userspace writes through KVM_SET_MSRS only
check reserved bits.
In practice, this doesn't really matter. The true motiviation is getting
rid of the #ifdeffery when initializing efer_reserved_bits.
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260713181020.2735367-3-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move SVM-specific EFER bit enablement to generic x86 code, with the rest
of EFER bit enablement. Unifying the code for EFER bit enablement allows
for a later change to re-initialize EFER bits on module init.
No functional change intended.
Cc: stable@vger.kernel.org
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260713181020.2735367-2-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Fix an issue where userspace or the guest can program an Hyper-V
synthetic timer to have a deadline in the past via integer overflow,
preventing the CPU from making progress and triggering an RCU stall.
Hyper-V's SynIC exposes 4 per-vCPU synthetic timers to the
guest, which are emulated by KVM. Each is programmed through the
HV_X64_MSR_STIMERi_CONFIG and HV_X64_MSR_STIMERi_COUNT MSRs. Depending
on CONFIG, COUNT represents either the absolute expiration time or the
period of a periodic timer, both expressed in 100ns ticks. These timers
may be set both by the guest (WRMSR) and the host (KVM_SET_MSRS).
When the timer is enabled, stimer_start() translates COUNT to an
absolute monotonic deadline and arms an hrtimer. If COUNT is set to a
value close to U64_MAX, the deadline calculation can overflow.
ktime_add_ns(ktime_now, 100 * (stimer->exp_time - time_now))
This can result in a CPU livelock. stimer_start() arms the timer
via hrtimer_start() with a deadline in the past, which causes it to
immediately fire. The stimer callback then raises KVM_RQ_HV_STIMER, with
the intention of causing KVM to deliver a synthetic interrupt on the
next vCPU guest enter.
Then, once userspace issues KVM_RUN, vcpu_enter_guest() consumes the
request, calling kvm_hv_process_stimers(). This would normally disable
the timer via stimer_expiration() once the deadline is in the past.
However, the deadline comparison is done between the KVM reference
counter and stime->exp_time, which is a big value close to U64_MAX, so
this never happens for a few thousand years.
kvm_hv_process_timers() then re-arms the timer via stimer_start(), since
it was not disabled, which again fires immediately. Before entering
the guest, kvm_vcpu_exit_request() checks kvm_request_pending(),
which returns true due to the newly raised KVM_REQ_HV_STIMER. Then
vcpu_enter_guest() aborts the guest entry, returning early into
vcpu_run(), which loops back again into vcpu_enter_guest(), restarting
the cycle.
Since there are no manual yields in this loop, a task with SCHED_FIFO
may starve RCU grace-period kthreads, which exposes the stalls found
by syzcaller:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: (detected by 1, t=10502 jiffies, g=14269, q=1142 ncpus=2)
rcu: All QSes seen, last rcu_preempt kthread activity 10500 (4294965239-4294954739), jiffies_till_next_fqs=1, root ->qsmask 0x0
rcu: rcu_preempt kthread starved for 10500 jiffies! g14269 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
( ... )
Call Trace:
<IRQ>
__run_hrtimer kernel/time/hrtimer.c:1773 [inline]
__hrtimer_run_queues+0x408/0xc30 kernel/time/hrtimer.c:1841
hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1045 [inline]
__sysvec_apic_timer_interrupt+0x102/0x3e0 arch/x86/kernel/apic/apic.c:1062
instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1056 [inline]
sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1056
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:697
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xa8/0x110 kernel/locking/spinlock.c:194
Code: 74 05 e8 0b f4 5f f6 48 c7 44 24 20 00 00 00 00 9c 8f 44 24 20 f6 44 24 21 02 75 4f f7 c3 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> 23 6b 27 f6 65 8b 05 7c 60 5a 07 85 c0 74 40 48 c7 04 24 0e 36
RSP: 0018:ffffc900040a7320 EFLAGS: 00000206
RAX: 5de15cb931505900 RBX: 0000000000000a06 RCX: 5de15cb931505900
RDX: 0000000000000007 RSI: ffffffff8daa9dc3 RDI: 0000000000000001
RBP: ffffc900040a73b0 R08: ffffffff8fc3d077 R09: 1ffffffff1f87a0e
R10: dffffc0000000000 R11: fffffbfff1f87a0f R12: dffffc0000000000
R13: 0000000000000000 R14: ffff8880b8628240 R15: 1ffff92000814e64
hrtimer_start include/linux/hrtimer.h:259 [inline]
stimer_start arch/x86/kvm/hyperv.c:682 [inline]
kvm_hv_process_stimers+0xd0a/0x16a0 arch/x86/kvm/hyperv.c:893
vcpu_enter_guest arch/x86/kvm/x86.c:11193 [inline]
vcpu_run+0x2240/0x76b0 arch/x86/kvm/x86.c:11639
kvm_arch_vcpu_ioctl_run+0x1148/0x1c90 arch/x86/kvm/x86.c:11984
kvm_vcpu_ioctl+0x99a/0xed0 virt/kvm/kvm_main.c:4492
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0xfa/0xf80 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f635278f749
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f635365c038 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f63529e5fa0 RCX: 00007f635278f749
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000005
RBP: 00007f6352813f91 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f63529e6038 R14: 00007f63529e5fa0 R15: 00007ffd5b219358
</TASK>
Fix this by clamping the deadline computation to KTIME_MAX, which
preserves the intent of arming a timer very far in the future.
ktime_add_safe() already does this type of clamping, so use it after
checking that that multiplying by the 100ns time tick also does not
overflow.
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reported-by: syzbot+3d5461510f8dc4adfe30@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=3d5461510f8dc4adfe30
Fixes: 1f4b34f825 ("kvm/x86: Hyper-V SynIC timers")
Cc: stable@vger.kernel.org
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260714133212.3916611-3-clopez@suse.de
[sean: tag for stable]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Change to using KVM_X86_OP() instead of KVM_X86_OP_OPTIONAL() for the
.pi_update_irte() hook in kvm-x86-ops.h since now both VMX and SVM have
implemented it.
For the Fixes tag:
This hook was introduced for VMX posted-interrupt support. SVM later
added its implementation, but at this point KVM_X86_OP* had not been
introduced yet.
Initially KVM introduced KVM_X86_OP_NULL (and KVM_X86_OP) and used it
for this hook. But this was correct, because the use of KVM_X86_OP_NULL
was "to mark calls that do not follow the [svm|vmx]_func_name naming
convention" and the VMX one was named pi_update_irte(), i.e., did not
follow the convention. See commit 9af5471bdb ("KVM: x86: introduce
definitions to support static calls for kvm_x86_ops").
KVM later removed KVM_X86_OP_NULL (due to "the naming convention is not
in use anymore"), and added KVM_X86_OP_OPTIONAL for the hooks that can
be NULL pointer. It used KVM_X86_OP_OPTIONAL for this hook, but should
use KVM_X86_OP instead. See commit e4fc23bad8 ("KVM: x86: remove
KVM_X86_OP_NULL and mark optional kvm_x86_ops").
Note this hook was named .update_pi_irte() when it was introduced, but
got renamed to .pi_update_irte() at some point between the above two
commits.
Fixes: e4fc23bad8 ("KVM: x86: remove KVM_X86_OP_NULL and mark optional kvm_x86_ops")
Signed-off-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260715090505.601174-1-kai.huang@intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Commit aebc3ca190 ("KVM: x86: Enable CMCI capability by default and
handle injected UCNA errors") introduced kvm_vcpu_x86_set_ucna(), which
accesses @vcpu->arch.mci_ctl2_banks[] using @mce->bank as the index. The
@mce struct is user-controlled, provided via the KVM_X86_SET_MCE ioctl.
The caller of this function, kvm_vcpu_ioctl_x86_set_mce(), bounds-checks
@mce->bank and applies array_index_nospec() to advance the @banks
pointer, but @mce->bank itself is passed through unclamped. On a
speculative path that bypasses the bounds check, the raw @mce->bank
value can index mci_ctl2_banks[] out-of-bounds.
In practice this is a very weak gadget, and would at most allow leaking
a single bit in a 64-bit integer, but prevent potential future issues by
clamping @mce->bank in place with array_index_nospec(), before passing
the struct to kvm_vcpu_x86_set_ucna().
Fixes: aebc3ca190 ("KVM: x86: Enable CMCI capability by default and handle injected UCNA errors")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260609131856.2562222-3-clopez@suse.de
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin big-number arithmetic engine to guests via
CPUID 0xC0000001 EDX bits 27 (RSA) and 28 (RSA_EN). The RSA feature
provides two unprivileged instructions for modular arithmetic on big
integers, documented in the Zhaoxin PadLock Instruction Reference,
chapter 4 ("Modular Multiplication and Exponentiation Engine"). Both
support operand sizes from 256 to 32768 bits (in 128-bit increments):
- REP XMODEXP (encoding F3 0F A6 F8, subsection 4.1) computes
A^B mod M
- REP MONTMUL2 (encoding F3 0F A6 F0, subsection 4.2) computes
A*B mod M
REP MONTMUL2 is the long-mode replacement of legacy REP MONTMUL, which
is restricted to compatibility and 32-bit protected modes. These
primitives accelerate RSA and related public-key operations.
Both instructions are unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The RSA and RSA_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because different
software may probe either one when checking for RSA availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-6-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin PadLock Hash Engine v2 to guests via CPUID
0xC0000001 EDX bits 25 (PHE2) and 26 (PHE2_EN). PHE2 extends the
PadLock hash family with SHA-384 and SHA-512 support per FIPS 180-3,
complementing the existing PHE feature (SHA-1 and SHA-256).
Two unprivileged instructions are exposed, documented in the Zhaoxin
PadLock Instruction Reference, chapter 3 ("Hash Engine"):
- REP XSHA384 (encoding F3 0F A6 D8, subsection 3.3)
- REP XSHA512 (encoding F3 0F A6 E0, subsection 3.4)
Both consume software-padded 128-byte blocks (RCX = block count, RSI =
input, RDI = state) and produce hash output in the state buffer.
Both instructions are unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The PHE2 and PHE2_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because different
software may probe either one when checking for PHE2 availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-5-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin second-generation hardware RNG to guests via
CPUID 0xC0000001 EDX bits 22 (RNG2) and 23 (RNG2_EN).
RNG2 is exposed by the REP XRNG2 instruction (encoding F3 0F A7 F8),
documented in the Zhaoxin PadLock Instruction Reference, subsection 1.3
("REP XRNG2"). It produces random bytes from two on-die RNG sources
selectable via RAX bits[10:9] and an output mode (raw vs post-processed)
controlled by RDX bits[1:0], providing high-quality entropy intended
for cryptographic operations.
REP XRNG2 is unprivileged (no CPL restriction) and available in all
CPU modes, with no associated MSR control. The RNG2 and RNG2_EN bits
are redundant by hardware design (set or cleared together) and both
serve purely as CPUID-level feature-presence reporting flags requiring
no KVM emulation. Both bits are advertised because different software
may probe either one when checking for RNG2 availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-4-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin CCS (Chinese Cryptography Standard) feature to
guests via CPUID 0xC0000001 EDX bits 4 (CCS) and 5 (CCS_EN). CCS groups
two unprivileged instructions for Chinese national cryptographic
primitives, documented in the Zhaoxin GMI Instruction Set Reference,
chapter 2 ("CCS instruction group"):
- SM3 (encoding F3 0F A6 E8, subsection 2.1) implements the SM3 hash
algorithm specified in GM/T 0004-2012. It supports two modes
selected by RAX: auto-padding stream mode (RAX=0) and pre-padded
block mode (RAX=-1).
- SM4 (encoding F3 0F A7 F0, subsection 2.2) implements the SM4 block
cipher specified in GM/T 0002-2012, supporting ECB / CBC / CFB /
OFB / CTR modes via a control word in RAX, and CBC-MAC / CFB-MAC
when RAX bit[11] is set.
Both instructions are unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The CCS and CCS_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because different
software may probe either one when checking for CCS availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-3-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin SM2 instruction support to guests via CPUID
0xC0000001 EDX bits 0 (SM2) and 1 (SM2_EN).
The SM2 instruction (encoding F2 0F A6 C0) implements the SM2
elliptic-curve public-key cryptography algorithm specified in
GM/T 0003-2012; the hardware-level behavior is documented in the
Zhaoxin GMI Instruction Set Reference, chapter 1 ("SM2"). The
instruction multiplexes its sub-functions on the RDX[5:0] control
word: encryption (subsection 1.1), decryption (1.2), signing (1.3),
signature verification (1.4), the three key-exchange sub-operations
of section 1.5 (1.5.1 SM2 key-pair generation, which the spec also
uses for the initiator's ephemeral key; 1.5.2 responder shared-key
derivation; 1.5.3 initiator shared-key derivation), and two
preprocess steps for identity and message hashing (1.6.1 and 1.6.2).
The instruction is unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The SM2 and SM2_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because
different software may probe either one when checking for SM2
availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-2-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Rework KVM's handling of per-vendor nested ops to copy the vendor's ops
into a global structure owned by common x86, i.e. treat nested ops just
like x86 and PMU ops. In addition to providing consistency across all ops
implementations, making a copy of the ops prevents changes to the vendor's
ops after KVM is initialized, i.e. guards against goofs where KVM *thinks*
it is updating nested ops, but which won't take effect now that KVM uses
static calls to invoke vendor hooks.
Ignoring the side effects of tagging {svm,vmx}_nested_ops as __initdata,
no functional change intended.
Link: https://patch.msgid.link/20260630202828.440724-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Use static calls to invoke nested virtualization ops, as many of the calls
are in relatively hot paths when L2 is active, e.g. checking for events,
and because there's no reason not use static calls these days.
Opportunistically use a RET0 static call for get_evmcs_version() instead
of manually checking for a non-NULL vendor hook.
Link: https://patch.msgid.link/20260630202828.440724-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a flag to explicitly track if nested virtualization is enabled, and use
it enumerate that various nested CAPs are unsupported, and to reject
enablement of said CAPs. When the nested ops hooks were moved to their
own structure, KVM's NULL-by-default behavior was deliberately dropped,
with the changelog asserting that all was well. That wasn't quite true;
there is no danger to KVM, but now KVM is over-reporting support for
KVM_CAP_NESTED_STATE and KVM_CAP_HYPERV_ENLIGHTENED_VMCS.
Fixes: 33b2217245 ("KVM: x86: move nested-related kvm_x86_ops to a separate struct")
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Link: https://patch.msgid.link/20260630202828.440724-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move "struct kvm_vcpu_hv" and all of its child structures to hyperv.h,
guarded by CONFIG_KVM_HYPERV=y, as "struct kvm_vcpu_arch" holds a pointer
to the structure, i.e. only needs the structure to be declared, not fully
defined.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move the definition of "struct kvm_apic_map", a.k.a. the optimized local
APIC map, to lapic.h, as it is very nearly an implementation details that's
internal to KVM's local APIC emulation (KVM also uses the map to do quick
lookups when a vCPU is yielding to a different vCPU).
No functional change intended.
Suggested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Relocate KVM's TASK_SWITCH_<reason> enums from kvm_host.h to x86.h, as the
enums are arbitrary values, i.e. not architectural, and are intended to be
used only to translate vendor specific information to a common x86 reason
when invoking kvm_task_switch().
Opportunistically name the overall enum to help document the role of the
values.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add static asserts to sanity check the I/O permission map and TSS size
macros against tss_segment_32. Alternatively, the macros could simply use
offsetof() and sizeof(), but having literal numbers makes it easier to
understand the bigger picture, and provides a good excuse for the sanity
checks.
Opportunistically add the necessary includes to make tss.h self sufficient.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move KVM_GUESTDBG_VALID_MASK into x86.c so that it's not globally visible.
As explained by commit 462474588b ("KVM: x86: Move misc "VALID MASK"
defines from kvm_host.h => x86.c"), which unintentionally missed GUESTDBG,
the set of valid flags/bits is very much a KVM-internal detail, as the
values from the hardcoded #defines are often captured and massaged by KVM's
setup code, i.e. *directly* using the macros outside of KVM x86 would be
actively dangerous.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Relocate a variety of Control/Debug Register macros that unintentionally
got left behind when the related helper function prototypes were moved to
regs.h.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Tag tdp_enabled with __read_mostly as the variable is only ever written
during vendor module load, same as all the other global MMU variables that
are handled by kvm_configure_mmu().
Opportunistically annotate the tdp_mmu_enabled and eager_page_split
declarations with __read_mostly, to match their definitions. The compiler
will warn if there are conflicting annotations, i.e. there's minimal risk
of the declaration annotation becoming stale.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move the macros that define the mostly-obsolete apic_attention bits into
lapic.c, as the gory details of PV EOIs and the pre-APICv TPR acceleration
are 100% internal to KVM's local APIC emulation.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
kvm_init_debug() runs before several steps that can fail
(kvm_vfio_ops_init(), kvm_gmem_init(), kvm_init_virtualization() and
misc_register()), but none of the corresponding error labels remove the
"kvm" debugfs directory. Any failure after kvm_init_debug() therefore
leaks the directory and its stat files for the lifetime of the boot.
kvm_exit() already calls debugfs_remove_recursive(kvm_debugfs_dir); add
the same at the err_vfio label, whose fall-through covers every path
taken after kvm_init_debug().
Fixes: 2b01281273 ("KVM: Register /dev/kvm as the _very_ last thing during initialization")
Signed-off-by: leixiang <leixiang@kylinos.cn>
Link: https://patch.msgid.link/20260706095910.39798-1-leixiang@kylinos.cn
Signed-off-by: Sean Christopherson <seanjc@google.com>
When activating Hyper-V's Synthetic Interrupt Controller (SynIC), mark it
active with WRITE_ONCE() and query it using READ_ONCE() in synic_get(),
the only known cross-task reader, to document that the flag is accessed
without holding the vCPU's mutex.
Note, there are no data dependencies on the SynIC being marked active,
e.g. the vector read by synic_set_irq() is set (usually in response to
guest activity) long after the SynIC is initially activated, and a false
negative on the SynIC being active would be benign (ignoring that such a
race is likely to be problematic for the guest irrespective of what KVM
does).
Link: https://patch.msgid.link/20260630225619.511632-12-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Assert that either vcpu->mutex is held or the VM is otherwise unreachable
when using the normal vCPU => HyperV accessor to help detect improper
cross-task usage of the HyperV structure. When accessing the structure
without holding the vCPU's mutex, e.g. to send interrupts or to queue TLB
flushes, KVM needs to use the more paranoid to_hv_vcpu_safe() to guarantee
that it can't see a half-baked structure.
To avoid false positives, open code accesses to vcpu->arch.hyperv in the
Synthetic Timer callbacks (can be reached if and only if HyperV state is
fully initialized).
Link: https://patch.msgid.link/20260630225619.511632-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
In the "vCPU locked or unreachable" lockdep assertion, treat a vCPU as
unreachable if its index is invalid, i.e. if the vCPU is in the process of
being created. Until the vCPU is inserted into the array of vCPUs, the
only way to get at the vCPU is via kvm_vm_ioctl_create_vcpu(). Note, the
actual index is set _before_ adding the vCPU to the array, i.e. there's no
risk of a false negative on the lockdep assertion.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Extract nVMX's lockdep assertion that a vCPU is locked or otherwise
unreachable into a common helper, as KVM x86 is about to gain another user,
but there is nothing x86-specific about the logic, i.e. the assertion may
be useful for other architectures.
No functional change intended.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Invalidate a vCPU's index immediately after allocating storage for the vCPU
so that KVM doesn't incorrectly treat a vCPU that is the process of being
created as being vCPU0. This will also allow detecting that a vCPU is in
the process of being created and thus otherwise unreachable, which is
useful for avoiding false positives in lockdep assertions on vcpu->mutex.
Unwind the index back to -1 if inserting the vCPU into the array or adding
the vCPU to the fd table fails, so that kvm_arch_vcpu_destroy() sees the
vCPU as unreachable, i.e. so that teardown logic doesn't hit false positive
lockdep assertions. Opportunistically add a comment to call out that the
"real" index needs to be set before making the vCPU visible to other tasks.
Note, kvm_wait_for_vcpu_online() naturally does the right thing thanks to
vcpu->vcpu_idx and kvm->online_vcpus being signed values.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Explicitly invalidate KVM's internal Xen vCPU ID during vCPU creation
instead of *trying* to set the Xen ID to the vCPU index by default, and
forward singleshot timer hypercalls to userspace if the VMM hasn't set the
Xen ID via KVM_XEN_VCPU_ATTR_TYPE_VCPU_ID. Using the vCPU's index as its
default Xen ID is reasonable in concept, but in practice is horribly flawed
as the index is left as '0' until after vCPU initialization completes, i.e.
every vCPU gets a Xen ID of '0' by default.
Forward hypercalls to userspace instead of trying to salvage any kind of
default behavior, as all userspace implementations that support multiple
vCPUs either don't enable the timer, are guaranteed to set Xen ID, or work
only because *all* guests also screw up the singleshot timer hypercalls.
The last scenarios is extremely unlikely given that Linux-as-a-guest uses
the actual Xen vCPU ID when making timer hypercalls. In other words, for
all intents and purposes, KVM's ABI is already that userspace must set the
Xen vCPU ID, so just commit to that ABI.
Note, KVM's handling of KVM_XEN_VCPU_ATTR_TYPE_VCPU_ID restricts the ID to
KVM_MAX_VCPUS, so there's no chance of a valid ID colliding with U32_MAX.
Add a compile-time assertion to ensure this holds true in the future (KVM
doesn't care what value is used for "invalid", only that there can't be a
collision).
Link: https://lore.kernel.org/all/20260612233017.1F9771F000E9@smtp.kernel.org
Suggested-by: David Woodhouse <dwmw2@infradead.org>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260630225619.511632-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Hoist the checks on the Xen vCPU ID when handling set_singleshot_timer and
stop_singleshot_timer hypercalls out of their individual if-statements,
so that both checks on the ID are in common code. kvm_xen_hcall_vcpu_op()
is already doubly committed to handling only singleshot timer hypercalls,
and even if that were to change in the future, the function could simply
be renamed and turned into a helper specifically for timer hypercalls.
Opportunistically add a comment to explain why the check exists; the code
looks rather nonsensical without the knowledge that @vcpu_id is a common
param for all per-vCPU hypercalls.
No functional change intended.
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When handling Xen vCPU hypercalls, explicitly route non-singleshot-timer
commands to userspace, *before* checking if in-kernel emulation of the Xen
timer is enabled. Punting hypercalls that are never accelerated by KVM
because some other hypercall happens to be disabled is confusing and
actively dangerous, e.g. it's easy to miss that the only reason KVM can
bail early is because the timer-disabled case provides the same semantics
as the implicit "default" path in the switch-statement.
Opportunistically convert the switch-statement to an if-else-statement to
avoid having to carry code for an impossible "default" case.
For all intents and purposes, no functional change intended.
Link: https://patch.msgid.link/20260630225619.511632-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When initializing a vCPU's Hyper-V object, ensure the object is fully
initialized prior to exposing it through the vCPU, and ensure accesses from
other tasks (e.g. other vCPUs) see the fully initialized object if
vcpu->arch.hyperv is non-NULL.
Lack of ordering manifests as a lockdep splat due to attempting to lock a
TLB flush FIFO before the spinlock is initialized.
INFO: trying to register non-static key.
The code is fine but needs lockdep annotation, or maybe
you didn't initialize this object before use?
turning off the locking correctness validator.
CPU: 1 PID: 5005 Comm: syz-executor189 Not tainted 6.6.120-smp-DEV #1
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/18/2026
Call Trace:
<TASK>
[<ffffffff810dd10c>] dump_stack_lvl+0xcc/0x130 lib/dump_stack.c:106
[<ffffffff8192bddd>] assign_lock_key+0x1fd/0x230 kernel/locking/lockdep.c:977
[<ffffffff8191cb97>] register_lock_class+0x187/0x7a0 kernel/locking/lockdep.c:1291
[<ffffffff8191e7a9>] __lock_acquire+0x179/0x7650 kernel/locking/lockdep.c:5016
[<ffffffff8191e28f>] lock_acquire+0x13f/0x3d0 kernel/locking/lockdep.c:5756
[<ffffffff8101a65b>] __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
[<ffffffff8101a65b>] _raw_spin_lock+0x2b/0x40 kernel/locking/spinlock.c:154
[<ffffffff81319d44>] spin_lock include/linux/spinlock.h:351 [inline]
[<ffffffff81319d44>] hv_tlb_flush_enqueue+0xb4/0x270 arch/x86/kvm/hyperv.c:1946
[<ffffffff813160c6>] kvm_hv_flush_tlb+0xa96/0x1dc0 arch/x86/kvm/hyperv.c:2145
[<ffffffff8131438b>] kvm_hv_hypercall+0x103b/0x1fe0 arch/x86/kvm/hyperv.c:-1
[<ffffffff8133bff3>] __vmx_handle_exit arch/x86/kvm/vmx/vmx.c:6624 [inline]
[<ffffffff8133bff3>] vmx_handle_exit+0x12e3/0x21f0 arch/x86/kvm/vmx/vmx.c:6641
[<ffffffff81215d11>] vcpu_enter_guest arch/x86/kvm/x86.c:11649 [inline]
[<ffffffff81215d11>] vcpu_run+0x4d01/0x79c0 arch/x86/kvm/x86.c:11832
[<ffffffff8120fe39>] kvm_arch_vcpu_ioctl_run+0xb49/0x1c80 arch/x86/kvm/x86.c:12179
[<ffffffff8119cd60>] kvm_vcpu_ioctl+0xc80/0xff0 virt/kvm/kvm_main.c:6029
[<ffffffff8226fefd>] vfs_ioctl fs/ioctl.c:52 [inline]
[<ffffffff8226fefd>] __do_sys_ioctl fs/ioctl.c:872 [inline]
[<ffffffff8226fefd>] __se_sys_ioctl+0xfd/0x170 fs/ioctl.c:858
[<ffffffff85ac97d9>] do_syscall_x64 arch/x86/entry/common.c:52 [inline]
[<ffffffff85ac97d9>] do_syscall_64+0x69/0xb0 arch/x86/entry/common.c:93
[<ffffffff85c000d0>] entry_SYSCALL_64_after_hwframe+0x68/0xd2
</TASK>
Use the "safe" variant in all paths that are known to access the Hyper-V
object, as detected by an upcoming lockdep assertion, with an assist or two
from Sashiko.
Link: https://lore.kernel.org/all/20260612232258.0D9131F000E9@smtp.kernel.org
Fixes: 0823570f01 ("KVM: x86: hyper-v: Introduce TLB flush fifo")
Fixes: fc08b628d7 ("KVM: x86: hyper-v: Allocate Hyper-V context lazily")
Reported-by: syzbot+5b32c49cd8f005e65654@syzkaller.appspotmail.com
Reported-by: syzbot+5d2b94b77112148d1744@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a396a66.52ae72c2.136ac7.0002.GAE@google.com
Tested-by: syzbot+5d2b94b77112148d1744@syzkaller.appspotmail.com
Link: https://patch.msgid.link/20260630225619.511632-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Check for a NULL Hyper-V object in kvm_hv_get_tlb_flush_fifo() instead of
relying on the caller to do so. This will allow fixing a cross-vCPU race
where KVM can access a vCPU's FIFO before it's fully initialized, without
having to jump through too many cognitive hoops to reason about the
correctness of the logic.
Ignoring changes in ordering that only affect the aforementioned race, no
functional change intended.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When handling Hyper-V PV TLB flushes, retrieve the to-be-used FIFO in
hv_tlb_flush_enqueue() instead of having the caller pass in the FIFO. This
will make it easier to fix a cross-vCPU race where KVM can access a vCPU's
FIFO before it's fully initialized.
No functional change intended.
Link: https://patch.msgid.link/20260630225619.511632-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
