When adding an MSR to the auto-load lists, update the MSR index in the
list entry if and only if a new entry is being inserted, as 'i' can only
be non-negative if vmx_find_loadstore_msr_slot() found an entry with the
MSR's index. Unnecessarily setting the index is benign, but it makes it
harder to see that updating the value is necessary even when an existing
entry for the MSR was found.
No functional change intended.
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-41-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
WARN and bug the VM if either MSR auto-load list is full when adding an
MSR to the lists, as the set of MSRs that KVM loads via the lists is
finite and entirely KVM controlled, i.e. overflowing the lists shouldn't
be possible in a fully released version of KVM. Terminate the VM as the
core KVM infrastructure has no insight as to _why_ an MSR is being added
to the list, and failure to load an MSR on VM-Enter and/or VM-Exit could
be fatal to the host. E.g. running the host with a guest-controlled PEBS
MSR could generate unexpected writes to the DS buffer and crash the host.
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-40-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Drop the "on VM-Enter only" parameter from add_atomic_switch_msr() as it
is no longer used, and for all intents and purposes was never used. The
functionality was added, under embargo, by commit 989e3992d2
("x86/KVM/VMX: Extend add_atomic_switch_msr() to allow VMENTER only MSRs"),
and then ripped out by commit 2f055947ae ("x86/kvm: Drop L1TF MSR list
approach") just a few commits later.
2f055947ae x86/kvm: Drop L1TF MSR list approach
72c6d2db64 x86/litf: Introduce vmx status variable
215af5499d cpu/hotplug: Online siblings when SMT control is turned on
390d975e0c x86/KVM/VMX: Use MSR save list for IA32_FLUSH_CMD if required
989e3992d2 x86/KVM/VMX: Extend add_atomic_switch_msr() to allow VMENTER only MSRs
Furthermore, it's extremely unlikely KVM will ever _need_ to load an MSR
value via the auto-load lists only on VM-Enter. MSRs writes via the lists
aren't optimized in any way, and so the only reason to use the lists
instead of a WRMSR are for cases where the MSR _must_ be load atomically
with respect to VM-Enter (and/or VM-Exit). While one could argue that
command MSRs, e.g. IA32_FLUSH_CMD, "need" to be done exact at VM-Enter, in
practice doing such flushes within a few instructons of VM-Enter is more
than sufficient.
Note, the shortlog and changelog for commit 390d975e0c ("x86/KVM/VMX: Use
MSR save list for IA32_FLUSH_CMD if required") are misleading and wrong.
That commit added MSR_IA32_FLUSH_CMD to the VM-Enter _load_ list, not the
VM-Enter save list (which doesn't exist, only VM-Exit has a store/save
list).
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-39-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a helper to remove an MSR from an auto-{load,store} list to dedup the
msr_autoload code, and in anticipation of adding similar functionality for
msr_autostore.
No functional change intended.
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-38-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Rework nVMX's use of the MSR auto-store list to snapshot TSC to sneak
MSR_IA32_TSC into the list _without_ updating KVM's software tracking,
and drop the generic functionality so that future usage of the store list
for nested specific logic needs to consider the implications of modifying
the list. Updating the list only for vmcs02 and only on nested VM-Enter
is a disaster waiting to happen, as it means vmcs01 is stale relative to
the software tracking, and KVM could unintentionally leave an MSR in the
store list in perpetuity while running L1, e.g. if KVM addressed the first
issue and updated vmcs01 on nested VM-Exit without removing TSC from the
list.
Furthermore, mixing KVM's desire to save an MSR with L1's desire to save
an MSR result KVM clobbering/ignoring the needs of vmcs01 or vmcs02.
E.g. if KVM added MSR_IA32_TSC to the store list for its own purposes, and
then _removed_ MSR_IA32_TSC from the list after emulating nested VM-Enter,
then KVM would remove MSR_IA32_TSC from the list even though saving TSC on
VM-Exit from L2 is still desirable (to provide L1 with an accurate TSC).
Similarly, removing an MSR from the list based on vmcs12's settings could
drop an MSR that KVM wants to save for its own purposes.
In practice, the issues are currently benign, because KVM doesn't use the
store list for vmcs01. But that will change with upcoming mediated PMU
support.
Alternatively, a "full" solution would be to track MSR list entries for
vmcs12 separately from KVM's standard lists, but MSR_IA32_TSC is likely
the only MSR that KVM would ever want to save on _every_ VM-Exit purely
based on vmcs12. I.e. the added complexity isn't remotely justified at
this time.
Opportunistically escalate from a pr_warn_ratelimited() to a full WARN as
KVM reserves eight entries in each MSR list, and as above KVM uses at most
one entry.
Opportunistically make vmx_find_loadstore_msr_slot() local to vmx.c as
using it directly from nested code is unsafe due to the potential for
mixing vmcs01 and vmcs02 state (see above).
Cc: Jim Mattson <jmattson@google.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-37-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Drop the intermediate "guest" field from vcpu_vmx.msr_autostore as the
value saved on VM-Exit isn't guaranteed to be the guest's value, it's
purely whatever is in hardware at the time of VM-Exit. E.g. KVM's only
use of the store list at the momemnt is to snapshot TSC at VM-Exit, and
the value saved is always the raw TSC even if TSC-offseting and/or
TSC-scaling is enabled for the guest.
And unlike msr_autoload, there is no need differentiate between "on-entry"
and "on-exit".
No functional change intended.
Cc: Jim Mattson <jmattson@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-36-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When loading a mediated PMU state, elide the WRMSRs to load PMCs with the
guest's value if the value in hardware already matches the guest's value.
For the relatively common case where neither the guest nor the host is
actively using the PMU, i.e. when all/many counters are '0', eliding the
WRMSRs reduces the latency of handling VM-Exit by a measurable amount
(WRMSR is significantly more expensive than RDPMC).
As measured by KVM-Unit-Tests' CPUID VM-Exit testcase, this provides a
a ~25% reduction in latency (4k => 3k cycles) on Intel Emerald Rapids,
and a ~13% reduction (6.2k => 5.3k cycles) on AMD Turin.
Cc: Manali Shukla <manali.shukla@amd.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-35-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Expose enable_mediated_pmu parameter to user space, i.e. allow userspace
to enable/disable mediated vPMU support.
Document the mediated versus perf-based behavior as part of the
kernel-parameters.txt entry, and opportunistically add an entry for the
core enable_pmu param as well.
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-34-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add MSRs that might be passed through to L1 when running with a mediated
PMU to the nested SVM's set of to-be-merged MSR indices, i.e. disable
interception of PMU MSRs when running L2 if both KVM (L0) and L1 disable
interception. There is no need for KVM to interpose on such MSR accesses,
e.g. if L1 exposes a mediated PMU (or equivalent) to L2.
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-33-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Merge KVM's PMU MSR interception bitmaps with those of L1, i.e. merge the
bitmaps of vmcs01 and vmcs12, e.g. so that KVM doesn't interpose on MSR
accesses unnecessarily if L1 exposes a mediated PMU (or equivalent) to L2.
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Co-developed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
[sean: rewrite changelog and comment, omit MSRs that are always intercepted]
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-32-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Mediated vPMU needs to accumulate the emulated instructions into counter
and load the counter into HW at vm-entry.
Moreover, if the accumulation leads to counter overflow, KVM needs to
update GLOBAL_STATUS and inject PMI into guest as well.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-30-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Don't handle exits in the fastpath if emulation is required, i.e. if an
instruction needs to be skipped, the mediated PMU is enabled, and one or
more PMCs is counting instructions. With the mediated PMU, KVM's cache of
PMU state is inconsistent with respect to hardware until KVM exits the
inner run loop (when the mediated PMU is "put").
Reviewed-by: Sandipan Das <sandipan.das@amd.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-29-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Implement the PMU "world switch" between host perf and guest mediated PMU.
When loading guest state, call into perf to switch from host to guest, and
then load guest state into hardware, and then reverse those actions when
putting guest state.
On the KVM side, when loading guest state, zero PERF_GLOBAL_CTRL to ensure
all counters are disabled, then load selectors and counters, and finally
call into vendor code to load control/status information. While VMX and
SVM use different mechanisms to avoid counting host activity while guest
controls are loaded, both implementations require PERF_GLOBAL_CTRL to be
zeroed when the event selectors are in flux.
When putting guest state, reverse the order, and save and zero controls
and status prior to saving+zeroing selectors and counters. Defer clearing
PERF_GLOBAL_CTRL to vendor code, as only SVM needs to manually clear the
MSR; VMX configures PERF_GLOBAL_CTRL to be atomically cleared by the CPU
on VM-Exit.
Handle the difference in MSR layouts between Intel and AMD by communicating
the bases and stride via kvm_pmu_ops. Because KVM requires Intel v4 (and
full-width writes) and AMD v2, the MSRs to load/save are constant for a
given vendor, i.e. do not vary based on the guest PMU, and do not vary
based on host PMU (because KVM will simply disable mediated PMU support if
the necessary MSRs are unsupported).
Except for retrieving the guest's PERF_GLOBAL_CTRL, which needs to be read
before invoking any fastpath handler (spoiler alert), perform the context
switch around KVM's inner run loop. State only needs to be synchronized
from hardware before KVM can access the software "caches".
Note, VMX already grabs the guest's PERF_GLOBAL_CTRL immediately after
VM-Exit, as hardware saves value into the VMCS.
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Co-developed-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-28-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
On AMD platforms, there is no way to restore PerfCntrGlobalCtl at
VM-Entry or clear it at VM-Exit. Since the register states will be
restored before entering and saved after exiting guest context, the
counters can keep ticking and even overflow leading to chaos while
still in host context.
To avoid this, intecept event selectors, which is already done by mediated
PMU. In addition, always set the GuestOnly bit and clear the HostOnly bit
for PMU selectors on AMD. Doing so allows the counters run only in guest
context even if their enable bits are still set after VM exit and before
host/guest PMU context switch.
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
[sean: massage shortlog]
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-27-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Refresh the event selectors that are programmed into hardware when a PMC
is "reprogrammed" for a mediated PMU, i.e. if userspace changes the PMU
event filters
Note, KVM doesn't utilize the reprogramming infrastructure to handle
counter overflow for mediated PMUs, as there's no need to reprogram a
non-existent perf event.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
[sean: add a helper to document behavior, split patch and rewrite changelog]
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-26-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Introduce eventsel_hw and fixed_ctr_ctrl_hw to store the actual HW value in
PMU event selector MSRs. In mediated PMU checks events before allowing the
event values written to the PMU MSRs. However, to match the HW behavior,
when PMU event checks fails, KVM should allow guest to read the value back.
This essentially requires an extra variable to separate the guest requested
value from actual PMU MSR value. Note this only applies to event selectors.
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Co-developed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-25-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When emulating a PMC counter read or write for a mediated PMU, bypass the
perf checks and emulated_counter logic as the counters aren't proxied
through perf, i.e. pmc->counter always holds the guest's up-to-date value,
and thus there's no need to defer emulated overflow checks.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
[sean: split from event filtering change, write shortlog+changelog]
Reviewed-by: Sandipan Das <sandipan.das@amd.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-24-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
For vCPUs with a mediated vPMU, disable interception of counter MSRs for
PMCs that are exposed to the guest, and for GLOBAL_CTRL and related MSRs
if they are fully supported according to the vCPU model, i.e. if the MSRs
and all bits supported by hardware exist from the guest's point of view.
Do NOT passthrough event selector or fixed counter control MSRs, so that
KVM can enforce userspace-defined event filters, e.g. to prevent use of
AnyThread events (which is unfortunately a setting in the fixed counter
control MSR).
Defer support for nested passthrough of mediated PMU MSRs to the future,
as the logic for nested MSR interception is unfortunately vendor specific.
Suggested-by: Sean Christopherson <seanjc@google.com>
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Co-developed-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
[sean: squash patches, massage changelog, refresh VMX MSRs on filter change]
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-23-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When running a guest with a mediated PMU, context switch PERF_GLOBAL_CTRL
via the dedicated VMCS fields for both host and guest. For the host,
always zero GLOBAL_CTRL on exit as the guest's state will still be loaded
in hardware (KVM will context switch the bulk of PMU state outside of the
inner run loop). For the guest, use the dedicated fields to atomically
load and save PERF_GLOBAL_CTRL on all entry/exits.
For now, require VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL support (introduced by
Sapphire Rapids). KVM can support such CPUs by saving PERF_GLOBAL_CTRL
via the MSR save list, a.k.a. the MSR auto-store list, but defer that
support as it adds a small amount of complexity and is somewhat unique.
To minimize VM-Entry latency, propagate IA32_PERF_GLOBAL_CTRL to the VMCS
on-demand. But to minimize complexity, read IA32_PERF_GLOBAL_CTRL out of
the VMCS on all non-failing VM-Exits. I.e. partially cache the MSR.
KVM could track GLOBAL_CTRL as an EXREG and defer all reads, but writes
are rare, i.e. the dirty tracking for an EXREG is unnecessary, and it's
not obvious that shaving ~15-20 cycles per exit is meaningful given the
total overhead associated with mediated PMU context switches.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-22-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Disable RDPMC interception for vCPUs with a mediated vPMU that is
compatible with the host PMU, i.e. that doesn't require KVM emulation of
RDPMC to honor the guest's vCPU model. With a mediated vPMU, all guest
state accessible via RDPMC is loaded into hardware while the guest is
running.
Adust RDPMC interception only for non-TDX guests, as the TDX module is
responsible for managing RDPMC intercepts based on the TD configuration.
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Co-developed-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-21-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Register a dedicated PMI handler with perf's callback when mediated PMU
support is enabled. Perf routes PMIs that arrive while guest context is
loaded to the provided callback, by modifying the CPU's LVTPC to point at
a dedicated mediated PMI IRQ vector.
WARN upon receipt of a mediated PMI if there is no active vCPU, or if the
vCPU doesn't have a mediated PMU. Even if a PMI manages to skid past
VM-Exit, it should never be delayed all the way beyond unloading the vCPU.
And while running vCPUs without a mediated PMU, the LVTPC should never be
wired up to the mediated PMI IRQ vector, i.e. should always be routed
through perf's NMI handler.
Signed-off-by: Xiong Zhang <xiong.y.zhang@linux.intel.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-20-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Require host PMU version 2+ for AMD mediated PMU support, as
PERF_GLOBAL_CTRL and friends are hard requirements for the mediated PMU.
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
[sean: extract to separate patch, write changelog]
Reviewed-by: Sandipan Das <sandipan.das@amd.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-19-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Implement Intel PMU requirements and constraints for mediated PMU support.
Require host PMU version 4+ so that PERF_GLOBAL_STATUS_SET can be used to
precisely load the guest's status value into hardware, and require full-
width writes so that KVM can precisely load guest counter values.
Disable PEBS and LBRs if mediated PMU support is enabled, as they won't be
supported in the initial implementation.
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
[sean: split to separate patch, add full-width writes dependency]
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-18-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Introduce enable_mediated_pmu as a global variable, with the intent of
exposing it to userspace a vendor module parameter, to control and reflect
mediated vPMU support. Wire up the perf plumbing to create+release a
mediated PMU, but defer exposing the parameter to userspace until KVM
support for a mediated PMUs is fully landed.
To (a) minimize compatibility issues, (b) to give userspace a chance to
opt out of the restrictive side-effects of perf_create_mediated_pmu(),
and (c) to avoid adding new dependencies between enabling an in-kernel
irqchip and a mediated vPMU, defer "creating" a mediated PMU in perf
until the first vCPU is created.
Regarding userspace compatibility, an alternative solution would be to
make the mediated PMU fully opt-in, e.g. to avoid unexpected failure due
to perf_create_mediated_pmu() failing. Ironically, that approach creates
an even bigger compatibility issue, as turning on enable_mediated_pmu
would silently break VMMs that don't utilize KVM_CAP_PMU_CAPABILITY (well,
silently until the guest tried to access PMU assets).
Regarding an in-kernel irqchip, create a mediated PMU if and only if the
VM has an in-kernel local APIC, as the mediated PMU will take a hard
dependency on forwarding PMIs to the guest without bouncing through host
userspace. Silently "drop" the PMU instead of rejecting KVM_CREATE_VCPU,
as KVM's existing vPMU support doesn't function correctly if the local
APIC is emulated by userspace, e.g. PMIs will never be delivered. I.e.
it's far, far more likely that rejecting KVM_CREATE_VCPU would cause
problems, e.g. for tests or userspace daemons that just want to probe
basic KVM functionality.
Note! Deliberately make mediated PMU creation "sticky", i.e. don't unwind
it on failure to create a vCPU. Practically speaking, there's no harm to
having a VM with a mediated PMU and no vCPUs. To avoid an "impossible" VM
setup, reject KVM_CAP_PMU_CAPABILITY if a mediated PMU has been created,
i.e. don't let userspace disable PMU support after failed vCPU creation
(with PMU support enabled).
Defer vendor specific requirements and constraints to the future.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Mingwei Zhang <mizhang@google.com>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-17-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a parameter-less API for registering perf callbacks in anticipation of
introducing another x86-only parameter for handling mediated PMU PMIs.
No functional change intended.
Acked-by: Anup Patel <anup@brainfault.org>
Tested-by: Xudong Hao <xudong.hao@intel.com>
Tested-by: Manali Shukla <manali.shukla@amd.com>
Link: https://patch.msgid.link/20251206001720.468579-15-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Running KASAN KUnit tests with {HW,SW}_TAGS mode triggers a fault in
change_memory_common():
Call trace:
change_memory_common+0x168/0x210 (P)
set_memory_ro+0x20/0x48
vmalloc_helpers_tags+0xe8/0x338
kunit_try_run_case+0x74/0x188
kunit_generic_run_threadfn_adapter+0x30/0x70
kthread+0x11c/0x200
ret_from_fork+0x10/0x20
---[ end trace 0000000000000000 ]---
# vmalloc_helpers_tags: try faulted
not ok 67 vmalloc_helpers_tags
Commit a06494adb7 ("arm64: mm: use untagged address to calculate page index")
fixed a KASAN warning in the BPF subsystem by adding kasan_reset_tag() to
the index calculation. In the execmem flow:
bpf_prog_pack_alloc()
-> bpf_jit_alloc_exec()
-> execmem_alloc()
The returned address from execmem_vmalloc/execmem_cache_alloc is passed
through kasan_reset_tag(), so start has no tag while area->addr still
retains the original tag. The fix correctly handled this case by resetting
the tag on area->addr:
(start - (unsigned long)kasan_reset_tag(area->addr)) >> PAGE_SHIFT
However, in normal vmalloc paths, both start and area->addr have matching
tags(or no tags). Resetting only area->addr causes a mismatch when
subtracting a tagged address from an untagged one, resulting in an
incorrect index.
Fix this by resetting tags on both addresses in the index calculation.
This ensures correct results regardless of the tag state of either address.
Tested with KASAN KUnit tests under CONFIG_KASAN_GENERIC,
CONFIG_KASAN_SW_TAGS, and CONFIG_KASAN_HW_TAGS - all pass. Also verified
the original BPF KASAN warning from [1] is still fixed.
[1] https://lore.kernel.org/all/20251118164115.GA3977565@ax162/
Fixes: a06494adb7 ("arm64: mm: use untagged address to calculate page index")
Signed-off-by: Jiayuan Chen <jiayuan.chen@shopee.com>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The scripts in scripts/crypto/ are used to generate files in
lib/crypto/, so they should be included in "CRYPTO LIBRARY".
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260107033948.29368-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
On my development machine the generic, memcpy()-only implementation of
polyval_preparekey() is too fast for the IRQ workers to actually fire.
The test fails.
Increase the iterations to make the test more robust.
The test will run for a maximum of one second in any case.
[EB: This failure was already fixed by commit c31f4aa8fe ("kunit:
Enforce task execution in {soft,hard}irq contexts"). I'm still applying
this patch too, since the iteration count in this test made its running
time much shorter than the other similar ones.]
Fixes: b3aed551b3 ("lib/crypto: tests: Add KUnit tests for POLYVAL")
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://lore.kernel.org/r/20260102-kunit-polyval-fix-v1-1-5313b5a65f35@linutronix.de
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Dan has reported two uses of uninitialized variables in __ris_msmon_read().
If an unknown monitor type is encountered then the local variable, now, is
used uninitialized. Fix this by returning early on error. If a non-mbwu
monitor is being read then the local variable, overflow, is not initialized
but still read. Initialize it to false as overflow is not relevant for csu
monitors.
Fixes: 823e7c3712 ("arm_mpam: Add mpam_msmon_read() to read monitor value")
Fixes: 9e5afb7c32 ("arm_mpam: Use long MBWU counters if supported")
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
Closes: https://lore.kernel.org/r/202512091519.RBwiJcSq-lkp@intel.com/
Closes: https://lore.kernel.org/r/202512100547.N7QPYgfb-lkp@intel.com/
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The VOP on rk3506:
- Support 2 lane MIPI DSI interface, 1.5Gbps/lane.
- Support RGB interface.
- Max output resolution is 1280x1280@60fps.
- WIN1 layer support RGB888/ARGB8888/RGB565.
- Support Gamma LUT.
Signed-off-by: Chaoyi Chen <chaoyi.chen@rock-chips.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251106020632.92-10-kernel@airkyi.com
The dsi controller found on RK3506 supports up to 2 lanes.
Signed-off-by: Hongming Zou <hongming.zou@rock-chips.com>
Signed-off-by: Chaoyi Chen <chaoyi.chen@rock-chips.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251106020632.92-8-kernel@airkyi.com
The rk3506 VOP has adopted a new implementation.
Add a new compatible string for it.
Signed-off-by: Chaoyi Chen <chaoyi.chen@rock-chips.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251106020632.92-5-kernel@airkyi.com
Make it a little less convoluted, and just directly check if the
combination of plane + format + modifier is supported.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-8-83463c075a8d@collabora.com
We don't need to do a long open-coded walk here; we can simply check the
modifier value.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-7-83463c075a8d@collabora.com
Make sure we can't break the hardware by requesting an unsupported
configuration.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-6-83463c075a8d@collabora.com
Planes can only source AFBC framebuffers at multiples of 4px wide on
RK3566/RK3568. Instead of clipping on all SoCs when the user asks for an
unaligned source rectangle, reject the configuration in the plane's
atomic check on RK3566/RK3568 only.
Signed-off-by: Daniel Stone <daniels@collabora.com>
[Make RK3566/RK3568 specific, reword message, s/byte/pixel/]
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-5-83463c075a8d@collabora.com
It seems only cluster windows are capable of applying downscaling when
the source region has an odd width. Instead of applying a workaround
inside atomic_update, fail the plane check if this is requested.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-4-83463c075a8d@collabora.com
If we want to find out if a window is Esmart or not, test for not being
a cluster window, rather than AFBDC presence.
No functional effect as only cluster windows support AFBC decode.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-3-83463c075a8d@collabora.com
We already clip the plane to the display bounds in atomic_check, and
ensure that it is sufficiently sized. Instead of trying to catch this
and adjust for it in atomic_update, just assert that atomic_check has
done its job.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-2-83463c075a8d@collabora.com
We should never be able to create a framebuffer with an unsupported
format, so throw a warning if this ever happens.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Link: https://patch.msgid.link/20251215-vop2-atomic-fixups-v5-1-83463c075a8d@collabora.com
When handling intercepted #PFs, reflect EPCM (Enclave Page Cache Map)
violations, i.e. #PFs with the SGX flag set, back into the guest. KVM
doesn't shadow EPCM entries (the EPCM deals only with virtual/linear
addresses), and so EPCM violation cannot be due to KVM interference,
and more importantly can't be resolved by KVM.
On pre-SGX2 hardware, EPCM violations are delivered as #GP(0) faults, but
on SGX2+ hardware, they are delivered as #PF(SGX). Failure to account for
the SGX2 behavior could put a vCPU into an infinite loop due to KVM not
realizing the #PF is the guest's responsibility.
Take care to deliver the EPCM violation as a #GP(0) if the _guest_ CPU
model is only SGX1.
Fixes: 72add915fb ("KVM: VMX: Enable SGX virtualization for SGX1, SGX2 and LC")
Cc: Kai Huang <kai.huang@intel.com>
Reviewed-by: Richard Lyu <richard.lyu@suse.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20251121222018.348987-1-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a few extra TPR / CR8 tests to x86's xapic_state_test to see if:
* TPR is 0 on reset,
* TPR, PPR and CR8 are equal inside the guest,
* TPR and CR8 read equal by the host after a VMExit
* TPR borderline values set by the host correctly mask interrupts in the
guest.
These hopefully will catch the most obvious cases of improper TPR sync or
interrupt masking.
Do these tests both in x2APIC and xAPIC modes.
The x2APIC mode uses SELF_IPI register to trigger interrupts to give it a
bit of exercise too.
Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
Acked-by: Naveen N Rao (AMD) <naveen@kernel.org>
[sean: put code in separate test]
Link: https://patch.msgid.link/20251205224937.428122-1-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
- Remove useless assignment of soft_mode variable
The function __ftrace_event_enable_disable() sets "soft_mode" in one of
the branch paths but doesn't use it after that. Remove the setting of that
variable.
- Add a cond_resched() in ring_buffer_resize()
The resize function that allocates all the pages for the ring buffer was
causing a soft lockup on PREEMPT_NONE configs when allocating large
buffers on machines with many CPUs. Hopefully this is the last
cond_resched() needed to be added as PREEMPT_LAZY becomes the norm in the
future.
- Make ftrace_graph_ent depth field signed
The "depth" field of struct ftrace_graph_ent was converted from "int" to
"unsigned long" for alignment reasons to work with being embedded in other
structures. The conversion from a signed to unsigned caused integrity
checks to always pass as they were comparing "depth" to less than zero.
Make the field signed long.
- Add recursion protection to stack trace events
A infinite recursion was triggered by a stack trace event calling RCU
which internally called rcu_read_unlock_special(), which triggered an
event that was also doing stacktraces which cause it to trigger the same
RCU lock that called rcu_read_unlock_special() again.
Update the trace_test_and_set_recursion() to add a set of context checks
for events to use, and have the stack trace event use that for recursion
protection.
- Make the variable ftrace_dump_on_oops static
The cleanup of sysctl that moved all the updates to the files that use
them moved the reference of ftrace_dump_on_oops to where it is used.
It is no longer used outside of the trace.c file. Make it static.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCaV/2wRQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qmMqAQD+LyAOb7bKlgFjwRABjszg1yDhJPb0
gQGSNPchQyq/7gD8Cu3/ze5UxrNV8cNNsbAPu0/xEg4eyozbRiP/VjzZ4gU=
=uLUP
-----END PGP SIGNATURE-----
Merge tag 'trace-v6.19-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing fixes from Steven Rostedt:
- Remove useless assignment of soft_mode variable
The function __ftrace_event_enable_disable() sets "soft_mode" in one
of the branch paths but doesn't use it after that. Remove the setting
of that variable.
- Add a cond_resched() in ring_buffer_resize()
The resize function that allocates all the pages for the ring buffer
was causing a soft lockup on PREEMPT_NONE configs when allocating
large buffers on machines with many CPUs. Hopefully this is the last
cond_resched() needed to be added as PREEMPT_LAZY becomes the norm in
the future.
- Make ftrace_graph_ent depth field signed
The "depth" field of struct ftrace_graph_ent was converted from "int"
to "unsigned long" for alignment reasons to work with being embedded
in other structures. The conversion from a signed to unsigned caused
integrity checks to always pass as they were comparing "depth" to
less than zero. Make the field signed long.
- Add recursion protection to stack trace events
A infinite recursion was triggered by a stack trace event calling RCU
which internally called rcu_read_unlock_special(), which triggered an
event that was also doing stacktraces which cause it to trigger the
same RCU lock that called rcu_read_unlock_special() again.
Update the trace_test_and_set_recursion() to add a set of context
checks for events to use, and have the stack trace event use that for
recursion protection.
- Make the variable ftrace_dump_on_oops static
The cleanup of sysctl that moved all the updates to the files that
use them moved the reference of ftrace_dump_on_oops to where it is
used. It is no longer used outside of the trace.c file. Make it
static.
* tag 'trace-v6.19-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
trace: ftrace_dump_on_oops[] is not exported, make it static
tracing: Add recursion protection in kernel stack trace recording
ftrace: Make ftrace_graph_ent depth field signed
ring-buffer: Avoid softlockup in ring_buffer_resize() during memory free
tracing: Drop unneeded assignment to soft_mode
Remove KVM's internal pseudo-overlay of kvm_stats_desc, which subtly
aliases the flexible name[] in the uAPI definition with a fixed-size array
of the same name. The unusual embedded structure results in compiler
warnings due to -Wflex-array-member-not-at-end, and also necessitates an
extra level of dereferencing in KVM. To avoid the "overlay", define the
uAPI structure to have a fixed-size name when building for the kernel.
Opportunistically clean up the indentation for the stats macros, and
replace spaces with tabs.
No functional change intended.
Reported-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Closes: https://lore.kernel.org/all/aPfNKRpLfhmhYqfP@kspp
Acked-by: Marc Zyngier <maz@kernel.org>
Acked-by: Christian Borntraeger <borntraeger@linux.ibm.com>
[..]
Acked-by: Anup Patel <anup@brainfault.org>
Reviewed-by: Bibo Mao <maobibo@loongson.cn>
Acked-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Link: https://patch.msgid.link/20251205232655.445294-1-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Current release - fix to a fix:
- net: do not write to msg_get_inq in callee
- arp: do not assume dev_hard_header() does not change skb->head
Current release - regressions:
- wifi: mac80211: don't iterate not running interfaces
- eth: mlx5: fix NULL pointer dereference in ioctl module EEPROM
Current release - new code bugs:
- eth: bnge: add AUXILIARY_BUS to Kconfig dependencies
Previous releases - regressions:
- eth: mlx5: dealloc forgotten PSP RX modify header
Previous releases - always broken:
- ping: fix ICMP out SNMP stats double-counting with ICMP sockets
- bonding: preserve NETIF_F_ALL_FOR_ALL across TSO updates
- bridge: fix C-VLAN preservation in 802.1ad vlan_tunnel egress
- eth: bnxt: fix potential data corruption with HW GRO/LRO
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE6jPA+I1ugmIBA4hXMUZtbf5SIrsFAmlf6N8ACgkQMUZtbf5S
Irt9lxAAote4zPojTEJvE1DkABVsx31gPm6DF8zEae39XsFKtOMfb876s5bchOUA
Rkd27+k28l/U5HFrrhUhqDlaOikjXx6baT12qTPsuxGrOL6Um23EfIgVuNFzxMwE
lcBgkSNe1tfT8WBirWEVWLc5xme+vvll7ViX7kkQJ7fEdk1mvTPZ5roIq1+pE1U0
V5Gu4l9QBcWC/IymAO8Z2UE08terMcYt1G4H6mSIoooeMM1QElbPwVEiRWAzJ/NP
9cTjvnHJDAdRnA4bMa76CGWxg4wgPhgj3+ydlouWjgJADL6hlMj4sIZxaXgjDuoE
XyCEuk6Y/rUTSmX1yn7rha9FQwJOyMu9XlEjnNSvH0LRdnSa7xO7NzeXtrWv7HSg
kQOMTnMVgVlabOuMbR6xNqY6UyulQgK/2E56RgOO4Iw6U7crZsbyZx3OFkIKhq8g
ZWaRBQNdYBBftjJA7FwQSyj/K75sLfbYAS5YizguNyFPBCBhSBJdgFWoGb+XhT0/
k0KwsX/NWN0apHmZNiD4mT/UdX2PhJRdiTWPczNyEzqJcxh1P2HMWHVZOsIyZQHT
EK3w6LLqp1eshEERPqFsqCFYX4LUuifQbPPF0kBkL1hTMa2NuXnkIsOzbcWal88o
qdej9TY9VC5ycabgDI4/9ZNhnAzho1Nk/e6YuHsBcu2pIVDKgNI=
=vR9r
-----END PGP SIGNATURE-----
Merge tag 'net-6.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
Pull networking fixes from Jakub Kicinski:
"Including fixes from netfilter and wireless.
Current release - fix to a fix:
- net: do not write to msg_get_inq in callee
- arp: do not assume dev_hard_header() does not change skb->head
Current release - regressions:
- wifi: mac80211: don't iterate not running interfaces
- eth: mlx5: fix NULL pointer dereference in ioctl module EEPROM
Current release - new code bugs:
- eth: bnge: add AUXILIARY_BUS to Kconfig dependencies
Previous releases - regressions:
- eth: mlx5: dealloc forgotten PSP RX modify header
Previous releases - always broken:
- ping: fix ICMP out SNMP stats double-counting with ICMP sockets
- bonding: preserve NETIF_F_ALL_FOR_ALL across TSO updates
- bridge: fix C-VLAN preservation in 802.1ad vlan_tunnel egress
- eth: bnxt: fix potential data corruption with HW GRO/LRO"
* tag 'net-6.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (70 commits)
arp: do not assume dev_hard_header() does not change skb->head
net: enetc: fix build warning when PAGE_SIZE is greater than 128K
atm: Fix dma_free_coherent() size
tools: ynl: don't install tests
net: do not write to msg_get_inq in callee
bnxt_en: Fix NULL pointer crash in bnxt_ptp_enable during error cleanup
net: usb: pegasus: fix memory leak in update_eth_regs_async()
net: 3com: 3c59x: fix possible null dereference in vortex_probe1()
net/sched: sch_qfq: Fix NULL deref when deactivating inactive aggregate in qfq_reset
wifi: mac80211: collect station statistics earlier when disconnect
wifi: mac80211: restore non-chanctx injection behaviour
wifi: mac80211_hwsim: disable BHs for hwsim_radio_lock
wifi: mac80211: don't iterate not running interfaces
wifi: mac80211_hwsim: fix typo in frequency notification
wifi: avoid kernel-infoleak from struct iw_point
net: airoha: Fix schedule while atomic in airoha_ppe_deinit()
selftests: netdevsim: add carrier state consistency test
net: netdevsim: fix inconsistent carrier state after link/unlink
selftests: drv-net: Bring back tool() to driver __init__s
net/sched: act_api: avoid dereferencing ERR_PTR in tcf_idrinfo_destroy
...
