When waiting for a sub-module to register, we return a bare
-EPROBE_DEFER that ends up showing the end user:
platform arm-cs-arch-pmu.1: deferred probe pending (no reason)
wherein it's not necessarily clear that they might need to take some
action to ensure the appropriate module is available to load. Let's use
dev_err_probe() here so we can show exactly what we're waiting for.
Similarly, in the case where something's gone horribly wrong with an
already-registered module, we can use dev_WARN() to standardise the
device/driver attribution rather than just open-coding "arm_cspmu".
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
On ACPI systems, it has not actually been possible for userspace to
reliably tell which PMU corresponds to which APMT entry for types
other than "ACPI device" - the evidence trail only leads from the
arbitrarily-numbered PMU device to its arbitrarily-numbered parent
platform device that has no distinguishing features either.
While we've now improved the platform device creation to associate the
actual APMT unique ID, we may as well also tweak the PMU devices to
substitute the arbitrary number with a different arbitrary number that
might be more directly meaningful based on the APMT definitions.
We don't have an equivalent for Devicetree, but in that case the
platform devices are at least identifiable via their sysfs-visible
of_node.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
[will: Remove unneeded semicolon reported by coccinelle]
Signed-off-by: Will Deacon <will@kernel.org>
The APMT node format includes a unique identifier, so we can use this as
the platform device ID to give userspace stable and identifiable device
names, rather than auto numbering dependent on how the table is parsed.
Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
Cc: Hanjun Guo <guohanjun@huawei.com>
Cc: Sudeep Holla <sudeep.holla@kernel.org>
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Hanjun Guo <guohanjun@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
Implement CRB filtering per CXL 4.0 8.2.7.2.2, and extend the
current filtering support beyond HDM. CRB filtering is only
permitted for the DDR Interface, Queue Occupancy, Queue Residency
and Retry event groups (CXL 4.0 Table 13-5), and only when counting
a single event (a single mask bit). Because these group IDs are
scoped by the CXL vendor ID, events from other vendors are also
rejected. For example, to count DDR activates on channel 2 only:
perf stat -a -e cxl_pmu_mem0.0/ddr_act,crb_filter_en=1,crb=0x02FFFFFF/
Placing the 32-bit CRB value at config2:32-63 leaves the
existing HDM value at config2:0-15 untouched and avoids needing
a new config3.
Signed-off-by: Harshal Thakkar <harshal.t@samsung.com>
[davidlohr: multiple fixes]
Signed-off-by: Davidlohr Bueso <dave@stgolabs.net>
Reviewed-by: Richard Cheng <icheng@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Will Deacon <will@kernel.org>
Add support for CXL 4.0 events that are exposed by the CPMU hardware
but not present in the driver. Such events are defined in Table 13-5
of the spec.
Signed-off-by: Harshal Thakkar <harshal.t@samsung.com>
[davidlohr: add missing throttle and queue occupancy events]
Signed-off-by: Davidlohr Bueso <dave@stgolabs.net>
Reviewed-by: Richard Cheng <icheng@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Will Deacon <will@kernel.org>
The HDM decoder filter configuration register is 32 bits wide, but the
driver programs it with a 64-bit writeq(). The filter value never
exceeds 32 bits, so the upper half of the write is always zero and
lands in the adjacent Filter ID 1 (Channel/Rank/Bank) configuration
register at offset+4.
Fixes: 5d7107c727 ("perf: CXL Performance Monitoring Unit driver")
Signed-off-by: Davidlohr Bueso <dave@stgolabs.net>
Reviewed-by: Richard Cheng <icheng@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Will Deacon <will@kernel.org>
An SMMU implementing SMMU_IDR5.DS extends the range invalidation commands:
the SCALE field grows a 6th bit, raising its maximum value from 31 to 39,
and TTL == 0b01 becomes a valid level hint for a 16KB translation granule.
Add a new ARM_SMMU_FEAT_DS feature detecting the DS bit, and widen the
CMDQ_TLBI_0_SCALE field to its architectural 6 bits. Mask the scale value
explicitly in arm_smmu_cmdq_batch_add_range(), so the range invalidation
path emits the same commands as before, keeping the pre-existing 5-bit
truncation of a scale above 31.
Also list DS as a valid IDR5 field in the iommu_hw_info_arm_smmuv3 kdoc:
iommufd has always reported the raw IDR5 register, so a VMM may conclude
from that bit alone that it can expose DS to its guest.
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Assisted-by: Claude:claude-fable-5
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
The arm_smmu_write_reg_sync() helper is currently implemented using
readl_relaxed_poll_timeout() (that relies on usleep_range() internally)
which becomes a critical issue when used in the gerror irq handler.
If the SMMU hits a gerror and enters Service Failure Mode
(GERROR_SFM_ERR), the gerror handler calls arm_smmu_device_disable() in
hard-irq context. This becomes a problem as arm_smmu_device_disable()
inevitably calls arm_smmu_write_reg_sync() which might attempt to sleep
inside a hard-irq context.
Fix this by converting the arm_smmu_write_reg_sync to use the
readl_relaxed_poll_timeout_atomic() polling helper.
(Discovered while running Sashiko locally on another patch series).
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 48ec83bcbc ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Signed-off-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Since table access flags cannot be software-managed, if process
pagetables are using HAFT then SVA must require the SMMU to support and
enable it too, otherwise page aging is liable to get out of whack.
For unbinding, we can't disable HAFT atomically with HA as might be
desired, but luckily we can get away with just not disabling HA either.
Cc: stable@vger.kernel.org
Fixes: 62df5870eb ("arm64: Enable ARCH_HAS_NONLEAF_PMD_YOUNG")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
In general it might be nice to have the ability to disable hardware
access/dirty bit management for debugging or performance comparison
purposes without having to rebuild the kernel. However once FEAT_HAFT
comes into the picture we also start to have a real functional concern
where the decision to use HAFT based on the boot CPUs can prevent SVA
or late-onlining if SMMUs/CPUs are later found to lack HAFT support.
To that end, add the appropriate MMFR1 override, with an easy "nohaft"
alias for the significant case, partly since the feature/field naming
isn't the most obvious, but also so it could potentially be redirected
in future if someone wanted to attempt a higher-level means of turning
off just HAFT usage independently from FEAT_HDBSS.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Commit 3e360ef0c0 ("kselftest/arm64: Corrupt P0 in the irritator when
testing SSVE") added corruption of P0 to the sve-test case in order to
ensure that the predicate registers were covered as part of the
corruption. On SME only systems this results in an illegal instruction
since signal handlers are run out of streaming mode and the predicate
registers do not exist out of streaming mode without SVE. Switch to
entering and exiting streaming mode in the irritator, this will reset
all relevant registers to 0 if they somehow weren't already by the
signal entry.
Fixes: 3e360ef0c0 ("kselftest/arm64: Corrupt P0 in the irritator when testing SSVE")
Reported-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
syscall_get_arguments() returns 'regs->orig_x0' for the first system
call argument so as to avoid aliasing with the syscall return value in
'regs->regs[0]' on the return path, however the actual syscall
invocation passes 'regs->regs[0]' as the first parameter.
Although the two registers should be kept in sync during syscall entry
for native tasks, pass 'regs->orig_x0' as the first syscall parameter
for consistency with the syscall argument APIs. Compat tasks continue to
use 'regs->regs[0]' for compatibility with the behaviour of the 32-bit
kernel.
Suggested-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
Commit e057b94772 ("arm64: syscall: Ensure saved x0 is kept in-sync
with tracer updates") attempted to resolve a long-standing issue with
syscall entry tracing, where a tracer is able to manipulate the first
syscall argument without being subjected to seccomp or audit checking.
Unfortunately, that fix was incomplete [1], as it failed to update
'orig_x0' between a tracer updating x0 during a seccomp ptrace exit
(SECCOMP_RET_TRACE) and the seccomp filter being re-evaluated.
Rather than add hooks to the core seccomp code, instead move the
synchronisation code into the ptrace GPR and syscall setting code so
that 'orig_x0' is kept up to date with x0 whenever we're stopped on the
syscall entry path.
Cc: Kees Cook <kees@kernel.org>
Cc: Jinjie Ruan <ruanjinjie@huawei.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Link: https://sashiko.dev/#/patchset/20260716120640.6590-1-will@kernel.org [1]
Reported-by: Yiqi Sun <sunyiqixm@gmail.com>
Link: https://lore.kernel.org/all/20260529065444.1336608-1-sunyiqixm@gmail.com/
Fixes: e057b94772 ("arm64: syscall: Ensure saved x0 is kept in-sync with tracer updates")
Fixes: a5cd110cb8 ("arm64/ptrace: run seccomp after ptrace")
Tested-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
Add a selftest that verifies the kernel re-evaluates a seccomp filter
with the correct (ptrace-modified) first argument after
a SECCOMP_RET_TRACE stop. On arm64, syscall_get_arguments() reads
the first argument from orig_x0, which may be stale if the tracer modified
regs->regs[0] but orig_x0 was not synced. This can cause the filter to
see an old argument and incorrectly allow a syscall that it should
have rejected.
The child installs a filter that:
- TRACEs write() when fd == 2
- returns ERRNO(EPERM) when fd == 1
The parent catches the SECCOMP event, changes x0 (fd) from 2 to 1,
and resumes the child.
If the seccomp re-evaluation sees the stale orig_x0 (fd=2) the filter
returns TRACE again and the kernel (with recheck_after_trace=true)
allows the syscall to proceed – write succeeds and the child exits 0.
If the seccomp re-evaluation sees the new value (fd=1) the filter
returns ERRNO(EPERM), write fails and the child exits non-zero.
The test passes only when the write fails (child exit != 0).
Before the fix:
# ./seccomp_ret_trace_x0_bypass
TAP version 13
1..1
not ok 1 write succeeded, orig_x0 bypass likely
# Totals: pass:0 fail:1 xfail:0 xpass:0 skip:0 error:0
After the fix:
# ./seccomp_ret_trace_x0_bypass
TAP version 13
1..1
ok 1 seccomp correctly denied modified syscall
# Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0
Cc: Kees Cook <kees@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/all/20260717182758.17111-1-will@kernel.org/
Link: https://lore.kernel.org/all/20260716120640.6590-1-will@kernel.org/
Link: https://lore.kernel.org/all/202607152004.DEA95D63@keescook/
Suggested-by: Kees Cook <kees@kernel.org>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
As Kees suggested, add a test that verifies that seccomp observes the
correct first argument after a ptracer modifies x0 at a syscall-enter-stop
on arm64.
The first syscall argument and the return value share register x0.
The original value is saved in orig_x0 on entry and used by
syscall_get_arguments(), but ptrace changes to x0 were not
automatically reflected there. This test checks the kernel re-syncs
orig_x0 after a ptrace stop so that seccomp sees the modified
argument.
A seccomp filter allows write(2,...) and kills the task for any other
fd. The tracer changes fd from 2 to 1 at entry. If orig_x0 remains
stale, the child exits normally (bypass, test fails). If orig_x0 is
correctly updated, the child is killed by SIGSYS (test passes).
Before the fix:
./seccomp_ptrace_x0_bypass
TAP version 13
1..1
not ok 1 seccomp_ptrace_x0_bypass
# Totals: pass:0 fail:1 xfail:0 xpass:0 skip:0 error:0
After the fix:
# ./seccomp_ptrace_x0_bypass
TAP version 13
1..1
[ 19.475951] audit: type=1326 audit(1784254846.284:2): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=227 comm="seccomp_ptrace_" exe="/mnt/seccomp0
[ 19.477852] audit: type=1701 audit(1784254846.284:3): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=227 comm="seccomp_ptrace_" exe="/mnt/seccomp1
ok 1 seccomp_ptrace_x0_bypass
# Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0
Cc: Kees Cook <kees@kernel.org>
Cc: Will Deacon <will@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/all/20260716120640.6590-1-will@kernel.org/
Link: https://lore.kernel.org/all/202607152004.DEA95D63@keescook/
Suggested-by: Kees Cook <kees@kernel.org>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
The checks on the header size reported for the inactive regset of the
NT_ARM_SVE/NT_ARM_SSVE pair compare it against sizeof(sve), but sve is
a struct user_sve_header *, so this is 8 rather than the intended 16.
The kernel carried the identical typo when filling in the header, so
kernel and test agreed on the wrong value and the test passed.
Compare against sizeof(*sve), stop after the header checks for an
inactive regset since it has no payload to compare, and prefill the
buffer with a sentinel to verify that reading an inactive regset
leaves everything after the header untouched. This also covers the
getter's return value, which determines how many bytes ptrace copies
back to userspace.
Fixes: 864f3ddcd7 ("kselftest/arm64: fp-ptrace: Adjust to new inactive mode behaviour")
Assisted-by: Claude:claude-opus-5
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Will Deacon <will@kernel.org>
sve_init_header_from_task() takes header as a pointer, so for the
inactive mode
header->size = sizeof(header);
stores 8 rather than sizeof(struct user_sve_header), which is 16.
Userspace sees an impossible size smaller than the header it
describes.
The inactive-mode check in sve_get_common() compares header.size
against sizeof(header) as well, but there header is a struct, so the
check can never fire. Reads of NT_ARM_SVE and NT_ARM_SSVE for the
inactive mode therefore still return the other mode's FPSIMD data,
exactly the situation the check was added to prevent.
Fix the size, and make the check return the remaining membuf space
instead of 0, which regset_get() would interpret as the entire
(zero-filled) buffer having been populated.
Fixes: b93e685ecf ("arm64/fpsimd: ptrace: Do not present register data for inactive mode")
Assisted-by: Claude:claude-opus-5
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Will Deacon <will@kernel.org>
The trailing semicolon belongs at the point of use, not in the macro
definition. All uses have been verified to have their own semicolons.
This was found using the following Coccinelle semantic patch:
@r@
identifier i : script:ocaml() { String.lowercase_ascii i = i };
expression e;
@@
*#define i(...) e;
Signed-off-by: Julia Lawall <Julia.Lawall@inria.fr>
Signed-off-by: Tejun Heo <tj@kernel.org>
T241-MPAM-6 causes all MBWU counter formats to count 64-byte
requests instead of bytes. Commit dc48eb1ff2 excluded the 63-bit
MSMON_MBWU_LWD format while scaling the shorter counters. Systems
selecting the preferred 63-bit counter consequently report bandwidth
values that are 64 times too small.
Apply the scale to both the sampled value and overflow correction for
the 63-bit format. Unsigned arithmetic retains modulo-u64 behavior
when the scaled counter range exceeds u64.
Fixes: dc48eb1ff2 ("arm_mpam: Add workaround for T241-MPAM-6")
Link: https://lore.kernel.org/lkml/20240816131432.993859-1-sdonthineni@nvidia.com/
Signed-off-by: Shanker Donthineni <sdonthineni@nvidia.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Tested-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Memory bandwidth monitoring make uses of MBWU monitors and is now exposed
to the user via resctrl. Add some documentation so the user knows what to
expect.
Co-developed-by: James Morse <james.morse@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Will Deacon <will@kernel.org>
ABMC, mbm_event mode, has a helper resctrl_arch_config_cntr() for changing
the mapping between 'cntr_id' and a CLOSID/RMID pair.
Add the helper.
For MPAM this is done by updating the mon->mbwu_idx_to_mon[] array, and as
usual CDP means it needs doing in three different ways.
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Tested-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Tested-by: Fenghua Yu <fenghuay@nvidia.com>
Tested-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Will Deacon <will@kernel.org>
MPAM is able to emulate ABMC, i.e. mbm_event mode, by making memory
bandwidth monitors assignable. Rather than supporting the 'default'
mbm_assign_mode always use 'mbm_event' mode even if there are sufficient
memory bandwidth monitors. The per monitor event configuration is only
provided by resctrl when in 'mbm_event' mode and so only allowing
'mbm_event' mode will make it easier to support per-monitor event
configuration for MPAM. For the moment, the only event supported is
mbm_total_event with no bandwidth type configuration. The 'mbm_assign_mode'
file will still show 'default' when there is no support for memory
bandwidth monitoring.
The monitors need to be allocated from the driver, and mapped to whichever
control/monitor group resctrl wants to use them with.
Add a second array to hold the monitor values indexed by resctrl's cntr_id.
When CDP is in use, two monitors are needed so the available number of
counters halves. Platforms with one monitor will have zero monitors when
CDP is in use.
Co-developed-by: James Morse <james.morse@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Tested-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Tested-by: Fenghua Yu <fenghuay@nvidia.com>
Tested-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Will Deacon <will@kernel.org>
resctrl has two types of bandwidth counters, NUMA-local and global. MPAM
can only count globally; either using MSC at the L3 cache or in the memory
controllers. When global and local equate to the same thing continue just
to call it global.
Pick the corresponding MPAM classes to back the MBM counters. As resctrl
requires all monitors to be at the L3 cache, we can only use the counters
at the memory controllers when they have the same topology as the L3 cache
and the traffic they see if the same. In particular, for the bandwidth
counters at the memory controllers to be exposed to resctrl it is required
there is a single L3 cache and a single NUMA node as otherwise cross NUMA
traffic will be counted at the wrong instance.
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Tested-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Tested-by: Zeng Heng <zengheng4@huawei.com>
Tested-by: Fenghua Yu <fenghuay@nvidia.com>
Tested-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Will Deacon <will@kernel.org>
Sashiko reported that during CPU offlining, __cpu_disable() is
executed by the stopper thread via take_cpu_down() with local
interrupts disabled. __cpu_disable() calls ipi_teardown(), which
invokes ipi_lpi_disable(). For the GICv5 flow, this eventually calls
the sleepable disable_irq().
This can be reproduced easily with CONFIG_DEBUG_ATOMIC_SLEEP=y by
offlining a CPU:
BUG: sleeping function called from invalid context at kernel/irq/manage.c:702
in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 20, name: migration/1
preempt_count: 1, expected: 0
no locks held by migration/1/20.
irq event stamp: 186
hardirqs last enabled at (185): [<ffff800080b084c8>] _raw_spin_unlock_irq+0x38/0x68
hardirqs last disabled at (186): [<ffff8000801f8e08>] multi_cpu_stop+0xc8/0x190
softirqs last enabled at (80): [<ffff8000800c48b8>] handle_softirqs+0x410/0x468
softirqs last disabled at (75): [<ffff8000800102f4>] __do_softirq+0x1c/0x28
Fix this by using disable_irq_nosync() instead, which is safe in this
atomic context.
Fixes: ba1004f861 ("arm64: smp: Support non-SGIs for IPIs")
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
A CPU wedged with interrupts masked ignores the stop IPI, and without
pseudo-NMI there is no NMI IPI to escalate to: a reboot proceeds with
the CPU still running, and a kdump misses its registers.
Add a third rung to smp_send_stop(): once the IPI (and pseudo-NMI IPI,
if enabled) rungs have run, signal SDEI event 0 at whatever stayed
online. Firmware delivers it regardless of the target's DAIF, so it
reaches a CPU a plain IPI cannot; the target acks by going offline,
which the caller already polls for.
Fold the stop bookkeeping into one arm64_nmi_cpu_stop(regs,
die_on_crash), shared by the stop IPI handlers, panic_smp_self_stop()
and the SDEI handler, replacing the near-duplicate local_cpu_stop() and
ipi_cpu_crash_stop(). @die_on_crash is the only difference: the IPI
handlers pass true and PSCI CPU_OFF the CPU on a crash stop so a capture
kernel can reclaim it; the SDEI handler and self-stop pass false and
park. The SDEI park is required, not conservative -- its handler runs
inside an SDEI event that is never completed (completing it resumes the
wedged context), and a CPU_OFF from that unfinished-event context wedges
EL3 on some firmware (left as a follow-up). The dump is unaffected; only
re-onlining the CPU in an SMP capture kernel is lost.
Suggested-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Yin Fengwei <fengwei_yin@linux.alibaba.com>
Signed-off-by: Will Deacon <will@kernel.org>
Deliver an NMI-like event to an interrupt-masked arm64 CPU via the
standard SDEI software-signalled event (event 0), without the pseudo-NMI
hot-path cost: register a handler for event 0 and poke a target with
sdei_event_signal(0, mpidr).
First user is arch_trigger_cpumask_backtrace() (sysrq-l, RCU stalls,
hung-task/soft-lockup dumps), which otherwise rides an IPI that can't
reach a masked CPU. Falls back to the IPI path when SDEI is absent; no
watchdog backend yet, so the stock detector is untouched.
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Yin Fengwei <fengwei_yin@linux.alibaba.com>
Signed-off-by: Will Deacon <will@kernel.org>
Add sdei_event_signal(), a thin wrapper over the SDEI_EVENT_SIGNAL call
(DEN0054) that makes the software-signalled event (event 0) pending on a
target PE -- delivered NMI-like even when that PE has interrupts masked.
It takes no locks, so it is safe to call from NMI / crash context.
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Yin Fengwei <fengwei_yin@linux.alibaba.com>
Signed-off-by: Will Deacon <will@kernel.org>
invoke_sdei_fn() returns -EIO when no SDEI conduit was probed, and the
core warns ("Failed to create event ...") on any registration that hits
that. An optional consumer that registers an event from an unconditional
initcall would therefore make every boot on a non-SDEI system emit that
warning for what is simply absent firmware.
Expose whether SDEI firmware is present so such a consumer can skip
registration -- and the warning -- when there is nothing to talk to.
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Yin Fengwei <fengwei_yin@linux.alibaba.com>
Signed-off-by: Will Deacon <will@kernel.org>
There is no functional requirement to keep the entries in arm64's
tools/cpucaps file sorted alphabetically and, in fact, they have fallen
out of order over time. Given that only the arm64 tree touches this file
and the perceived benefit of sorting was to help with conflicts, just
remove the stale comment and accept that we're collectively not very
good at the alphabet.
In the limited cases where a specific ordering is important, we enforce
it through build-time assertions (e.g. in can_use_gic_priorities()).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Reported-by: Marc Zyngier <maz@kernel.org>
Acked-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
On systems where CLIDR_EL1.LoC == 0, no cache maintenance is required
when cleaning or invalidating to the Point of Coherency and therefore
all DMA agents can be treated as coherent.
Extend arch_setup_dma_ops() to take CLIDR_EL1.LoC into account when
setting the DMA ops for a device, emitting a warning message if the
firmware advertises a non-coherent device on a fully coherent system.
Cc: Steffen Eiden <seiden@linux.ibm.com>
Cc: Andreas Grapentin <gra@linux.ibm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
A robust futex unlock stores 0 over the whole futex value - wiping
FUTEX_WAITERS - and wakes a single waiter. That wakeup is a one-shot
notification: the protocol relies on its recipient to either acquire the
futex (and eventually unlock while aware of the remaining contention) or
re-arm FUTEX_WAITERS before sleeping again. If the woken waiter is killed
before it can do either, the kernel must jump in and wake the next task
down the line.
This is a known complication of the futex protocol with a previous
partial fix in commit ca16d5bee5 ("futex: Prevent robust futex exit
race"). Unfortunately, that fix is insufficient.
If a third task re-acquired the futex through the uncontended fast
path in the meantime, the notification is lost: robust exit processing
sees that it is owned by another task and does nothing, while the new
owner sees no FUTEX_WAITERS when it unlocks and wakes nobody.
The remaining waiters sleep forever behind a free futex:
A owns the futex, B and C sleep in FUTEX_WAIT
uval == A | FUTEX_WAITERS
A robust unlock: store 0, FUTEX_WAKE(1) wakes B
uval == 0
D fast path acquire: cmpxchg(0 -> D)
uval == D, no FUTEX_WAITERS
B killed before acting on the wakeup
B exit walk, pending op: owner D != B -> no action
D unlock: no FUTEX_WAITERS -> no wake
C sleeps forever
This is clearly a shortcoming in the implementation, which fails to keep
the FUTEX_WAITERS bit consistent.
Work around this by augmenting the robust list exit processing to also
perform the extra wakeup if the futex word is owned by another thread but
FUTEX_WAITERS is not set.
This does not fix the problem of a non-contended take over/release and free
sequence, which has been discussed for years and has been addressed by
commit 3ca9595d9f ("futex: Add support for unlocking robust futexes") and
subsequent changes, but failed to take the problem described above into
account.
A more complete solution which is based on the in kernel unlock of
contended robust futexes has been discussed in the context of this change
and should show up in mainline sooner than later.
[ tglx: Amend change log slightly and fixup coding style ]
Fixes: ca16d5bee5 ("futex: Prevent robust futex exit race")
Signed-off-by: Keno Fischer <keno@juliahub.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Assisted-by: ClaudeCode:claude-fable-5 tla+
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260730194705.38981-1-keno@juliacomputing.com
The i2c subsystem currently blocks during adapter deregistration
whenever there are consumers holding a reference.
Switch to using of_get_i2c_adapter_by_node() which also takes a
reference to the adapter module so that an attempt to unload the module
while in use fails gracefully instead of blocking uninterruptibly.
Signed-off-by: Johan Hovold <johan@kernel.org>
Signed-off-by: Inki Dae <inki.dae@samsung.com>
Not handling the return code on drm_encoder_init can lead to silent
failure and/or a drm_encoder_cleanup on a non-initialized encoder.
This patch adds error handling to the drm_encoder_init calls to prevent
that from happening.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
Signed-off-by: Inki Dae <inki.dae@samsung.com>
Simple KMS helper are deprecated since they only add an intermediate
layer between drivers and the atomic modesetting.
This patch removes the dependency on drm simple helpers from exynos
DRM drivers.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
Dropped unnecessary blank line and reorder header definition.
Signed-off-by: Inki Dae <inki.dae@samsung.com>
RTL8922DE is a WiFi 7 chipset, supporting 2x2 2GHz/5GHz/6GHz 4096/1024-QAM
160MHz channels. As STA, AP and P2P modes work well, enable this chipset.
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260730060220.55844-8-pkshih@realtek.com
The 9.24.1 coexistence version included firmware 0.35.111.X support for
RTL8922A/D, 0.24.97.X support for RTL8852C, 0.29.133.X support for
RTL8852B chip family.
Signed-off-by: Ching-Te Ku <ku920601@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260730060220.55844-7-pkshih@realtek.com
The old lps_ctrl_scbd logic in _update_bt_scbd() blocked WiFi LPS only
when mode == BTC_WLINK_V0_2G_STA, which created a dead-lock while
WiFi already in LPS suppressed TDD binding so mode was never set to
BTC_WLINK_V0_2G_STA, lps_ctrl_scbd stayed 0, and WiFi remained stuck
in LPS. Port _update_bt_ctrl_lps() from the reference implementation to
check BT profile existence (A2DP, HFP, PAN) for both radios directly,
with out_of_band and freerun guards so LPS is only blocked when WiFi and
Bluetooth actually share a band. Call it from both _update_bt_scbd() and
_update_bt_info() so profile changes reported via either SCBD or BT info
C2H correctly update lps_ctrl_scbd. Also remove the dead
lps_ctrl_scbd_last field.
Signed-off-by: Ching-Te Ku <ku920601@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260730060220.55844-6-pkshih@realtek.com
Logic should only get explicit link_mode only, to cover the old branch
link_mode_v0 using (old branch firmware needed) should do translating
after link_mode is settled. And the coexistence logic has already updated
to new branch style, so the logic should use new link_mode, don't need
to consider link_mode_v0. To prevent unexpected logic bug, refine the
related logic.
Signed-off-by: Ching-Te Ku <ku920601@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260730060220.55844-5-pkshih@realtek.com
When a BT device is disabled, _update_bt_link_cnt() is no longer called
for that device, so its link_weight[] retains stale values from the last
active period. _set_bind_info() then computes a non-zero band score for
the disabled device, which can lead to an incorrect tdd_bind.rf_band
selection and ultimately wrong coexistence policy.
Clear link_weight[] for any disabled BT device at the start of the
_set_bind_info() loop so that stale scores are not carried forward.
Signed-off-by: Ching-Te Ku <ku920601@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260730060220.55844-4-pkshih@realtek.com
The _update_bt_info() function always parsed BT-info for bt0 and used a
single raw_info buffer regardless of which BT device sent the packet or
which RF band it belongs to. This caused two bugs:
1. BT-info packets from bt1 were incorrectly parsed into bt0's state.
2. BT-info packets from 5/6 GHz BT (L1 bit7=1) overwrote the 2.4 GHz
link_info and raw_info, breaking duplicate detection across bands.
Fix by adding the bid parameter to select bt0/bt1, detecting the 5/6 GHz
band flag (L1 bit7) to route packets into link_info_56g with a dedicated
raw_info_56g buffer, and updating the early-return mask to ignore bit7
when checking the length field.
Signed-off-by: Ching-Te Ku <ku920601@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260730060220.55844-3-pkshih@realtek.com
_update_bt_link_cnt() was missing the bt->link_weight[].
Without it, link_weight stays at the initial value of 5
(set on BT re-enable) regardless of the active BT profile.
_set_bind_info() uses link_weight to compute b2g_score/b5g_score, which
determines tdd_bind.rf_band. With a stale weight of 5 (below the active
profile threshold), tdd_bind.rf_band may not reflect the actual RF band,
causing mode_v0 in _run_coex() to remain 0 (BTC_WLINK_NOLINK), which
incorrectly triggers _action_wl_nc() instead of the BT-profile action.
Add the link_weight calculation to _update_bt_link_cnt() using the same
weighting table as Formal (BIS/A2DP-sink: 70, A2DP: 40-60, PAN/active: 30,
no-profile: 5, else: 9) and call _update_bt_link_cnt() from
_update_bt_info() after parsing the profile exist flags, replacing the
open-coded link_cnt increment that omitted le-audio profiles.
Signed-off-by: Ching-Te Ku <ku920601@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260730060220.55844-2-pkshih@realtek.com
Adapt H2C format to fit firmware version after 0.35.113.2, which
only supports active scanning with a provided SSID list.
The wildcard_6ghz field is no longer required for WiFi 7 chips' new
format, but not removed since WiFi 6 still requires it.
Keep original H2C style also to support older firmwares.
Signed-off-by: Po-Hao Huang <phhuang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260729124354.3231368-4-pkshih@realtek.com
The version check with less than should check for smallest upper-bound
first, or some branch will be unreachable.
Signed-off-by: Po-Hao Huang <phhuang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260729124354.3231368-3-pkshih@realtek.com
When an AP interface is registered and starts AP, mac80211 calls the
configure_filter() to clear B_AX_A_UC_CAM_MATCH and B_AX_A_BC_CAM_MATCH
so that frames from un-associated stations can be received. However, for
a dedicated AP interface created via iw command, configure_filter() is
only triggered on the initial AP startup. Since the interface remains up
even after hostapd stops, and it isn't triggered again on later restarts.
Additionally, each AP start causes IPS leave, reverting the RX filter to
its default value.
For WiFi 7 chips, the default value is hardcoded in rx_fltr_init_be(),
so the reverted value does not match hal.rx_fltr. Therefore, refine
the behavior of WiFi 7 chips to align with the WiFi 6 implementation.
Signed-off-by: Dian-Syuan Yang <dian_syuan0116@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260729124354.3231368-2-pkshih@realtek.com
Since the hardware rfkill polling was introduced, arm64 platforms can
panic with an asynchronous SError during warm reboot:
SError Interrupt on CPU8, code 0x00000000be000011 -- SError
Workqueue: events_power_efficient rfkill_poll [rfkill]
rtw89_pci_ops_read8+0x94/0x160 [rtw89_pci]
rtw89_core_rfkill_poll+0x50/0x1e0 [rtw89_core]
rtw89_ops_rfkill_poll+0x40/0x68 [rtw89_core]
ieee80211_rfkill_poll+0x3c/0x70 [mac80211]
cfg80211_rfkill_poll+0x40/0x2a0 [cfg80211]
rfkill_poll+0x30/0x88 [rfkill]
Kernel panic - not syncing: Asynchronous SError Interrupt
On the reboot path the kernel only runs device_shutdown(), which calls
each driver's .shutdown callback; .remove is not invoked. The rtw89 PCI
driver had no .shutdown callback, so nothing stopped the rfkill polling
work while the platform was tearing the PCIe link down. Once the link
is gone, the next MMIO read from the poll handler targets a
non-responding device and is reported as a fatal asynchronous SError on
arm64.
Add rtw89_pci_shutdown(), wired to all rtw89 PCI device drivers, which
sets a new RTW89_FLAG_SHUTDOWN flag (mirroring the USB
RTW89_FLAG_UNPLUGGED pattern). When the flag is set,
rtw89_ops_rfkill_poll() returns early, so no MMIO read is issued to the
chip after shutdown begins and the SError no longer occurs.
This does not call the full .remove path from .shutdown, to keep the
shutdown handler minimal and avoid running the non-idempotent teardown
twice.
Fixes: 0b38e6277a ("wifi: rtw89: add support for hardware rfkill")
Cc: stable@vger.kernel.org
Suggested-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Yuhang.chen <yhchen312@gmail.com>
Acked-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Link: https://patch.msgid.link/20260729014142.2746777-1-yhchen312@gmail.com
Janne Grunau <j@jannau.net> says:
This series adds device trees for Apple silicon devices with M3 Pro, Max
and Ultra SoCs. The M3 generation has fewer devices than their M1 and M2
predecessors. The only non-laptop device is the M3 Ultra Mac Studio. The
Laptops are the known 14 and 16-inch Macbook Pros now with M3 Pro and
M3 Max SoCs. The M3 Max variant with fewer CPU and GPU cores has
additionally only a 384-bit wide memory bus instead of 512-bit of the
full M3 Max. It has a separate identifier (T6034) and so there are six
laptop device trees.
Another difference to M1 and M2 Pro/Max/Ultra is that the M3 Pro is
distinct SoC design and not a smaller M3 Max. For this reason both M3
Max variants and the M3 Ultra will use "apple,t6030" as compatible
prefix. In the M1 and M2 generations Pro, Max and Ultra SoCs shared
"apple,t6000" / "apple,t6020" as common prefix. There is currently no
known difference but M3 Pro and M3 Max are not as closely related as
previously.
This series adds the same level of hardware as the base M3 (T8122) has
in v7.2-rc1. This includes CPU cores, interrupt controller, power
states, watchdog, serial, pin controller, i2c and the boot framebuffer.
This is intended as base so that support for additional hardware can be
added to all M3 based devices at the same time.
Link: https://patch.msgid.link/20260724-apple-t603x-initial-devices-v3-0-bbeba0420603@jannau.net
Signed-off-by: Sven Peter <sven@kernel.org>
Minimal device trees for the M3 Pro based 14-inch and 16-inch MacBook
Pro released in November 2023. The M3 Pro is a distinct SoC design and
not a cut down version of the Max variant like for M1 and M2 Pro.
The M3 Pro has only a single cluster with up to 6 performance cores and
one cluster with 6 efficiency cores.
While it no longer shares a common SoC .dtsi the devices itself are
still similar to the ones with M3 Max. It is still useful to share the
basic device composition in t603x-j514-j516.dtsi. For the M3 Pro based
devices additional overrides for diverging pin and irq numbers are
expected.
The device trees have devices nodes for CPU cores, timer, interrupt
controller, power states, watchdog, serial, pin controller, i2c,
PWM based keyboard LED illumination and the boot framebuffer.
Reviewed-by: Sven Peter <sven@kernel.org>
Signed-off-by: Janne Grunau <j@jannau.net>
Signed-off-by: Sven Peter <sven@kernel.org>
Contains minimal device trees for all M3 Max and Ultra Apple silicon
devices. Those are 14-inch and 16-inch MacBook Pros with M3 Max released
in November 2023 and Mac Studio with M3 Ultra released in 2025.
This does not include M3 Pro since it is separate SoC and not a smaller
variant of the M3 Max contrary to the M1 and M2 generations.
The smaller M3 Max variant (10 performance cores) has its own chip
variant (T6034) but is clearly the same design as T6031. Besides fewer
CPU performance cores and GPU cores it misses also on fourth of the
memory controllers and thus has an aggregated bus width of 384 bit
instead of 512 bit.
Both M3 Ultra variants (28 or 32 CPU cores) are based on T6031 judging
by the advertised memory bandwidth of 819GB/s.
This uses the same multi-die macros as t600x*.dtsi and t602x.dtsi to
support M3 Max and M3 Ultra without duplicating device nodes. Since the
M3 Pro can't use the same .dtsi files "t6031*.dtsi" are used to define
common nodes for T6031, T6032 and T6034.
The device trees have devices nodes for CPU cores, timer, interrupt
controller, power states, watchdog, serial, pin controller, i2c,
PWM based keyboard LED illumination and the boot framebuffer.
Reviewed-by: Sven Peter <sven@kernel.org>
Signed-off-by: Janne Grunau <j@jannau.net>
Signed-off-by: Sven Peter <sven@kernel.org>
