Update the existing packet socket test to include a test for the sockopt
PACKET_STATISTICS.
Signed-off-by: Joe Damato <joe@dama.to>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Since commit 1b9707e6f1 ("net: stmmac: enable RPS and RBU
interrupts"), suspending causes an interrupt storm from the RPS
interrupt.
Fix this by adding a deinit_chan() op to stmmac_dma_ops, which
masks all default dma channel interrupts. This is called from
stmmac_stop_all_dma(), so interrupts don't trigger while suspending.
Fixes: 1b9707e6f1 ("net: stmmac: enable RPS and RBU interrupts")
Suggested-by: Andrew Lunn <andrew@lunn.ch>
Suggested-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
Signed-off-by: Luis Lang <luis.la@mail.de>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Tested-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
Reviewed-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
Link: https://patch.msgid.link/20260720111534.163416-1-luis.la@mail.de
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
arm64 implements CPPC FFH feedback-counter reads using AMU counters.
Because those counters must be sampled on the target CPU, reading the
delivered and reference counters separately widens the observation window
between them.
Implement the paired FFH feedback-counter read hook on arm64 and sample
both AMU counters together before decoding the requested CPC register
values.
Also factor the FFH bitfield extraction logic into a helper and reuse
it from the existing single-counter FFH read path.
Tested-by: Sumit Gupta <sumitg@nvidia.com>
Reviewed-by: Sumit Gupta <sumitg@nvidia.com>
Tested-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com>
Reviewed-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com>
Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com>
Tested-by: Jeremy Linton <jeremy.linton@arm.com>
Reviewed-by: Jeremy Linton <jeremy.linton@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
cppc_get_perf_ctrs() reads the delivered and reference performance
counters one at a time.
Allow architectures to provide both FFH feedback counters in one
operation when that either narrows the sampling window or avoids extra
cross-CPU reads. Add a small FFH-specific hook for that case and fall
back to the existing per-register reads when unsupported.
Tested-by: Sumit Gupta <sumitg@nvidia.com>
Reviewed-by: Sumit Gupta <sumitg@nvidia.com>
Tested-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com>
Reviewed-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com>
Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com>
Acked-by: Rafael J. Wysocki (Intel) <rafael@kernel.org>
Tested-by: Jeremy Linton <jeremy.linton@arm.com>
Reviewed-by: Jeremy Linton <jeremy.linton@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
After executing cpufreq sptest, the system governor
will be overwritten with the governor switched
during the test. Restore this setting to maintain
consistency before and after the test.
Signed-off-by: Yiwei Lin <s921975628@gmail.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://patch.msgid.link/20260707163647.6646-1-s921975628@gmail.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
The cpufreq selftests are always executed through main.sh, which verifies
that the test is run as root before dispatching any test case.
Therefore, invoking sudo inside quick_shuffle() is redundant and may cause
failures in environments where sudo is unavailable.
Signed-off-by: Jinseok Kim <always.starving0@gmail.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://patch.msgid.link/20260706143857.3306-2-always.starving0@gmail.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
switch_show_governor() assigns the current governor and frequency
to local variables before switching governors.
However, these variables are never referenced afterwards. The function
does not restore the previous governor or use the saved frequency, as
backup_governor() and restore_governor() already handle state preservation
elsewhere.
Signed-off-by: Jinseok Kim <always.starving0@gmail.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Link: https://patch.msgid.link/20260706143857.3306-1-always.starving0@gmail.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Before converting to the phylink interface, the init function would have
set a non-reserved I/F mode in the maccfg2 register. After converting to
phylink, 0 is written as mode, which is a reserved value (although it's
the hardware default). Without a valid mode, a SGMII link is never
established between the MAC and the PHY and thus .link_up() is never
called which could set the correct mode according to the actual speed.
Fix it by setting the maximum speed of the phy_interface_t in use in
.mac_config() - just like the driver did before the phylink conversion.
Fixes: 5d93cfcf73 ("net: dpaa: Convert to phylink")
Suggested-by: Sean Anderson <sean.anderson@linux.dev>
Signed-off-by: Michael Walle <mwalle@kernel.org>
Reviewed-by: Sean Anderson <sean.anderson@linux.dev>
Reviewed-by: Sean Anderson <sean.anderson@linux.dev>
Link: https://patch.msgid.link/20260717132401.2653252-1-mwalle@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Return the bitmap size, rather than size + 1, when
bitmap_find_next_zero_area_off() cannot find a suitable area. This
matches the conventional find_bit() failure sentinel and still lets
callers detect failure with an out-of-range check.
Document the public failure contract as a value greater than or equal
to the bitmap size, without requiring callers to depend on the exact
sentinel.
Signed-off-by: Yury Norov <ynorov@nvidia.com>
bitmap_find_next_zero_area() uses an out-of-range return value to
indicate failure. Check for values greater than or equal to the bitmap
size so the caller does not depend on the exact failure sentinel.
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
bitmap_find_next_zero_area() uses an out-of-range return value to
indicate failure. Accept only offsets strictly below the bitmap size so
the callers do not depend on the exact failure sentinel.
Signed-off-by: Yury Norov <ynorov@nvidia.com>
bitmap_find_next_zero_area() uses an out-of-range return value to
indicate failure. Check for values greater than or equal to the bitmap
size so the caller does not depend on the exact failure sentinel.
Acked-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
bitmap_find_next_zero_area() uses an out-of-range return value to
indicate failure. Check for values greater than or equal to the bitmap
size so the caller does not depend on the exact failure sentinel.
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
__nodemask_pr_numnodes() currently returns MAX_NUMNODES as the field
width for '%*pb[l]' nodemask printing. MAX_NUMNODES is a compile-time
upper bound and can be much larger than the runtime node id range,
resulting in excessive zero padding in bitmap-form output.
For example, /proc/<pid>/status prints Mems_allowed with '%*pb' using
the nodemask_pr_args() helper. On systems built with MAX_NUMNODES=1024
but booted with a much smaller possible-node range, this produces:
Mems_allowed: 00000000,00000000,...,00000003
Switch to nr_node_ids, matching the behavior of cpumask_pr_args() which
uses nr_cpu_ids. This reduces the output width from MAX_NUMNODES bits
to the runtime node id range:
Mems_allowed: 3
Visible impact on in-tree users:
- Bitmap format ('%*pb') users:
* /proc/<pid>/status Mems_allowed (format changes as shown above)
- List format ('%*pbl') users, output is unchanged, as list formatter
only prints set bit ranges:
* /sys/devices/system/node/{possible,online,has_normal_memory, ...}
* NVMe multipath sysfs numa_nodes
* memory tier sysfs nodelist
* cpuset cgroup mems and effective_mems files
* /proc/<pid>/status Mems_allowed_list
* mempolicy strings in /proc/<pid>/numa_maps
* SLUB debugfs output
* Kernel log messages printing nodemasks
Move nr_node_ids and nr_online_nodes declarations earlier in the file
to allow __nodemask_pr_numnodes() to use nr_node_ids.
Cc: Yury Norov <yury.norov@gmail.com>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Erhard reports failure of bitmap tests on powerpc:
test_bitmap: loaded.
test_bitmap: [lib/test_bitmap.c:397] bitmaps contents differ: expected "1,3-4,9", got "1,3-4,9,65-71,73-79,81-87,89-95,97-99"
test_bitmap: parselist('0-2047:128/256'): 912
test_bitmap: scnprintf("%*pbl", '0-32767'): 5977
test_bitmap: test_bitmap_read_perf: 1191082
test_bitmap: test_bitmap_write_perf: 1270153
test_bitmap: failed 1 out of 208655 tests
It happens mainly when CONFIG_INIT_STACK_ALL_PATTERN is set.
Commit 6b5a4b6873 ("bitmap: Add test for out-of-boundary
modifications for scatter & gather") extended the test to
out-of-boundary bits, but those bits were left uninitialised.
Properly initialise the entire result bitmap before the test.
[Yury: minor commit message tweaks]
Reported-by: Erhard Furtner <erhard_f@mailbox.org>
Closes: https://lore.kernel.org/all/ca3547ae-8b79-43a2-a758-23ec980bfd9a@mailbox.org
Fixes: 6b5a4b6873 ("bitmap: Add test for out-of-boundary modifications for scatter & gather")
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
These callbacks are sysfs show paths. Use sysfs_emit() and
cpumask_pr_args() to emit the masks.
This prepares for removing cpumap_print_to_pagebuf().
Link: https://lore.kernel.org/all/akANJ-AT7nHpRMq-@yury/
Acked-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
These callbacks are sysfs show paths.
Use sysfs_emit() and cpumask_pr_args() to emit the masks.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
sdma_get_cpu_to_sde_map() is used by a sysfs show callback.
Use sysfs_emit() and cpumask_pr_args() to emit the mask.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
cpumask_show() is a sysfs show callback. Use sysfs_emit() and
cpumask_pr_args() to emit the mask.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
cpumask_show() is a sysfs show callback. Use sysfs_emit() and
cpumask_pr_args() to emit the mask.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
These callbacks are sysfs show paths. Use sysfs_emit() and
cpumask_pr_args() to emit the masks.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
show_cpus_attr() is a sysfs show callback. Use sysfs_emit() and
cpumask_pr_args() to emit the mask.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
These callbacks are sysfs show paths.
Use sysfs_emit() and cpumask_pr_args() to emit the masks.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
These callbacks are sysfs show paths.
Use sysfs_emit() and cpumask_pr_args() to emit the masks.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
These callbacks are sysfs show paths.
Use sysfs_emit() and cpumask_pr_args() to emit the masks.
This prepares for removing cpumap_print_to_pagebuf().
Signed-off-by: Yury Norov <ynorov@nvidia.com>
The *_bit_le functions use a signed integer for the bit number.
However, the *_bit functions can use an unsigned long. This causes
problems if there is a large bitmap and a bit number > 0x80000000 is
passed in. Since that is a negative int, it will get sign extended to a
long when getting passed to the *_bit function, turning it into a huge
bit number. This usually ends up with the memory address wrapping around
and the function accessing memory before the start of the bitmap.
Avoid this by making the *_bit_le functions take an unsigned long.
This can be triggered by faking an almost 4TB dm-mirror device, which
uses bitmaps to track the mirror regions:
$ dmsetup create bigzero --table '0 8589934590 zero'
$ dmsetup create mymirror --table '0 8589934590 mirror core 2 2 nosync 2 /dev/mapper/bigzero 0 /dev/mapper/bigzero 0'
This will access memory before the start of the sync_bits bitmap, and
likely hit the guard page of the previously allocated clean_bits bitmap,
causing a kernel panic with the old code.
I looked and didn't see any crazy code using the signed int to
intentionally try and access bits before some address within the bitmap.
Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Finding a contiguous free region in a highly fragmented
bitmap is not easy and may require many repeated attempts.
Therefore, find_next_bit(map, end, index) is not the optimal choice.
This is because there may be multiple scattered free regions
within the range [index, end) and none of them will meet the length
requirement of @nr.
Instead, it's sufficient to directly find the last bit within
the range [index, end), thus reducing unnecessary repeated calls.
An example of a bitmap:
Bits 0-3: cleared(4 bits)
Bits 4-5: set (2 bits)
Bits 6-8: cleared(4 bits)
Bits 9-10: set (2 bits)
Bits 11-20: cleared(10 bits)
The goal is to find a 10-bit free region.
The old code logic is as follows:
find_next_zero_bit(start = 0, find bit 0) -> find_next_bit(find bit 4) ->
next loop ->
find_next_zero_bit(start = 5, find bit 6) -> find_next_bit(find bit 9) ->
next loop ->
find_next_zero_bit(start = 10, find bit 11) -> success
The new code logic is as follows:
find_next_zero_bit(start = 0, find bit 0) -> find_last_bit(find bit 9) ->
next loop ->
find_next_zero_bit(start = 10, find bit 11) -> success
Performance test results on my hardware(use lib/find_bit_benchmark.c):
before after change p-value
dense 1211 688 -43.2% 8.3e-11
sparse 13.3 13.4 0.8% 0.27
Yury:
The less micro-benchmark kselftest/dmabuf-heaps/dmabuf-heap gives
even better numbers:
Metric Before After Change
Trace span 194.0 ms 87.1 ms -55.1%
Total CMA alloc time 48.46 ms 16.11 ms -66.8%
Avg alloc latency 184.94 us 61.49 us -66.8%
Median alloc latency 73.72 us 20.59 us -72.1%
p90 alloc latency 329.76 us 55.63 us -83.1%
p99 alloc latency 1866.76 us 859.83 us -53.9%
Max alloc latency 4821.91 us 2324.41 us -51.8%
By request size:
Request Before Avg After Avg Change
1 page 79.68 us 34.47 us -56.7%
256 pages 285.50 us 87.30 us -69.4%
Co-developed-by: Yury Norov <yury.norov@gmail.com>
Signed-off-by: Yury Norov <yury.norov@gmail.com>
Signed-off-by: sunyi <279644543@qq.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Switch to pr_info("... %pbl"), and drop the temporary buffer allocation.
This prepares for removing cpumap_print_to_pagebuf().
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
While the GCC and Clang compilers already define __ASSEMBLER__
automatically when compiling assembly code, __ASSEMBLY__ is a
macro that only gets defined by the Makefiles in the kernel.
This can be very confusing when switching between userspace
and kernelspace coding, or when dealing with uapi headers that
rather should use __ASSEMBLER__ instead. So let's standardize now
on the __ASSEMBLER__ macro that is provided by the compilers.
This is a completely mechanical patch (done with a simple "sed -i"
statement).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
scmi_driver_register() requests protocol devices before registering the
driver. If driver_register() fails, those requests remain in the global
IDR and retain pointers to the module's ID table. Once the failed module
load releases that storage, later request matching or SCMI device creation
can dereference the stale pointers.
Unrequest the complete protocol table before returning the registration
failure. At this point table registration succeeded, so every entry is
owned by the current registration attempt.
Fixes: d3cd7c525f ("firmware: arm_scmi: Refactor protocol device creation")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://patch.msgid.link/20260722173521.2184378-2-sudeep.holla@kernel.org
Signed-off-by: Sudeep Holla <sudeep.holla@kernel.org>
scmi_protocol_table_register() can leave earlier requests registered when
a later entry in the same ID table fails. Each request retains a pointer
to the driver's ID table, so a failed module load can leave a dangling
pointer after the module storage is released.
Unrequest only the successfully registered prefix, in reverse order,
before returning the failure. Leave the failed entry and the remaining
entries untouched because matching requests can be owned by another
driver.
Fixes: 2858f6e5f0 ("firmware: arm_scmi: Add multiple protocols registration support")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://patch.msgid.link/20260722173521.2184378-1-sudeep.holla@kernel.org
Signed-off-by: Sudeep Holla <sudeep.holla@kernel.org>
Implement the "ccm(aes)" crypto_aead algorithm using the corresponding
library functions.
Among other benefits, this allows the architecture-optimized AES-CCM
code to be migrated into the library while still leaving it accessible
via crypto_aead, eliminating lots of boilerplate code.
For now the cra_priority is set to just 110, since the
architecture-optimized implementations of this algorithm haven't yet
been migrated into the library. It will be boosted once that happens.
Link: https://patch.msgid.link/20260715221153.246410-14-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Implement the "gcm(aes)" and "rfc4106(gcm(aes))" crypto_aead algorithms
using the corresponding library functions.
Among other benefits, this allows the architecture-optimized AES-GCM
code to be migrated into the library while still leaving it accessible
via crypto_aead, eliminating lots of boilerplate code.
For now the cra_priority is set to just 110, since the
architecture-optimized implementations of these algorithms haven't yet
been migrated into the library. It will be boosted once that happens.
Link: https://patch.msgid.link/20260715221153.246410-13-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Implement the "xts(aes)" crypto_skcipher algorithm using the
corresponding library functions.
Among other benefits, this allows the architecture-optimized AES-XTS
code to be migrated into the library while still leaving it accessible
via crypto_skcipher, eliminating lots of boilerplate code.
Fast paths similar to what x86_64 uses (to eliminate the scatterlist
walking overhead) are included. So we'll get that optimization for all
architectures.
For now the cra_priority is set to just 110, since the
architecture-optimized implementations of this algorithm haven't yet
been migrated into the library. It will be boosted once that happens.
Link: https://patch.msgid.link/20260715221153.246410-12-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Implement the "ctr(aes)" and "xctr(aes)" crypto_skcipher algorithms
using the corresponding library functions.
Among other benefits, this allows the architecture-optimized AES-CTR and
AES-XCTR code to be migrated into the library while still leaving it
accessible via crypto_skcipher, eliminating lots of boilerplate code.
For now the cra_priority is set to just 110, since the
architecture-optimized implementations of these algorithms haven't yet
been migrated into the library. It will be boosted once that happens.
Link: https://patch.msgid.link/20260715221153.246410-11-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Implement the "cbc(aes)" and "cts(cbc(aes))" crypto_skcipher algorithms
using the corresponding library functions.
Among other benefits, this allows the architecture-optimized AES-CBC and
AES-CBC-CTS code to be migrated into the library while still leaving it
accessible via crypto_skcipher, eliminating lots of boilerplate code.
For now the cra_priority is set to just 110, since the
architecture-optimized implementations of these algorithms haven't yet
been migrated into the library. It will be boosted once that happens.
Link: https://patch.msgid.link/20260715221153.246410-10-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Implement the "ecb(aes)" crypto_skcipher algorithm using the
corresponding library functions.
Among other benefits, this allows the architecture-optimized AES-ECB
code to be migrated into the library while still leaving it accessible
via crypto_skcipher, eliminating lots of boilerplate code.
For now the cra_priority is set to just 110, since the
architecture-optimized implementations of this algorithm haven't yet
been migrated into the library. It will be boosted once that happens.
Link: https://patch.msgid.link/20260715221153.246410-9-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Add support for AES-CCM to the crypto library.
This will be used to provide a streamlined implementation of the
"ccm(aes)" crypto_aead algorithm. Most users of "ccm(aes)" will also be
able to switch to the library, which as usual will be faster and
simpler, e.g.:
- fs/smb/client/
- fs/smb/server/
- net/mac80211/
- net/mac802154/
(I've already written proof-of-concept patches for all the above, and
they helped inform the API design.)
As in the AES-GCM API, incremental operation is supported. It has to be
used carefully, especially when decrypting, but it makes the API general
enough to work well for all users.
The AES-CCM library code calls aes_cbcmac_blocks() directly, bypassing
the higher-level aes_cbcmac_init(), aes_cbcmac_update(), and
aes_cbcmac_final(). The latter set of functions is useful only for
AES-CCM, so they don't make sense to keep around and will be removed
once the "ccm(aes)" crypto_aead starts using the AES-CCM library.
Initial test coverage is provided by the crypto_aead support added in a
later commit. I'm planning a KUnit test suite as well.
Link: https://patch.msgid.link/20260715221153.246410-8-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Add support for AES-GCM to the crypto library.
This will be used to provide streamlined implementations of the
"gcm(aes)" and "rfc4106(gcm(aes))" crypto_aead algorithms. Most users
of these will also be able to switch to the library, which as usual will
be faster and simpler, e.g.:
- drivers/net/macsec.c
- fs/smb/client/
- fs/smb/server/
- net/ceph/messenger_v2.c
- net/mac80211/ (for both GMAC and GCMP)
- net/tipc/crypto.c
- security/keys/trusted-keys/trusted_dcp.c
(I've already written proof-of-concept patches for all the above, and
they helped inform the API design.)
As usual, the architecture-optimized AES-GCM code will be migrated into
the library as well (using the hooks provided in this commit as well as
the GHASH ones), eliminating lots of repetitive boilerplate code.
Incremental en/decryption is supported. Incremental operation is a bit
controversial in AEAD APIs because users have to be careful not to
consume any decrypted data that hasn't been authenticated yet. But I do
think it's the right choice here. It's not fundamentally different from
the existing incremental MAC APIs, and it's the only approach that's
general enough to work well for all users in the kernel:
- An array of virtually-addressed buffers (like that used by
BoringSSL's EVP_AEAD_CTX_sealv() and EVP_AEAD_CTX_openv()) doesn't
work in the kernel in general, since in some cases the data for a
single AES-GCM message is contained in a large number of highmem
pages that each need to be mapped into memory individually. That
can be done efficiently only by using CPU-local mappings, but there
is a limited number of those.
Ceph messenger v2 is a great example, as it can send or receive up
to 32 MiB in a single AES-GCM message. And it needs the
en/decrypted data to go into a (potentially large) number of bvecs
provided by a custom iterator, as well as into four
virtually-addressed buffers, two of which can be large buffers in
the vmalloc region.
Even just allocating an array big enough to store all the pointers
can be problematic in the kernel. There are cases in which
decryption runs in GFP_NOIO context or even in softirq context,
where memory allocations are not as reliable as they normally are.
- Meanwhile, 'struct scatterlist' (the choice of crypto_aead) has
turned out to be really inconvenient for anyone who *does* just have
virtually-addressed buffers. This is especially true if they can be
in the vmalloc region, including the stack, as in that case the
conversion to a scatterlist has to be done page-by-page.
And even for users who have all of their data in bare 'struct page',
none of them actually use 'struct scatterlist' as their native data
structure anyway. They actually use skbs, bvecs, or other formats.
- iov_iter is attractive, but ultimately not general enough either
(considering the Ceph case for example), but also too general in
some ways (like having support for userspace addresses). Additional
iter types like ITER_SKB would help a bit, but bloating iov_iter
with more types would reduce performance elsewhere in the kernel.
Initial test coverage is provided by the crypto_aead support added in a
later commit. I'm planning a KUnit test suite as well.
Link: https://patch.msgid.link/20260715221153.246410-7-ebiggers@kernel.org
Link: https://patch.msgid.link/20260722021730.16897-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Commit b0473dcd4b ("smp: Improve smp_call_function_single()
CSD-lock diagnostics") changed smp_call_function_single() so that,
when CSD lock debugging is enabled, async !wait calls use the
destination CPU csd_data. That improves diagnostics, but it also removes
the single-writer property that made the old csd_lock() safe: multiple
CPUs can now prepare the same destination CPU CSD concurrently.
csd_lock() currently waits for CSD_FLAG_LOCK to clear and then sets the
bit with a non-atomic read-modify-write. Two senders can both see an
unlocked CSD, set the bit, overwrite the callback fields, and enqueue
the same llist node. Re-adding a node that is already the queue head can
make node->next point to itself, leaving the target CPU stuck walking
call_single_queue. Later synchronous work, such as a TLB shootdown, can
then remain queued and trigger soft-lockup warnings or panics.
Keep the single csd_lock() implementation, but when CSD lock debugging is
enabled, acquire CSD_FLAG_LOCK with try_cmpxchg_acquire(). This makes the
destination CPU CSD a real atomic lock in the only configuration where it
can be shared by multiple remote senders, while preserving the existing
non-debug fast path.
Fixes: b0473dcd4b ("smp: Improve smp_call_function_single() CSD-lock diagnostics")
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260716004539.13983-2-paulmck@kernel.org
Commit b0473dcd4b ("smp: Improve smp_call_function_single()
CSD-lock diagnostics") made smp_call_function_single() use the destination
CPU's csd_data when CSD lock debugging is enabled. That lets the debug code
associate a stuck CSD lock with the target CPU, but it also means the CPU
argument is used in per_cpu_ptr() before generic_exec_single() has a chance
to validate it.
This becomes unsafe when smp_call_function_any() cannot find an online CPU
in the supplied mask. In that case the selected CPU can be nr_cpu_ids, and
the !wait path calls get_single_csd_data(cpu) before generic_exec_single()
returns -ENXIO. With csdlock_debug_enabled set, that indexes the per-CPU
offset array with an invalid CPU number.
Use the destination CPU's csd_data only when the CPU number is within
nr_cpu_ids. For invalid CPU numbers, fall back to the local CPU's csd_data
and let generic_exec_single() perform the existing validation and return
-ENXIO.
Fixes: b0473dcd4b ("smp: Improve smp_call_function_single() CSD-lock diagnostics")
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Muchun Song <muchun.song@linux.dev>
Link: https://patch.msgid.link/20260716004539.13983-1-paulmck@kernel.org
byd_disconnect() uses timer_delete() before freeing the driver's private
data. This does not wait for a running byd_clear_touch() callback, which
dereferences the private data and its psmouse pointer. A callback racing
with disconnect can therefore access the private data after it has been
freed. The timer can also still be re-armed by byd_process_byte() while
the disconnect is in progress.
Use timer_shutdown_sync() before freeing the private data: it waits for
a running callback and turns any later re-arm attempt into a no-op.
Fixes: 2d5f5611dd ("Input: byd - enable absolute mode")
Cc: stable@vger.kernel.org
Signed-off-by: Linmao Li <lilinmao@kylinos.cn>
Link: https://patch.msgid.link/20260720061259.1601281-1-lilinmao@kylinos.cn
Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
cpufreq_cpu_get() returns NULL when no cpufreq policy is associated with
the requested CPU, for example because the CPU is offline or the policy
has already been torn down. Both amd_pstate_power_supply_notifier() and
amd_pstate_profile_set() acquire a policy via cpufreq_cpu_get() and then
pass that pointer to amd_pstate_get_balanced_epp() and
amd_pstate_set_epp(), which dereference it unconditionally. A racing
CPU hotplug or driver teardown can therefore lead to a NULL pointer
dereference on either of these dynamic EPP paths.
The third cpufreq_cpu_get() caller in this file, amd_pstate_verify(),
already handles the NULL case. Bring the two new callers in line with
that pattern: return NOTIFY_OK from the power-supply notifier (matching
the other "nothing to do" exits) and -ENODEV from amd_pstate_profile_set()
(the usual cpufreq error for a missing CPU policy).
Found by code inspection; not tested on hardware.
Fixes: e30ca6dd53 ("cpufreq/amd-pstate: Add dynamic energy performance preference")
Fixes: 798c47593c ("cpufreq/amd-pstate: Add support for platform profile class")
Signed-off-by: EDAMAMEX <edame8080@gmail.com>
Link: https://lore.kernel.org/r/20260520070211.2753183-1-edame8080@gmail.com
Signed-off-by: Mario Limonciello <superm1@kernel.org>
At CPU EPP initialization, the private cpudata structure is allocated via
kzalloc, which means cpudata->cppc_req_cached is initialized to 0. This
makes the default cached EPP value 0 (AMD_CPPC_EPP_PERFORMANCE).
When initializing a system that defaults to performance EPP, the driver
attempts to configure the EPP via amd_pstate_set_epp(). Because the
requested EPP (0) matches the uninitialized cached value (0), the cache
guard check triggers, and the driver skips writing to the hardware.
Cache the firmware-programmed default EPP value in cppc_req_cached during
CPU EPP initialization. This saves on an unnecessary reprogramming later
when the EPP is first set.
Assisted-by: Antigravity:gemini-3.5-flash
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Marco Scardovi <scardracs@disroot.org>
Link: https://lore.kernel.org/r/20260609073042.81275-4-scardracs@disroot.org
Signed-off-by: Mario Limonciello <superm1@kernel.org>
On shared memory systems, the EPP configuration path (handled via
cppc_set_epp_perf()) is responsible for toggling on the CPPC autonomous
selection register (auto_sel).
Currently, shmem_init_perf() returns early without doing any of the auto_sel
configuration steps if cppc_state is AMD_PSTATE_ACTIVE. This skips enabling
auto_sel, leaving the CPU in non-autonomous mode.
Remove the early return check in shmem_init_perf() when cppc_state is
AMD_PSTATE_ACTIVE. Toggling auto_sel is necessary for the active mode on
shared memory systems to function based on the ACPI spec for CPPC v2 and
below.
Fixes: 2dd6d0ebf7 ("cpufreq: amd-pstate: Add guided autonomous mode")
Assisted-by: Antigravity:gemini-3.5-flash
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Marco Scardovi <scardracs@disroot.org>
Reviewed-by: K Prateek Nayak <kprateek.anayk@amd.com>
Link: https://lore.kernel.org/r/20260609073042.81275-3-scardracs@disroot.org
Signed-off-by: Mario Limonciello <superm1@kernel.org>
Currently, the EPP getter helper functions (msr_get_epp, shmem_get_epp, and
the static call wrapper amd_pstate_get_epp) return u8 or s16. This makes it
difficult to correctly propagate negative error values returned by the
underlying MSR read or CPPC helpers (such as rdmsrq_on_cpu or
cppc_get_epp_perf).
Modify the return type of these functions to int, allowing them to return
negative error codes properly.
Additionally, in amd_pstate_epp_cpu_init(), fetch the firmware-programmed
default EPP value and validate it before assigning it to the EPP variables.
If amd_pstate_get_epp() returns an error code, propagate the error and abort
the CPU initialization to prevent subsequent configuration failures.
Fixes: 555bbe67a6 ("cpufreq/amd-pstate: Convert all perf values to u8")
Assisted-by: Antigravity:gemini-3.5-flash
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Marco Scardovi <scardracs@disroot.org>
Reviewed-by: K Prateek Nayak <kprateek.anayk@amd.com>
Link: https://lore.kernel.org/r/20260609073042.81275-2-scardracs@disroot.org
Signed-off-by: Mario Limonciello <superm1@kernel.org>
