max_threads is initialized once by fork_init()->set_max_threads(), and
thereafter is mostly read in hot path, such as copy_process().
The fact that it is mostly read and not written to makes it candidates for
__read_mostly declarations.
I have already tested on my machine(arm64,256core,kernel-7.1.3) with
'hackbench -P -l 500 -g 100/500/1000/2000' and the results show that there
is an average of 13% improvement in performance.
before patch:
hackbench -g 100 500 1000 2000
1.094 4.651 13.305 36.322
after patch:
hackbench -g 100 500 1000 2000
0.823 4.487 11.107 32.987
Above data are the average values obtained from multiple tests, and there
was indeed some fluctuation in the data during the tests.
Link: https://lore.kernel.org/20260707092354.465847-1-jansef.jian@hj-micro.com
Signed-off-by: JianChunfu <jansef.jian@hj-micro.com>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Kees Cook <kees@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
This reverts commit 8901ac9d2c.
The commit 94bfc7f3b0 ("err.h: use __always_inline on all error pointer
helpers") fixes the problem with a dead code elimination by always
inlining IS_ERR() function. It prevents the related linker failures as
well.
The workaround with disabling branch profiling is not longer necessary.
Link: https://lore.kernel.org/20260707135522.120447-1-pmladek@suse.com
Signed-off-by: Petr Mladek <pmladek@suse.com>
Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Closes: https://lore.kernel.org/r/ah6WDkwO8eYY5f2a@ashevche-desk.local
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Aleksander Lobakin <aleksander.lobakin@intel.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Sergey Senozhatsky <senozhatsky@chromium.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
ocfs2_validate_xattr_block() checks the xattr block header before the
block reaches higher-level xattr users, but it does not verify that a
non-indexed block's xh_count and entry offsets fit inside the block.
Indexed buckets likewise reach list/get consumers after ECC without an
entry-bounds check.
Use the flat xattr entry validator for non-indexed external xattr blocks,
and use a bucket-specific validator for indexed buckets at metadata read
time. The bucket validator keeps the entry array bounded by the first
bucket block while checking name/value offsets against the bucket block
they target.
Reject corrupted external xattr metadata before listxattr() or getxattr()
can walk out-of-range entry arrays or name/value offsets.
Validation reproduced this kernel report:
BUG: KASAN: use-after-free in ocfs2_xattr_list_entries+0xd7/0x190
Read of size 1 at addr ffff88810a654007 by task ocfs2_xattr_lis/630
Call Trace:
dump_stack_lvl+0x66/0xa0
print_report+0xce/0x630
kasan_report+0xe0/0x110
ocfs2_xattr_list_entries+0xd7/0x190
ocfs2_listxattr+0x3f6/0x610
listxattr+0x90/0xe0
path_listxattrat+0xed/0x220
do_syscall_64+0x115/0x6a0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Link: https://lore.kernel.org/20260705025311.3429854-3-zzzccc427@gmail.com
Fixes: cf1d6c763f ("ocfs2: Add extended attribute support")
Fixes: 0c044f0b24 ("ocfs2: Add xattr bucket iteration for large numbers of EAs")
Signed-off-by: Cen Zhang <zzzccc427@gmail.com>
Assisted-by: Codex:gpt-5.5
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "ocfs2: validate xattr entry bounds", v7.
This series validates OCFS2 xattr entry name/value bounds when xattr
metadata is read and validated, before getxattr() or listxattr() can walk
out-of-range entry arrays or offsets from corrupted metadata.
This patch (of 2):
ocfs2_validate_inode_block() verifies a dinode before OCFS2 users walk
metadata from it, but inline xattr metadata is still checked only in
operation-specific consumers. The existing ibody lookup helper validates
inline header placement and entry count, but inode block validation does
not reject entry name/value bounds.
Add a flat xattr entry validator and call it from inode block validation
for inline xattrs. Keep the operation paths on their existing
header/count lookup checks; the full entry bounds check now runs when the
inode block is validated at read time.
Reject corrupted inline xattr metadata before ocfs2_xattr_ibody_get() or
listxattr() can walk past the inline storage.
Validation reproduced this kernel report:
BUG: KASAN: use-after-free in ocfs2_xattr_find_entry+0x5a/0x170
Read of size 2 at addr ffff8881242a2000 by task python3/529
Call Trace:
dump_stack_lvl+0x66/0xa0
print_report+0xce/0x630
kasan_report+0xe0/0x110
ocfs2_xattr_find_entry+0x5a/0x170
ocfs2_xattr_get_nolock+0x20a/0x820
ocfs2_xattr_get+0x10c/0x1e0
__vfs_getxattr+0xe2/0x130
vfs_getxattr+0x185/0x1b0
Link: https://lore.kernel.org/20260705025311.3429854-1-zzzccc427@gmail.com
Link: https://lore.kernel.org/20260705025311.3429854-2-zzzccc427@gmail.com
Fixes: cf1d6c763f ("ocfs2: Add extended attribute support")
Signed-off-by: Cen Zhang <zzzccc427@gmail.com>
Assisted-by: Codex:gpt-5.5
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Heming Zhao <heming.zhao@suse.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Mark Fasheh <mark@fasheh.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
The capture_clear field is never set, and all code conditional on it being
set has been unreachable since the platform data logic was removed from
pps-gpio in ee89646619 ("pps: clients: gpio: Get rid of legacy platform
data").
I think the only logical thing to do here is to remove it all, since no
in-tree code ever actually used it in the first place, and it has been
completely dead code for over five years (since v5.13).
Sashiko asked some questions about the gpiod_get_value() call which caused
me to look deeper and figure this out, but it did not actually notice
capture_clear is never set.
Link: https://lore.kernel.org/f70196bafcf75d9782dd36ed784e42345b6e8a1b.1783355507.git.calvin@wbinvd.org
Fixes: ee89646619 ("pps: clients: gpio: Get rid of legacy platform data")
Signed-off-by: Calvin Owens <calvin@wbinvd.org>
Closes: https://sashiko.dev/#/patchset/cover.1779733602.git.calvin%40wbinvd.org?part=1
Acked-by: Rodolfo Giometti <giometti@enneenne.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
If userspace holds its file descriptor open, it can call PPS_KC_BIND on a
device which has been unplugged, leaving pps_kc_hardpps_dev as a dangling
pointer after close().
After that sequence, PPS_KC_BIND is broken until the system is rebooted,
because the pointer comparison in pps_kc_bind() can never be true.
calling pps_ktimer_init+0x0/0x1000 [pps_ktimer] @ 1081
initcall pps_ktimer_init+0x0/0x1000 [pps_ktimer] returned 0 after 811 usecs
pps pps0: bound kernel consumer: edge=0x1
pps pps0: unbound kernel consumer on device removal
pps pps0: bound kernel consumer: edge=0x1
calling pps_ktimer_init+0x0/0x1000 [pps_ktimer] @ 1085
initcall pps_ktimer_init+0x0/0x1000 [pps_ktimer] returned 0 after 340 usecs
pps pps0: another kernel consumer is already bound
Here is a short reproducer, which uses rmmod of the pps-ktimer testcase
to simulate a device being unplugged:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/pps.h>
#include <errno.h>
#include <err.h>
int main(void)
{
while (1) {
int fd;
if (system("insmod ./pps-ktimer.ko"))
err(1, "insmod failed");
fd = open("/dev/pps0", O_RDWR);
if (fd == -1)
err(1, "open failed");
struct pps_bind_args args = {
.tsformat = PPS_TSFMT_TSPEC,
.edge = PPS_CAPTUREASSERT,
.consumer = PPS_KC_HARDPPS,
};
if (ioctl(fd, PPS_KC_BIND, &args))
err(1, "first PPS_KC_BIND failed");
if (system("rmmod pps-ktimer"))
err(1, "rmmod failed");
if (ioctl(fd, PPS_KC_BIND, &args)) {
if (errno != ENODEV)
err(1, "second PPS_KC_BIND failed");
else
puts("Got ENODEV, kernel is patched");
}
close(fd);
}
}
Fix this by setting a flag when the device is unplugged, returning -ENODEV
from PPS_KC_BIND if the flag is set.
For userspace to encounter this new behavior, it must do something which
breaks the interface today, so this fix shouldn't cause any observable
behavior change for working programs.
Link: https://lore.kernel.org/672778c177ac9b6fdcb445e35c97ac4ca7d1149f.1780506611.git.calvin@wbinvd.org
Signed-off-by: Calvin Owens <calvin@wbinvd.org>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/cover.1779733602.git.calvin%40wbinvd.org?part=1
Acked-by: Rodolfo Giometti <giometti@enneenne.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
If userspace passes a negative timeout to PPS_FETCH, it triggers a kernel
splat from schedule_timeout():
schedule_timeout: wrong timeout value fffffffffff0bfb4
CPU: 17 UID: 0 PID: 4720 Comm: a.out Not tainted 7.1.0-rc5-x86-kvm-00150-g331d97e36b37 #1 PREEMPT_RT
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-20240910_120124-localhost 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x4b/0x70
schedule_timeout+0xb7/0xe0
pps_cdev_pps_fetch.isra.0+0x93/0x150
pps_cdev_ioctl+0x70/0x310
__x64_sys_ioctl+0x7b/0xc0
do_syscall_64+0xb6/0xfc0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
Here is a trivial reproducer that works with the PPS_CLIENT_KTIMER test
device enabled in the kernel:
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/pps.h>
#include <err.h>
int main() {
struct pps_fdata fdata;
int fd;
fd = open("/dev/pps0", O_RDWR);
if (fd == -1)
err(1, "Failed to open /dev/pps0");
fdata.timeout.sec = -1;
fdata.timeout.nsec = 0;
if (ioctl(fd, PPS_FETCH, &fdata))
err(2, "PPS_FETCH failed");
close(fd);
return 0;
}
Sashiko imagines this to be some sort of security problem, which is
obviously really silly. But I think it is still worth fixing, so buggy
userspace code can't trigger the splat.
Silence the splat by using timespec64_to_jiffies(), which hard limits the
timeout to LONG_MAX jiffies. To be safe, explicitly preserve the
-ETIMEDOUT return value userspace sees today if it passes a negative
timeout.
If you really squint, this is still a slight behavior change in that there
are "denormalized" combinations of tv_sec and tv_nsec which used to work
but will now return -ETIMEDOUT. I can't imagine anybody will care about
that...
Link: https://lore.kernel.org/c5c97c3b3c9d66010382094fd538e59a38f4aacf.1781289959.git.calvin@wbinvd.org
Fixes: eae9d2ba0c ("LinuxPPS: core support")
Signed-off-by: Calvin Owens <calvin@wbinvd.org>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/cover.1779733602.git.calvin%40wbinvd.org?part=3
Acked-by: Rodolfo Giometti <giometti@enneenne.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Convert the existing prandom selftest (lib/random32.c) to use the KUnit
framework (lib/tests/random32_kunit.c). Unlike typical KUnit tests, this
file is directly #included into lib/random32.c.
The new test:
- Removes the legacy CONFIG_RANDOM32_SELFTEST from lib/random32.c.
- Adds CONFIG_PRANDOM_KUNIT_TEST (defaulting to KUNIT_ALL_TESTS).
- Moves the test logic to lib/tests/random32_kunit.c.
This commit is verified by `./tools/testing/kunit/kunit.py run`
with the .kunit/.kunitconfig:
CONFIG_KUNIT=y
CONFIG_PRANDOM_KUNIT_TEST=y
Link: https://lore.kernel.org/20260703050100.23944-1-note351@hotmail.com
Signed-off-by: Kir Chou <note351@hotmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David S. Miller <davem@davemloft.net>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Jakub Kacinski <kuba@kernel.org>
Cc: Kuan-Wei Chiu <visitorckw@gmail.com>
Cc: Paolo Abeni <pabeni@redhat.com>
Cc: Simon Horman <horms@kernel.org>
Cc: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Split the pps-gpio interrupt handler into a primary (hardirq) handler that
captures the PPS timestamp at interrupt entry, and a threaded handler that
processes the event. This produces the same two-part handler structure on
both PREEMPT_RT and non-RT kernels.
On non-RT kernels the threaded portion runs immediately after the primary,
with no behavioral change compared to the previous single-handler
implementation.
On PREEMPT_RT, where interrupt handlers are force-threaded by default, the
previous single-handler implementation captured the timestamp inside the
threaded portion, after IRQ-thread scheduling delay. With the split, the
timestamp is captured in true hardirq context as it is on non-RT kernels,
eliminating a significant source of PPS jitter on RT systems.
Link: https://lore.kernel.org/2e32729029fbf6977ecf04665eb00f2efd3e2c17.1780359378.git.calvin@wbinvd.org
Signed-off-by: Michael Byczkowski <by@by-online.de>
Signed-off-by: Calvin Owens <calvin@wbinvd.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Tested-by: Michael Byczkowski <by@by-online.de>
Tested-by: Calvin Owens <calvin@wbinvd.org>
Acked-by: Rodolfo Giometti <giometti@enneenne.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
A crafted OCFS2 image with corrupted orphan-directory extent metadata can
make umount hang.
During unmount, ocfs2_recovery_disable() waits for the
ocfs2_complete_recovery work item to finish. The worker scans the orphan
directory through ocfs2_queue_orphans() and ocfs2_dir_foreach(). If
ocfs2_read_dir_block() fails on a corrupted directory block,
ocfs2_dir_foreach_blk_el() skips the block and continues walking. On a
badly corrupted directory this can keep orphan recovery busy for a long
time, leaving umount blocked while flushing osb->ocfs2_wq.
Return the read error immediately for full directory scans and propagate
the error from ocfs2_dir_foreach(). When ocfs2_empty_dir() receives such
an error, report the directory as non-empty so unlink/rmdir does not
proceed on an unreadable directory.
Link: https://lore.kernel.org/20260702090507.446517-1-r772577952@gmail.com
Closes: https://lore.kernel.org/lkml/CANypQFbWH76Y6LWHEwAvTP7aQL04uMJ=dDyL6YDmxa3fv3Tyjg@mail.gmail.com/
Assisted-by: Codex:gpt-5.5-xhigh
Signed-off-by: Jiaming Zhang <r772577952@gmail.com>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
A node receiving a DLM_MIG_LOCKRES message trusts several fields of the
peer-supplied dlm_migratable_lockres without validation. num_locks and
lockname_len are bounded only on the sending side, and the message is
never checked to actually carry num_locks migratable_lock entries. As a
result dlm_process_recovery_data() walks mres->ml[0..num_locks) past the
kmalloc(data_len) copy of the message (an out-of-bounds read that ends in
a BUG_ON panic), and dlm_init_lockres() copies lockname_len bytes into the
fixed 32-byte o2dlm_lockname slab object (a heap out-of-bounds write).
Both are reachable by any node in the domain.
Validate these fields right after dlm_grab(), before anything uses them --
including the not-joined error path, which already prints mres->lockname
with the unbounded lockname_len as a %.*s precision. Reject the message
unless lockname_len <= DLM_LOCKID_NAME_MAX, num_locks <=
DLM_MAX_MIGRATABLE_LOCKS (the bound the sender already asserts), and the
payload is large enough to hold the claimed locks. Conforming recovery
and migration messages are unaffected.
Link: https://lore.kernel.org/20260629-b4-disp-94fb6521-v1-2-6953bcc0421f@proton.me
Fixes: 6714d8e86b ("[PATCH] OCFS2: The Second Oracle Cluster Filesystem")
Signed-off-by: Bryam Vargas <hexlabsecurity@proton.me>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "ocfs2/dlm: bound peer-controlled lengths in the o2dlm".
The o2dlm receive handlers trust u8 length and count fields from the wire
without bounding them, so a node in a DLM domain can corrupt or panic any
other node with a malformed message. Three defects:
- dlm_migrate_request_handler() passes migrate->namelen unchecked to
dlm_init_mle(), which memcpy()s it into the 32-byte mname[] of an
o2dlm_mle slab object: a heap out-of-bounds write of up to ~215
attacker-controlled bytes.
- dlm_mig_lockres_handler() passes mres->lockname_len unchecked to
dlm_init_lockres(), which memcpy()s it into the 32-byte o2dlm_lockname
slab object: a heap out-of-bounds write of up to ~223 bytes.
- the same handler trusts mres->num_locks without checking that the
message is large enough to hold that many entries, so
dlm_process_recovery_data() walks mres->ml[] past the kmalloc(data_len)
copy and trips a BUG_ON (an out-of-bounds read ending in a panic).
The other o2dlm receive handlers already reject an oversized name; the
migration and recovery handlers have omitted it since the DLM was added
(see the Fixes tags). Patch 1 bounds namelen; patch 2 validates
lockname_len, num_locks, and the payload size. Conforming recovery and
migration traffic is unaffected.
o2net authenticates peers only by the DLM domain key, so any node that has
joined the domain -- including a compromised or malicious member -- can
send these messages. There is no local trigger; the attacker must already
be a member of the cluster.
Each sink was confirmed under KASAN with an out-of-tree module mirroring
it exactly -- a kmem_cache/kmalloc of the real destination size, then the
same unclamped memcpy/loop: slab-out-of-bounds Write for the two writes,
Read for the recovery walk, and a panic. A userspace AddressSanitizer
build faults identically under -m32 and -m64. Scrubbed logs are available
on request.
I reported this privately to security@kernel.org and the ocfs2 maintainers
on 2026-06-20; with no response after the standard embargo period I am
posting the fix publicly. I have no embargo requirement.
This patch (of 2):
A node receiving a DLM_MIGRATE_REQUEST message trusts the peer-supplied
name length (migrate->namelen) without bounding it. dlm_init_mle() then
copies that many bytes into the fixed DLM_LOCKID_NAME_MAX-byte mname[]
array of an o2dlm_mle slab object, so a malformed message from a cluster
peer overflows the slab object by up to ~215 bytes: a heap out-of-bounds
write of attacker-controlled data, reachable by any node in the domain.
Reject an oversized name, the way dlm_master_request_handler() and the
other o2dlm receive handlers already do; the migration handler omits the
check entirely. Conforming messages are unaffected.
Link: https://lore.kernel.org/20260629-b4-disp-94fb6521-v1-0-6953bcc0421f@proton.me
Link: https://lore.kernel.org/20260629-b4-disp-94fb6521-v1-1-6953bcc0421f@proton.me
Fixes: 6714d8e86b ("[PATCH] OCFS2: The Second Oracle Cluster Filesystem")
Signed-off-by: Bryam Vargas <hexlabsecurity@proton.me>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Heming Zhao <heming.zhao@suse.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
glibc 2.42 added __attribute_const__ to sys/cdefs.h:
# define __attribute_const__ __attribute__ ((__const__))
GCC 15 warns when a macro is redefined to a different replacement list
(-Wbuiltin-macro-redefined). Since host tool Makefiles (resolve_btfids,
objtool) pass -Werror, this conflict becomes fatal.
The warning is suppressed on standard native builds because GCC treats
/usr/include as a system header path (-isystem), and macro-redefinition
warnings from system headers are silently suppressed by GCC. It fires
when glibc headers are on a regular include path (-I) instead, which
is the case in cross-compilation setups such as NixOS, where the
sysroot's glibc is passed explicitly via -I rather than -isystem.
Per (C11 6.10.3), identical replacement lists are accepted silently.
Match the glibc definition exactly, including the space before "((", so
the redefinition is accepted without warning regardless of whether
glibc headers are treated as system or non-system includes.
Link: https://lore.kernel.org/20260701200635.3992767-1-rkr0k0r@gmail.com
Signed-off-by: Joy H.J. Lee <rkr0k0r@gmail.com>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
In has_pending_signals there was a switch/case used for optimizations.
However, today's compilers perform loop unrolling efficiently, thus it is
not needed anymore.
Put i inside the for declaration so we do not risk its escape from the
scope. Moreover, i starts now from 0 and counts up, as it is a more usual
pattern.
Link: https://lore.kernel.org/20260520062849.183621-2-andrea.calabrese@amarulasolutions.com
Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Cc: Adrian Huang <adrianhuang0701@gmail.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Marco Elver <elver@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
fat_ent_write() may have committed the new link to the primary FAT but
then failed on the mirror copy, leaving the chain pointing to new_dclus
even though the caller will free it. Restore the original value to keep
the chain consistent.
Link: https://lore.kernel.org/20260525085649.781643-1-n05ec@lzu.edu.cn
Fixes: 1da177e4c3 ("Linux-2.6.12-rc2")
Signed-off-by: Yemu Lu <prcups@krgm.moe>
Signed-off-by: Ren Wei <n05ec@lzu.edu.cn>
Reported-by: Yuan Tan <yuantan098@gmail.com>
Reported-by: Yifan Wu <yifanwucs@gmail.com>
Reported-by: Juefei Pu <tomapufckgml@gmail.com>
Reported-by: Xin Liu <bird@lzu.edu.cn>
Acked-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Cc: Christian Brauner <brauner@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Add a KUnit test suite for the polynomial_calc() function, which had no
in-kernel test coverage. The tests verify correct evaluation of constant,
linear, quadratic, and cubic polynomials, including negative coefficients,
negative input data, zero-coefficient terms.
The Kconfig entry uses 'select POLYNOMIAL' rather than 'depends on
POLYNOMIAL' because POLYNOMIAL is a promptless tristate that cannot
be manually enabled on UML without an explicit selector.
Link: https://lore.kernel.org/20260606030319.316752-1-adinata.softwareengineer@gmail.com
Signed-off-by: Adi Nata <adinata.softwareengineer@gmail.com>
Cc: Maxim Kaurkin <maxim.kaurkin@baikalelectronics.ru>
Cc: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Cc: Guenter Roeck <linux@roeck-us.net>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Gow <david@davidgow.net>
Cc: Rae Moar <raemoar63@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
send_signal_locked() unconditionally copies siginfo before the namespace
translation to avoid corrupting a shared siginfo.
Not that I think this can actually hurt performance-wise, just it doesn't
look clean to me; the copy is only needed in the unlikely case when the
translation will actually change something.
Defer it to the two cases where si_pid/si_uid are rewritten, and while at
it add #ifdef's just for completeness.
Link: https://lore.kernel.org/akkaAgNfUby5_3nM@redhat.com
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Eric Biederman <ebiederm@xmission.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
prepare_kill_siginfo(PIDTYPE_TGID) fills si_code = SI_USER and sets
si_pid/si_uid in the sender's namespace. Then send_signal_locked()
translates si_pid/si_uid to the target's namespace.
SEND_SIG_NOINFO exists precisely for the case when si_code == SI_USER
and si_pid/si_uid are the sender's ids; this is exactly what sys_kill()
does via prepare_kill_siginfo(PIDTYPE_TGID). Change sys_kill() to use
it directly.
SEND_SIG_NOINFO produces the same result: si_code = SI_USER, and
__send_signal_locked() computes si_pid/si_uid directly in the target's
namespace. The force computation is also the same: both check if the
sender is visible in the target's pid namespace.
This is just a cleanup and microoptimization (especially with [1]),
this skips the has_si_pid_and_uid() block in send_signal_locked() and
offloads the namespace translation logic to
__send_signal_locked(SEND_SIG_NOINFO) which uses the simpler
computations.
NOTE: As a "side effect" this also fixes the kill(pid < 0, sig) case
where send_signal_locked() rewrites si_pid/si_uid in the shared
siginfo, corrupting it for subsequent recipients. But for other group
senders like __kill_pgrp_info() we still need the fix from Bradley
Morgan [1] who found this problem.
TODO: kill prepare_kill_siginfo() and change other users to use
SEND_SIG_NOINFO too. This needs trivial changes in
__send_signal_locked() and TP_STORE_SIGINFO().
Link: https://lore.kernel.org/aj6btAZqYuv59a8w@redhat.com
Link: https://lore.kernel.org/all/20260622164029.11474-1-include@grrlz.net/ [1]
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Cc: Eric Biederman <ebiederm@xmission.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
send_signal_locked() rewrites sender ids for the target namespace. Group
sends reuse the same siginfo, so one recipient can affect the next.
Copy the siginfo before changing it.
Link: https://lore.kernel.org/86a8857d58d43ee26a8b365b837fd24830343494.1782159692.git.include@grrlz.net
Fixes: 7a0cf09494 ("signal: Correct namespace fixups of si_pid and si_uid")
Signed-off-by: Bradley Morgan <include@grrlz.net>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Adrian Huang <adrianhuang0701@gmail.com>
Cc: Aleksandr Nogikh <nogikh@google.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Marco Elver <elver@google.com>
Cc: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Fix misspellings of "softlockup" in the watchdog enabled bit definitions
and related comments. Also fix a nearby "successful" typo.
No functional change.
Link: https://lore.kernel.org/20260615174557.1836562-1-edcr1790@gmail.com
Signed-off-by: Matthew Chen <edcr1790@gmail.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Inside parse_one(), the core parameter-parsing engine prints the address
of the parameter-set callback function using %p: pr_debug("handling %s
with %p\n", param, params[i].ops->set);
Since the string value of the parameter being parsed (val) is already
available, print the parameter name and its value instead, and avoid
tainting the kernel by exposing a kernel-ptr.
Link: https://lore.kernel.org/20260618190715.3563047-1-jim.cromie@gmail.com
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
memchr_inv() takes a size_t length but counts 8 byte words in an unsigned
int. At 32GiB that count wraps, so the scan can quietly miss most of the
range.
Use size_t for the word count.
Link: https://lore.kernel.org/20260621121133.16460-1-include@grrlz.net
Fixes: 798248206b ("lib/string.c: introduce memchr_inv()")
Signed-off-by: Bradley Morgan <include@grrlz.net>
Cc: Akinbou Mita <akinobu.mita@gmail.com>
Cc: Andy Shevchenko <andy@kernel.org>
Cc: Christoph Lameer <cl@linux-foundation.org>
Cc: Joern Engel <joern@logfs.org>
Cc: Kees Cook <kees@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
PREEMPT_RT's rtmutex PI chain walker warns about a lock dependency cycle
when inode_lock(orphan_dir_inode) is called while holding
inode_lock(file_inode):
ocfs2_file_write_iter()
inode_lock(file_inode) [class 0]
ocfs2_dio_end_io_write()
ocfs2_del_inode_from_orphan()
inode_lock(orphan_dir_inode) [class 0] <- warning!
However this is a false positive. write_iter() is never called on a
directory, and orphan_dir is always a directory, so these two locks can
never actually conflict in practice.
Fix by using inode_lock_nested(orphan_dir_inode, I_MUTEX_NONDIR2) in all
three places where orphan_dir_inode is locked in namei.c, placing it in a
separate lock class so the rtmutex PI chain walker understands these locks
have distinct roles and does not warn about their ordering.
Link: https://lore.kernel.org/20260620231223.46588-1-kartikey406@gmail.com
Signed-off-by: Deepanshu Kartikey <kartikey406@gmail.com>
Suggested-by: Matthew Wilcox <willy@infradead.org>
Reported-by: syzbot+ce129763ce7d7e914739@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=ce129763ce7d7e914739
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
o2nm_node_local_store() handles local=0 by stopping o2net and setting
cl_local_node to O2NM_INVALID_NODE_NUM, but it leaves cl_has_local set.
That stale state makes o2nm_this_node() return 255, blocks a later local=1
attempt with -EBUSY, and can feed 255 to heartbeat users that call
o2nm_this_node() dynamically.
Clearing cl_has_local is required when the local node is reset. But
heartbeat threads can still be running at that point. They pin the local
node config item at startup, yet o2hb_do_disk_heartbeat() and thread
teardown re-read o2nm_this_node() for the local slot and for
o2nm_undepend_this_node(). Once local=0 has cleared the live local-node
state, those dynamic reads return O2NM_MAX_NODES, which is also the
invalid node number 255.
Store the local node number in the heartbeat region when the region
starts. Use that stable node for heartbeat slot writes/checks,
negotiation messages, and the final configfs undepend. Stop the heartbeat
loop when the current local node no longer matches the stored node, and
clear cl_has_local together with cl_local_node in the local=0 path so
nodemanager state matches node removal.
Validation reproduced this kernel report:
KASAN slab-out-of-bounds in o2hb_do_disk_heartbeat+0x372/0xb30
RIP: 0010:memset+0xf/0x20
Read of size 8
Call trace:
dump_stack_lvl+0x66/0xa0
print_report+0xd0/0x630
o2hb_do_disk_heartbeat+0x372/0xb30 (fs/ocfs2/cluster/heartbeat.c:1079)
srso_alias_return_thunk+0x5/0xfbef5
__virt_addr_valid+0x188/0x2f0
kasan_report+0xe4/0x120
o2hb_do_disk_heartbeat+0x5/0xb30 (fs/ocfs2/cluster/heartbeat.c:1079)
o2hb_thread+0x14e/0x770
kthread_affine_node+0x139/0x180
lockdep_hardirqs_on_prepare+0xda/0x190
trace_hardirqs_on+0x18/0x130
kthread+0x19d/0x1e0
ret_from_fork+0x37a/0x4d0
__switch_to+0x2d5/0x6f0
ret_from_fork_asm+0x1a/0x30
Link: https://lore.kernel.org/20260616074931.3774929-1-zzzccc427@gmail.com
Fixes: a7f6a5fb4b ("[PATCH] OCFS2: The Second Oracle Cluster Filesystem")
Assisted-by: Codex:gpt-5.5
Signed-off-by: Cen Zhang <zzzccc427@gmail.com>
Suggested-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
The "resource sanity check" warning need not mean that there is a
functional issue and in the cases when there isn't one, it is just useless
and confusing noise.
For this reason, downgrade the log level of it from "warn" to "debug".
This reduces log clutter while keeping the diagnostic information
available for debugging purposes if needed.
Link: https://lore.kernel.org/20260523000131.7086-1-sakunix@yahoo.com
Signed-off-by: Manuel Quintero Fonseca <sakunix@yahoo.com>
Cc: Andriy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Hans de Goede <hansg@kernel.org>
Cc: Mika Westeberg <mika.westerberg@linux.intel.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
vhost_scsi_setup_vq_cmds() runs from VHOST_SCSI_SET_ENDPOINT and allocates
each command's protection scatterlist array (prot_sgl) according to the
acknowledged VIRTIO_SCSI_F_T10_PI bit. The command pools are not rebuilt
when VHOST_SET_FEATURES changes that bit later.
Although virtio feature bits must not change after feature negotiation,
vhost_scsi_set_features() currently accepts such a request after the
endpoint is active and updates acked_features. Enabling T10-PI after
endpoint setup therefore leaves prot_sgl NULL while the I/O path follows
the new feature bit.
For a 129-page protection payload, vhost_scsi_mapal() passes the missing
first chunk to sg_alloc_table_chained():
sg_alloc_table_chained(table, 129, first_chunk=NULL,
nents_first_chunk=inline_sg_cnt)
sg_pool_index() then hits:
BUG_ON(nents > SG_CHUNK_SIZE); /* 129 > 128 */
The kernel reported the following call trace and register state:
Call Trace:
<TASK>
? __sg_alloc_table+0x1d8/0x250
? __pfx_vhost_run_work_list+0x10/0x10 [vhost]
sg_alloc_table_chained+0x59/0xf0
? __pfx_sg_pool_alloc+0x10/0x10
? vhost_scsi_calc_sgls.constprop.0+0x43/0x60 [vhost_scsi]
vhost_scsi_handle_vq+0xf02/0x1700 [vhost_scsi]
? __pfx_vhost_scsi_handle_vq+0x10/0x10 [vhost_scsi]
vhost_scsi_handle_kick+0x37/0x50 [vhost_scsi]
vhost_run_work_list+0x8e/0xd0 [vhost]
vhost_task_fn+0xe1/0x210
ret_from_fork+0x348/0x540
</TASK>
RIP: 0010:0x4
CR2 = 0x4
RSP: 0018:ffffc90000dbf940 EFLAGS: 00010202
RAX: ffffffff82396810 RBX: ffff88811dc28b80 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000820 RDI: 0000000000000081
VHOST_F_LOG_ALL is a vhost-specific runtime feature and remains the only
exception.
Reject changes to any feature other than VHOST_F_LOG_ALL while the
endpoint is active. This preserves the existing runtime log toggle while
preventing feature-dependent command resources and data-path state from
becoming inconsistent. Userspace must clear the endpoint before changing
any other negotiated feature and set the endpoint up again afterward.
Fixes: bf2d650391 ("vhost-scsi: Allocate T10 PI structs only when enabled")
Signed-off-by: Jia Jia <physicalmtea@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Message-ID: <20260726144314.1652934-1-physicalmtea@gmail.com>
vhost-scsi translates guest response descriptors into userspace iovecs
when commands are submitted. Target-core completes those commands
asynchronously, so VHOST_SET_MEM_TABLE can replace the memory table while
an in-flight command still retains response iovecs translated through the
old table.
If the old mapping is reused after VHOST_SET_MEM_TABLE returns, command
completion can write the response to an unrelated userspace object.
Flush the vhost-scsi backend after vhost_dev_ioctl() handles a device
ioctl. This waits for in-flight commands that can still use the old
response iovecs before the ioctl returns.
Signed-off-by: Jia Jia <physicalmtea@gmail.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Message-ID: <20260724060919.1569170-1-physicalmtea@gmail.com>
vq->meta_iotlb[] caches the vhost_iotlb_map that backs each vring
metadata region, and iotlb_access_ok() returns early on a cache hit,
taking the hit as proof that the region has already been validated:
if (vhost_vq_meta_fetch(vq, addr, len, type))
return true;
The cache is reset on VHOST_IOTLB_UPDATE and VHOST_IOTLB_INVALIDATE, on
device IOTLB (re)initialisation and on vq reset, but not when
VHOST_SET_VRING_ADDR replaces vq->desc, vq->avail and vq->used, nor when
VHOST_SET_VRING_NUM changes the region sizes.
With a device IOTLB attached both ioctls are accepted while the vq is
live, and neither validates the addresses at ioctl time: vq_access_ok()
and vq_log_used_access_ok() return true early because the addresses are
GIOVAs, deferring validation to prefetch time. Once the cache has been
populated that deferred validation no longer runs -- vq_meta_prefetch()
hits the stale entry and returns true -- and vhost_vq_meta_fetch() keeps
translating through the old mapping as
map->addr + addr - map->start
for an address the mapping no longer covers. vhost_copy_to_user() and
vhost_copy_from_user() consume the result with __copy_to_user() and
__copy_from_user(), which do not check it either, so a subsequent used
ring update or descriptor fetch accesses memory outside the region the
IOTLB actually maps.
Reset the metadata cache whenever the vring is reconfigured, so the new
addresses are pushed back through iotlb_access_ok()'s slow path.
Fixes: f889491380 ("vhost: introduce O(1) vq metadata cache")
Cc: stable@vger.kernel.org
Assisted-by: tencentos-corvus-ai:kimi-k3
Signed-off-by: Jun Yang <junvyyang@tencent.com>
Message-ID: <20260803014823.68623-1-juny24602@gmail.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
When T10 PI is negotiated, vhost-scsi splits protection bytes from
the data iterator before mapping the request scatterlists. A malformed
request can claim protection bytes that cover or exceed the full payload
length. The former leaves no data bytes to map, while the latter
underflows exp_data_len before advancing the iterator. Both cases can let
a zero data SGL count reach sg_alloc_table_chained(), which triggers
BUG_ON(!nents).
Reject protection lengths that cover or exceed the payload before
subtracting prot_bytes and advancing the iterator. Also propagate
negative errors from the protection SGL calculation before calling the
allocator, matching the data SGL path.
Fixes: bca939d5bc ("vhost-scsi: Dynamically allocate scatterlists")
Suggested-by: Jia Jia <physicalmtea@gmail.com>
Signed-off-by: Jia Jia <physicalmtea@gmail.com>
Assisted-by: OpenAI-Codex:GPT-5
Signed-off-by: Linfeng Sun <linfeng.sun.dev@gmail.com>
Message-ID: <20260727081841.923151-1-slf@hdu.edu.cn>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Batch AUTO_BUF_REG COMMIT must unregister the old auto_buf index before
storing the next one. Fixed per-tag indexing (A == B) masks bugs that
clear after overwriting io->buf.
Add kublk --rotate_auto_buf so each tag alternates between two sparse
buffer indices, and test_batch_04.sh to exercise that path. Without the
driver fix, the request ref stays stuck and I/O hangs; the test uses a
short timeout and kills the ublk daemon to recover. With the fix, a
small write completes quickly.
Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20260730024050.1062354-1-yangxiuwei@kylinos.cn
Signed-off-by: Jens Axboe <axboe@kernel.dk>
With UBLK_F_AUTO_BUF_REG, invalid sqe->addr can fail after
ublk_fill_io_cmd() has set UBLK_IO_FLAG_ACTIVE. The uring_cmd is
completed while the tag stays active, which can hang teardown.
Split validation from buffer apply so the check has no side effects,
then take the uring_cmd and store the already-validated buffer. Apply
the same order in FETCH so io->buf is not written before __ublk_fetch()
state checks.
Fixes: 52460dda3a ("ublk: move auto buffer register handling into one dedicated helper")
Suggested-by: Caleb Sander Mateos <csander@purestorage.com>
Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
ublk_map_io() and ublk_unmap_io() are no-ops for ublk devices that
enable user copy or zero copy, as well as for requests without data to
copy in the given direction. However, the implementation is a bit
convoluted, returning the full request data length and relying on the
caller to check the return value against the request length.
UBLK_F_SHMEM_ZC recently added branches to skip the ublk_{,un}map_io()
call for I/Os using a shared-memory buffer. This is a more logical place
for the device need_map and the ublk_need_{,un}map_req() checks, so move
them there from ublk_{,un}map_io().
Checking these conditions early also skips the expensive pointer-chasing
for the ublk_iod_is_shmem_zc() check in __ublk_complete_rq() for the
common case of a ublk device using user copy or zero copy.
Drop the req_op() filter in __ublk_complete_rq(), as it's redundant with
the ublk_need_unmap_req() check.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20260803211441.2538144-9-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add test loop_08, which creates a ublk device with UBLK_F_IO_DESC_SIZE
enabled and io_desc_size set to 64. The test issues verified I/O to the
device using fio.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260803211441.2538144-8-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add an optional --io_desc_size argument to the kublk add/recover
commands to enable UBLK_F_IO_DESC on the ublk device. The mmap()
arguments and ublk_get_iod() computation are adjusted accordingly.
Display the configured io_desc_size in the kublk list output for ublk
devices with UBLK_F_IO_DESC.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260803211441.2538144-7-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
ublk passes the parameters of incoming I/O in memory shared between the
kernel ublk driver and userspace ublk server in struct ublksrv_io_desc.
The size of this struct is currently fixed to 24 bytes, which has been
an obstacle to extending it with additional fields [1]. Additionally,
with multiple ublk server threads handling I/Os from the same ublk queue
(possible with UBLK_F_PER_IO_DAEMON or UBLK_F_BATCH_IO), false sharing
results from adjacent io_descs sharing the same cache line.
Add a ublk feature UBLK_F_IO_DESC_SIZE to allow a ublk server to
override the size of each io_desc. The size must be at least 24 and a
multiple of 8 to store a properly-aligned struct ublksrv_io_desc. It's
also limited to a maximum of 256, though this bound could be lifted in
the future.
The struct ublksrv_io_desc is located at the beginning of each io_desc
and the remainder is padding. The mmap() performed for each queue must
have a length of queue_depth * io_desc_size rounded up to the page size.
The mmap() offset must be q_id * UBLK_MAX_QUEUE_DEPTH * io_desc_size,
also rounded up to the page size.
[1]: https://lore.kernel.org/linux-block/aV8QfvaNO5P6vOs6@fedora/
Suggested-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260803211441.2538144-6-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
ublk_setup_iod() is currently called to populate struct ublksrv_io_desc
on the thread submitting I/O to a ublk device. However, only the ublk
server threads read the io_descs. This basically guarantees a cache miss
on both threads for each ublk I/O. There's really no need to initialize
the io_descs on the submitting thread. Move the ublk_setup_iod() call to
ublk_dispatch_req() (for non-UBLK_F_BATCH_IO) and
__ublk_batch_prep_dispatch() (for UBLK_F_BATCH_IO), which runs on the
ublk server daemon thread before dispatching the I/O to userspace.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20260803211441.2538144-5-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
In preparation for moving the struct ublksrv_io_desc initialization from
the thread submitting ublk requests to the daemon thread receiving them,
split the fallible part of ublk_setup_iod{,_zoned}() into new helper
ublk_validate_req{,_zoned}(). Only ublk_setup_iod{,_zoned}() accesses
the io_desc and cannot error out.
Return a bool value from ublk_validate_req{,_zoned}() as the existing
error code ublk_setup_iod{,_zoned}() returns is only checked against
BLK_STS_OK.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20260803211441.2538144-4-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
struct ublk_zoned_report_desc's operation field is only ever set to
UBLK_IO_OP_REPORT_ZONES, so remove it. Replace its one load with the
constant.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20260803211441.2538144-3-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The u16 nr_hw_queues and queue_depth fields of the ublk UAPI struct
ublksrv_ctrl_dev_info constrain the number of queues and queue depth of
each ublk device. However, the ublk driver is a bit inconsistent with
the type it uses to represent these values, mixing u16 with int and
unsigned int. Change all queue number, queue depth, q_id, and tag
variables/fields to u16 to save some space.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20260803211441.2538144-2-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io->res is set from struct ublksrv_io_cmd's result field, which is
controlled by the ublk server process, without any validation. It's thus
possible for userspace to trigger the io->res > rq_bytes warning.
ublk_copy_user_pages() already limits the copy length to the request
data length, so drop the warning.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260729171041.45061-4-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
If the userspace ublk server passes an unmapped address as the data
buffer for a completed ublk read, ublk_unmap_io() will return 0
indicating no bytes could be copied. Currently, this will result in
calling blk_update_request() with nr_bytes=0, which doesn't seem
supported. Fail the I/O with BLK_STS_IOERR in this case instead.
Fixes: 71f28f3136 ("ublk_drv: add io_uring based userspace block driver")
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260729171041.45061-3-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
import_ubuf() can fail if the address range (provided by the userspace
ublk server) is outside the allowed user address space. Return that 0
bytes were copied if import_ubuf() fails rather than passing an
uninitialized struct iov_iter to ublk_copy_user_pages().
Fixes: 981f95a571 ("ublk: cleanup ublk_copy_user_pages")
Reported-by: Ming Lei <tom.leiming@gmail.com>
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260729171041.45061-2-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
bfqg_prfill_stat_recursive() and bfq_bio_bfqg() read blkg->online
locklessly, same as blkg_rwstat_recursive_sum(). Annotate with
data_race() to silence KCSAN.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Reviewed-by: Yu Kuai <yukuai@fygo.io>
Link: https://patch.msgid.link/20260803134104.138411-1-cui.tao@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dipayaan Roy says:
====================
net: mana: add ethtool private flag for full-page RX buffers
On some ARM64 platforms with 4K PAGE_SIZE, utilizing page_pool
fragments for allocation in the RX refill path (~2kB buffer per
fragment) causes 15-20% throughput regression under high connection
counts (>16 TCP streams at 180+ Gbps). Using full-page buffers on
these platforms shows no regression and restores line-rate
performance.
This behavior is observed on a single platform; other platforms
perform better with page_pool fragments, indicating this is not a
page_pool issue but platform-specific.
This series adds an ethtool private flag "full-page-rx" to let the
user opt in to one RX buffer per page:
ethtool --set-priv-flags eth0 full-page-rx on
There is no behavioral change by default. The flag can be persisted
via udev rule for affected platforms.
====================
Link: https://patch.msgid.link/20260729063347.3388035-1-dipayanroy@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
On some ARM64 platforms with 4K PAGE_SIZE, page_pool fragment
allocation in the RX refill path can cause 15-20% throughput
regression under high connection counts (>16 TCP streams).
Add an ethtool private flag "full-page-rx" that allows the user to
force one RX buffer per page, bypassing the page_pool fragment path.
This restores line-rate (180+ Gbps) performance on affected platforms.
Usage:
ethtool --set-priv-flags eth0 full-page-rx on
There is no behavioral change by default. The flag must be explicitly
enabled by the user or udev rule.
The existing single-buffer-per-page logic for XDP and jumbo frames is
consolidated into a new helper mana_use_single_rxbuf_per_page() which
is now the single decision point for both the automatic and
user-controlled paths.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Link: https://patch.msgid.link/20260729063347.3388035-3-dipayanroy@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Refactor mana_get_strings() and mana_get_sset_count() from if/else to
switch statements in preparation for adding ethtool private flags
support which requires handling ETH_SS_PRIV_FLAGS.
No functional change.
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Link: https://patch.msgid.link/20260729063347.3388035-2-dipayanroy@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
io_vec_fill_bvec() computes the folio size with a plain int 1:
unsigned long folio_size = 1 << imu->folio_shift;
imu->folio_shift is unsigned int and comes from folio_shift() of the
folio backing the registered buffer, so it can be 32 or more on a 64 bit
kernel. Shifting int 1 that far is undefined, and on x86 and arm64 the
count is taken modulo 32, so a shift of 34 yields 4 rather than 16G.
Every other folio_shift shift in this file already uses 1UL.
The result is that the segment estimate and the fill loop disagree.
io_estimate_bvec_size() sizes the bvec array with the real shift:
max_segs += (iov[i].iov_len >> shift) + 2;
so a 1M iovec on a 16G folio is charged 2 segments, while
io_vec_fill_bvec() then walks the same iovec in folio_size chunks of 4
bytes and writes res_bvec[bvec_idx] a quarter of a million times, past
the end of the array it was given. src_bvec is advanced once per
iteration as well, so imu->bvec is read past its end at the same time.
validate_fixed_range() only checks that the range is inside the
registered buffer and does not bound the segment count.
Reaching it needs a folio with a shift of at least 32, which means a
gigantic hugetlb page: 16G on arm64 with 64K pages, where
CONT_PMD_SHIFT is 34 and hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT)
registers that size, and likewise on powerpc. x86_64 tops out at 1G, so
a shift of 30, which still fits in int and is unaffected.
Use 1UL, as the rest of the file does.
Fixes: 9ef4cbbcb4 ("io_uring: add infra for importing vectored reg buffers")
Cc: stable@vger.kernel.org
Signed-off-by: Ali Ahmet Memis <ali@iusegentoo.com>
Link: https://patch.msgid.link/20260802163030.51005-1-ali@iusegentoo.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
An io_async_cmd carries an iovec array in ->vec.iovec, allocated when the
vec has to grow and kept across recycling through ctx->cmd_cache. On two
paths nothing frees it and io_clean_op()'s kfree(req->async_data) drops
the io_async_cmd without it.
io_req_uring_cleanup() clears the async data flags only when
io_alloc_cache_put() succeeds, and the cache holds IO_ALLOC_CACHE_MAX ==
128 entries, so once it is full the put fails and the vec is left behind.
An NVMe passthrough workload gets there without doing anything unusual:
nvme_uring_cmd_io() returns -EIOCBQUEUED, so the io_async_cmd stays
attached for the lifetime of the command and the live object count tracks
the queue depth. Above 128 the puts start failing.
->cleanup is the last chance to free an inherited vec, since
io_req_uring_cleanup() returns early for an io-wq issued command and is
not called at all for one completed without ever being issued. But
io_clean_op() calls ->cleanup only if REQ_F_NEED_CLEANUP is set, and for
uring_cmd that happens only where the vec has to grow, so a command
reusing a large enough cached vec never sets it. io_rw_alloc_async() and
io_msg_alloc_async() flag an inherited vec for exactly this reason;
io_uring_cmd_prep() does not.
Flag an inherited vec in io_uring_cmd_prep(), and free the vec when the
cache put fails, as io_req_rw_cleanup() does.
The leak is invisible under KASAN, where io_alloc_cache_vec_kasan() frees
the vec unconditionally.
Fixes: 3a4689ac10 ("io_uring/cmd: add iovec cache for commands")
Cc: stable@vger.kernel.org
Signed-off-by: Woraphat Khiaodaeng <worapat.kd2@gmail.com>
Link: https://patch.msgid.link/20260802073518.419-1-worapat.kd2@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
