Use cpu_relax() to wait for the execution status SPE_EX_STATE_EXECUTED.
Drop the comments while at it.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260720231153.116827-2-thorsten.blum@linux.dev
fsnotify_pre_content() builds its file_range on the triggering task's
stack. fanotify_alloc_perm_event() saves a pointer to range.pos in the
heap-allocated permission event so copy_range_info_to_user() can report
the offset later.
The event reader can set the event state to FAN_EVENT_REPORTED and then
sleep while preparing the file descriptor. If a signal interrupts the
triggering task at that point, fanotify_get_response() changes the state
to FAN_EVENT_CANCELED and returns. This unwinds the file_range stack
frame while the reader still owns the event. The reader then dereferences
pevent->ppos and copies the stale stack value to userspace.
KASAN reported:
BUG: KASAN: use-after-free in fanotify_read+0x293e/0x2970
Read of size 8 at addr ffff88811434fc50 by task fanotify_inotif/95
Call Trace:
fanotify_read+0x293e/0x2970
vfs_read+0x177/0xa20
ksys_read+0xf7/0x1c0
do_syscall_64+0xf9/0x540
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Store the range position directly in the permission event and use
FANOTIFY_NO_RANGE when range information is unavailable. The event remains
alive until the reader finishes, so the reported offset no longer depends
on the triggering task's stack.
Fixes: 870499bc1d ("fanotify: report file range info with pre-content events")
Cc: stable@vger.kernel.org
Suggested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Chengfeng Ye <nicoyip.dev@gmail.com>
Link: https://patch.msgid.link/20260730134316.2085087-1-nicoyip.dev@gmail.com
Signed-off-by: Jan Kara <jack@suse.cz>
Check only the first byte instead of scanning the entire string with
strlen(). While at it, keep dt_cpu_name static, but move it into
dt_cpu_ftrs_scan_callback(), where it is assigned.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260701114428.818748-3-thorsten.blum@linux.dev
Now that struct rtas_error_log uses a flexible array member for the
extended log buffer, use struct_size() to calculate the total RTAS error
log size and avoid using the hard-coded header size of 8 bytes.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260627104730.276858-4-thorsten.blum@linux.dev
Now that struct rtas_error_log uses a flexible array member for the
extended log buffer, use struct_size() to calculate the total RTAS error
log size and avoid using the hard-coded header size of 8 bytes.
Use min() to replace the open-coded implementation while at it.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260627104730.276858-3-thorsten.blum@linux.dev
On s390, CPUs can be in a state where it is not possible to hotplug
them online before certain prequisite steps. For example the CPUs which
get introduced during runtime of a system can posses a "deconfigured"
state which prevents them from being hotplugged online before they get
configured. Another case is when users set the configured state of CPUs
themselves via "chcpu" or sysfs attributes.
On s390 available CPUs are being registered as new devices via
smp_add_core() either during boot or after a CPU rescan (for newly added
CPUs during runtime). Registered CPUs are marked as enabled without
considering the configure states. Add necessary checks to smp_add_core()
and userspace configure attribute handler. Reflect the configured CPUs
to cpu_enabled_mask to correctly represent which CPUs can be hotplugged
online.
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add SCLP Action Qualifiers used by the Spyre stack for reporting of the
card's initialization status, recoverable errors, and telemetry data.
Co-developed-by: Andreas Krebbel <krebbel@linux.ibm.com>
Signed-off-by: Andreas Krebbel <krebbel@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The queue depth field is defined as a 5 bit field in the Z architecture
instead of a 4 bit field.
The queue depth (qd) can be in range 0-31 and is reported in bits 59-63
of the TAPQ response. Currently this has no effect as all CEX generations
report a queue depth of 7, which fits into 4 bits. However, future CEX
generations reporting a value >15 would not be properly reflected by the
ap bus and therefore all user space applications relying on it.
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Finn Callies <fcallies@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The helper function _ip_cprb_helper() uses internal buffer memory for
building and processing CPRBs. After use this buffer was never
scrubbed which could lead to leaving for example clear key material in
memory which could be exposed via tricky reuse of this same memory.
Extend the _ip_cprb_helper() function with another parameter 'scrub'
used to steer scrubbing of this buffer. So now the caller has the
opportunity to decide if scrubbing is needed or not.
Extend the clear key to secure key token import process in function
cca_clr2cipherkey() to tell the helper function from above to scrub
the cprb buffer when the clear key value is part of the request data.
Add explicit scrubbing on return from function cca_clr2cipherkey() for
the random EXOR buffer and the cprb buffer.
Overall this cleans the internal used buffer in case of clear key
import to prevent sensitive data to get exposed.
Fixes: 4bc123b18c ("s390/zcrypt: Add low level functions for CCA AES cipher keys")
Cc: stable@vger.kernel.org
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Drop the local 10-byte buffer. The old code copied at most 9 bytes from
the user buffer, but only the first byte was used to select the RTAS
operation.
Use get_user() to read the command byte instead and compare it directly
with '0' and '1'. Drop the explicit user buffer check, since get_user()
will fail on a NULL pointer and correctly return -EFAULT instead of
-EINVAL. Remove the now-obsolete string constants as well as any
strncmp() and strlen() calls.
Return the original count instead of a potentially capped value, since
the full user write has been consumed once the command is accepted.
Use unsigned int op to better match the manage_flash() interface.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260528201226.1599977-3-thorsten.blum@linux.dev
The domain value is extracted from a given CCA or EP11 ioctl struct
when a CPRB is about to be sent. Thus this is a user controlled value.
Under some special conditions (custom device node used, administrative
load) this value is used as an array index after bounds checking, but
without speculation barrier.
Add the missing array_index_nospec() call to prevent speculative
execution where this domain value is used.
Fixes: cfd68b3309 ("s390/zcrypt: Filter admin CPRBs on custom devices")
Cc: stable@vger.kernel.org
Reported-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Finn Callies <fcallies@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
There is a wrong upper limit check for the domain value when an EP11
CPRB is processed for sending to a crypto card. This check is only
active on custom device nodes but may lead to access heap memory
behind perms->adm when an administrative CPRB is sent.
Add correct limit (AP_DOMAINS = 256) checking to fix this.
Fixes: cfd68b3309 ("s390/zcrypt: Filter admin CPRBs on custom devices")
Cc: stable@vger.kernel.org
Reviewed-by: Finn Callies <fcallies@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add validation of both the actual key buffer size and token length
fields in all the cca_check_sec*token() functions. Additionally check
in cca_gencipherkey() for possible underflow with returned key size.
The CCA token structures contain user-controlled len fields that
were used in operations without proper validation against both the
actual buffer size and minimum token structure size. An attacker
could set this field larger than the actual buffer size, leading to
reading beyond buffer boundaries. This may result in a kernel crash or
exposure of memory via sending this as part of a request down to the
crypto card. Also an attacker could have used a very small len value
and thus enforce a buffer under-run which may produce similar effects
as a over-read.
So now a key must
- key buf length must be at least sizeof the token struct
- the key len field inside the token must fit into the range of
sizeof key token struct ... key buf length
Fixes: 4bc123b18c ("s390/zcrypt: Add low level functions for CCA AES cipher keys")
Cc: stable@vger.kernel.org
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
cca_ecc2protkey() derives the copy length for the CPRB parameter
block directly from the length field in the key token. Reject the
request early if the token length exceeds the available space in the
parameter block.
Fixes: fa6999e326 ("s390/pkey: support CCA and EP11 secure ECC private keys")
Signed-off-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 5.10+
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
cca_cipher2protkey() derives the copy length for the CPRB parameter
block directly from the length field in the key token. Reject the
request early if the token length exceeds the available space in the
parameter block.
Fixes: 4bc123b18c ("s390/zcrypt: Add low level functions for CCA AES cipher keys")
Signed-off-by: Holger Dengler <dengler@linux.ibm.com>
Cc: stable@vger.kernel.org # 5.4+
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
On a machine without PCI memory-I/O (MIO) support or when running with
pci=nomio the s390 specific PCI MMIO write syscall checks if the MMIO
cookie is above ZPCI_IOMAP_ADDR_BASE as a sanity check before even
trying to perform the MMIO. If this check fails the return value was
left unchanged and thus 0 from prior operations falsely indicating
success. This could potentially confuse user-space into falsely
believing the MMIO, on a mapping not valid for MMIO was successful.
Fix this by setting the return value to -EFAULT prior to the check
following the same pattern as elsewhere in the same function.
Cc: stable@vger.kernel.org
Reviewed-by: Julian Ruess <julianr@linux.ibm.com>
Reviewed-by: Farhan Ali <alifm@linux.ibm.com>
Fixes: a67a88b0b8 ("s390/pci: remove races against pte updates")
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Drop the redundant buffer assignment in serial_edit_cmdline() since the
cp pointer is immediately overwritten.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260528082357.1397611-3-thorsten.blum@linux.dev
Replace the manual string length accounting, memcpy(), and NUL
termination with a single snprintf() call to prepend the elfcorehdr=
address and to detect string truncation at the same time.
Use kmalloc() to avoid unnecessarily zeroing the memory. While at it,
also use "prepending" instead of "appending" in the error message.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260527000105.1081651-3-thorsten.blum@linux.dev
Now that all kvm entry to guest paths handle the task work
using the generic framework, enable HAVE_POSIX_CPU_TIMERS_TASK_WORK
which allows running posix cpu timers in task context instead of running
them in hardirq. This would is a necessary step towards enabling
PREEMPT_RT on powerNV systems.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260709092140.1753715-5-vishalc@linux.ibm.com
Since commit 2cd571245b ("sched/fair: Add related data structure for
task based throttle") in v6.18, CFS bandwidth throttling no longer
dequeues a task directly; it queues task_work via TWA_RESUME and sets
TIF_NOTIFY_RESUME, relying on that work running before the task returns
to guest/user mode. The powerpc KVM run loops only checked for reschedule
and signals, never TIF_NOTIFY_RESUME, so the deferred throttle never ran
while a vCPU stayed in the run loop: a CPU-bound guest that rarely exits
to userspace ran far past its cpu.max quota and then appeared frozen for
minutes while the accrued throttle debt was repaid.
Use the generic infrastructure to check for and handle pending work
before transitioning into guest mode, replacing the open-coded
need_resched() and cond_resched() checks in the Book3S HV run loops and
in the common kvmppc_prepare_to_enter() used by the Book3S PR and BookE
run loops. The redundant signal_pending() recheck (and its sigpend label)
in kvmhv_run_single_vcpu() is also dropped, as
xfer_to_guest_mode_work_pending() is a superset of it.
This picks up handling for TIF_NOTIFY_RESUME, which was previously
ignored, meaning task work will now be correctly handled on every
guest re-entry.
Selecting VIRT_XFER_TO_GUEST_WORK disables RCU's last-resort self-IPI
fallback for vCPU tasks (see rcu_irq_work_resched()), which on
nohz_full CPUs was what forced a reschedule for deferred rcuog wakeups
queued right before guest entry. Take over that obligation the same way
x86 and s390 do: call xfer_to_guest_mode_prepare() with IRQs disabled
immediately before the final xfer_to_guest_mode_work_pending() check at
each guest-entry gate (kvmhv_run_single_vcpu(), kvmppc_run_core() and
kvmppc_prepare_to_enter()).
In kvmppc_prepare_to_enter(), IRQs are now disabled with
local_irq_disable() before hard_irq_disable(): on 32-bit,
hard_irq_disable() is a raw MSR[EE] clear that bypasses the
lockdep/irq-tracing state, and the strict xfer_to_guest_mode helpers
assert that IRQs are seen as disabled. This also allows upgrading the
racy __xfer_to_guest_mode_work_pending() check to the asserting
variant, as this loop is the terminal gate for the PR and BookE paths.
In kvmhv_run_single_vcpu(), the -EINTR exit and the pre-existing
kvmhv_setup_mmu() failure exit now leave via the done label instead of
returning directly, keeping the run_vcpu enter/exit tracepoints
balanced and vcpu->arch.ret consistent with the returned value.
In kvmppc_prepare_to_enter() the generic helper accounts the signal exit
(vcpu->stat.signal_exits and KVM_EXIT_INTR) but does not set the exit
type, so kvmppc_set_exit_type(SIGNAL_EXITS) is retained on the signal
path to preserve the E500 CONFIG_KVM_EXIT_TIMING histogram; it is a no-op
otherwise.
Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260709092140.1753715-4-vishalc@linux.ibm.com
The read buffer allocation was changed from kmalloc() to kvmalloc().
This buffer is part of mei_cl_cb structure that can be queued in
rd_complete queue protected by spinlock.
Releasing the structure leads to errors like below when freeing buffer
that allocated non-contiguous:
BUG: sleeping function called from invalid context at mm/vmalloc.c:3448
Separate mei_cl_cb structure dequeue and release to
perform only dequeue under spinlock and push release out of spinlock.
Cc: stable <stable@kernel.org>
Fixes: 4adf613e01 ("mei: use kvmalloc for read buffer")
Closes: https://gitlab.freedesktop.org/drm/i915/kernel/-/work_items/16359
Reviewed-by: Menachem Adin <menachem.adin@intel.com>
Signed-off-by: Alexander Usyskin <alexander.usyskin@intel.com>
Link: https://patch.msgid.link/20260719-kvfree_out_of_spinlock-v1-1-e07d6333bea7@intel.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
For now only Big Endian 32-bit PowerPC is supported, as that is the only
hardware I have. This has been tested on the Nintendo Wii so far, but I
plan on also using it on the GameCube, Wii U and Apple G4.
These changes aren’t the only ones required to get the kernel to compile
and link on PowerPC, libcore will also have to be changed to not use
integer division to format u64, u128 and core::time::Duration, otherwise
__udivdi3() and __umoddi3() will have to be added. I have tested this
change by replacing the three implementations with unimplemented!() and
it linked just fine.
Signed-off-by: Link Mauve <linkmauve@linkmauve.fr>
Link: https://github.com/Rust-for-Linux/linux/issues/105
Link: https://github.com/linuxppc/issues/issues/451
Acked-by: Gary Guo <gary@garyguo.net>
Link: https://github.com/rust-lang/compiler-team/issues/986
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-7-mkchauras@gmail.com
The core crate currently depends on these two functions for i64/u64/
i128/u128/core::time::Duration formatting, but we shouldn’t use that in
the kernel so let’s panic if they are ever called.
This doesn’t yet fix drm_panic_qr.rs, which also uses __udivdi3 when
CONFIG_CC_OPTIMIZE_FOR_SIZE=y, but at least makes the rest of the kernel
build on PPC32.
Signed-off-by: Link Mauve <linkmauve@linkmauve.fr>
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-6-mkchauras@gmail.com
Added support for a new macro ARCH_STATIC_BRANCH_ASM in powerpc
to avoid duplication of inline asm between C and Rust. This is
inline with 'commit aecaf18165 ("jump_label: adjust inline asm to be consistent")'
Co-developed-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://github.com/Rust-for-Linux/linux/issues/105
Link: https://github.com/linuxppc/issues/issues/451
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-4-mkchauras@gmail.com
When building with LLVM=1 for architectures like powerpc where
CONFIG_DMA_SHARED_BUFFER is not enabled, the build fails with:
ld.lld: error: undefined symbol: dma_resv_reset_max_fences
>>> referenced by helpers.c
>>> rust/helpers/helpers.o:(rust_helper_dma_resv_unlock)
The issue occurs because:
1. CONFIG_DEBUG_MUTEXES=y is enabled
2. CONFIG_DMA_SHARED_BUFFER is not enabled
3. dma_resv_reset_max_fences() is declared in the header when
CONFIG_DEBUG_MUTEXES is set
4. But the function is only compiled in drivers/dma-buf/dma-resv.c,
which is only built when CONFIG_DMA_SHARED_BUFFER is enabled
5. Rust helpers call dma_resv_unlock() which calls
dma_resv_reset_max_fences(), causing an undefined symbol
Fix this by compiling `dma-resv.c` file only when CONFIG_DMA_SHARED_BUFFER
is enabled.
Fixes: 9b836641d3 ("rust: helpers: Add bindings/wrappers for dma_resv_lock")
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-3-mkchauras@gmail.com
When building Rust code for powerpc64le with LLVM=1 and -j1, rustc
encounters an error: "multiple candidates for `rmeta` dependency `core`
found", with two candidates:
1. The host's standard library from the rustup toolchain
2. The kernel's custom libcore.rmeta in the rust/ directory
This occurs because the build system uses `-L$(objtree)/rust` for host
library builds (proc_macro2, quote, syn), which causes rustc to search
the rust/ directory. During this search, rustc finds both the kernel's
custom libcore.rmeta and gains access to the host's standard library,
creating a conflict.
The solution is to separate host libraries into a dedicated rust/host/
subdirectory and use `-L$(objtree)/rust/host` for host builds instead
of `-L$(objtree)/rust`. This ensures that:
1. Host library builds (proc_macro2, quote, syn) only search rust/host/
and never encounter the kernel's libcore.rmeta
2. Proc macro builds use `-L$(objtree)/rust/host` to find their
dependencies
Special handling is added for rustdoc-pin_init, which is a host build
(to access the alloc crate) but depends on proc macros from the main
rust/ directory. It uses explicit `--extern` paths to reference the
proc macros without adding `-L$(objtree)/rust`, which would reintroduce
the conflict.
The rust/host/ directory is added to clean-files to ensure it's removed
during `make clean`.
Link: https://github.com/Rust-for-Linux/linux/issues/105
Link: https://github.com/linuxppc/issues/issues/451
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-2-mkchauras@gmail.com
Simplify the code to manage the firmware loading with auto-cleanup.
By the use of __free(firmware), we can replace the manual mutex locks
with guard() gracefully, too.
Only the code refactoring, no functional changes.
Cc: patches@opensource.cirrus.com
Reviewed-by: Richard Fitzgerald <rf@opensource.cirrus.com>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260729083735.120219-7-tiwai@suse.de
Simplify the code to manage the firmware loading with auto-cleanup
with __free(firmware). A NULL clear is added at
cs35l41_request_firmware_file() for avoiding the double-free.
Note that the driver still keeps a few manual firmware releases
because it retries with different firmware files when one of firmware
pairs fails.
Only the code refactoring, no functional changes.
Cc: patches@opensource.cirrus.com
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260729083735.120219-6-tiwai@suse.de
Clean up the code for managing the firmware loading in the 6fire
driver with __free(firmware) and __free(kfree), so that the loaded
firmware and the name string are cleaned up automatically.
Only the code refactoring, no functional changes.
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260729083735.120219-2-tiwai@suse.de
No caller checks the return value from stop_one_cpu_nowait(). All
callers require the callback to run and arrange for the target CPU's
stopper to remain enabled while queuing the work. In particular, commit
f0498d2a54 ("sched: Fix stop_one_cpu_nowait() vs hotplug") added
preemption protection to the scheduler callers so that queuing must
succeed once the target CPU has been observed online.
Therefore, a failure is an unrecoverable violation rather than a condition
individual callers can recover from. Diagnose it with WARN_ON_ONCE() in
stop_one_cpu_nowait(). A check in the common helper covers current and
future callers consistently, while individual checks would duplicate
the same non-recoverable handling at every call site.
Make the function return void because there is no longer a meaningful
result for callers to consume.
On UP, warn if the supplied CPU is not the current CPU because the work
cannot be scheduled in that case.
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Reviewed-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20260729022355.325058-1-ynorov@nvidia.com
Joanne Koong <joannelkoong@gmail.com> says:
iomap: convert to in-iter iomap_next() model
This series implements a suggestion by Christoph for finishing the conversion
of iomap to an iterator model. This revives Matthew's previous RFC [1], which
had the same intention.
Every iomap operation currently drives its iteration through a struct
iomap_ops, which contains two callbacks, ->iomap_begin() and ->iomap_end().
iomap_iter() only ever sees these as pointers, so every step of every
iteration is an indirect call, including on the hottest paths.
This series replaces the begin/end pair with a single ->iomap_next() callback
that finishes the previous mapping (if any) and produces the next one.
Collapsing to one callback lets a performance-critical caller inline its
iteration loop and pass its ->iomap_next() function as a compile-time
constant, where the compiler can devirtualize the callback into a direct and
inlineable call rather than an indirect one. It also allows future callers
more flexibility in expressing custom logic in the IO path for driving the
iteration forward.
This series has no functional changes intended. The patches are broken down as
follows:
1) Patch 1: Brian's fix for folio batch release on iomap callback failures.
The bug was reported by Sashiko and is an unlikely/second order error
scenario [2] that doesn't need backporting to stable.
2) Patch 2: refactors existing iomap_iter() logic into an iomap_iter_next()
function. Sets up DEFINE_IOMAP_ITER_NEXT/DEFINE_IOMAP_ITER_NEXT_END macro.
3) Patch 3 and 4: Christoph's patches for decoupling simple direct i/o reads
from iomap_dio_rw and improvement for using GFP_NOWAIT for
non-blocking iocbs [3]
4) Patch 5: Adds ->iomap_next() callback as an iomap op
5) Patches 6 to 19: converts each filesystem to ->iomap_next() model
[1] https://lore.kernel.org/linux-fsdevel/20200728173216.7184-1-willy@infradead.org/T/#u
[2] https://lore.kernel.org/linux-fsdevel/amjztG-DisHYbV9W@bfoster/
[3] https://lore.kernel.org/linux-fsdevel/20260723050201.3381045-1-hch@lst.de/
* patches from https://patch.msgid.link/20260729192737.3190206-1-joannelkoong@gmail.com:
exfat: convert iomap ops to ->iomap_next()
fuse: convert iomap ops to ->iomap_next()
hpfs: convert iomap ops to ->iomap_next()
gfs2: convert iomap ops to ->iomap_next()
f2fs: convert iomap ops to ->iomap_next()
block: convert iomap ops to ->iomap_next()
ext2: convert iomap ops to ->iomap_next()
zonefs: convert iomap ops to ->iomap_next()
erofs: convert iomap ops to ->iomap_next()
ext4: convert iomap ops to ->iomap_next()
ntfs: convert iomap ops to ->iomap_next()
ntfs3: convert iomap ops to ->iomap_next()
btrfs: convert iomap ops to ->iomap_next()
xfs: convert iomap ops to ->iomap_next()
iomap: add ->iomap_next()
iomap: use GFP_NOWAIT when application for iomap_dio_simple allocations
iomap: decouple simple direct I/O reads from iomap_dio_rw
iomap: split iomap_iter() logic into iomap_iter_next()
iomap: release the folio batch on iomap callback failures
Link: https://patch.msgid.link/20260729192737.3190206-1-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Convert exfat iomap_ops to the new ->iomap_next() callback. Each
callback is generated with the DEFINE_IOMAP_ITER_NEXT() and
DEFINE_IOMAP_ITER_NEXT_END() macros, which wrap the iomap_iter_next()
helper to finish the previous mapping if needed and produce the next
one. No functional changes are intended.
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-20-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Convert fuse iomap_ops to the new ->iomap_next() callback. Each callback is
generated with the DEFINE_IOMAP_ITER_NEXT()/DEFINE_IOMAP_ITER_NEXT_END()
macros, which wrap the iomap_iter_next() helper to finish the previous
mapping if needed and produce the next one. No functional changes are
intended.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-19-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Convert hpfs iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-18-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
