linux/io_uring
Pavel Begunkov c844ace5b8 io_uring: fix overflow resched cqe reordering
[ Upstream commit a7d755ed9c ]

Leaving the CQ critical section in the middle of a overflow flushing
can cause cqe reordering since the cache cq pointers are reset and any
new cqe emitters that might get called in between are not going to be
forced into io_cqe_cache_refill().

Fixes: eac2ca2d68 ("io_uring: check if we need to reschedule during overflow flush")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/90ba817f1a458f091f355f407de1c911d2b93bbf.1747483784.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
History 2025-06-04 14:42:18 +02:00
..
advise.c io_uring: always go async for unsupported fadvise flags 2023-01-29 15:18:26 -07:00
advise.h io_uring: split out fadvise/madvise operations 2022-07-24 18:39:11 -06:00
alloc_cache.h io_uring/rsrc: consolidate node caching 2023-04-12 12:09:41 -06:00
cancel.c io_uring: use fget/fput consistently 2024-01-20 11:51:38 +01:00
cancel.h io_uring/cancel: support opcode based lookup and cancelation 2023-07-17 10:05:48 -06:00
epoll.c io_uring: undeprecate epoll_ctl support 2023-05-26 20:22:41 -06:00
epoll.h io_uring: move epoll handler to its own file 2022-07-24 18:39:11 -06:00
fdinfo.c io_uring/fdinfo: annotate racy sq/cq head/tail reads 2025-06-04 14:42:16 +02:00
fdinfo.h io_uring: move fdinfo helpers to its own file 2022-07-24 18:39:12 -06:00
filetable.c io_uring: drop any code related to SCM_RIGHTS 2024-03-26 18:19:09 -04:00
filetable.h io_uring: add helpers to decode the fixed file file_ptr 2023-06-20 09:36:22 -06:00
fs.c io_uring/fs: consider link->flags when getting path for LINKAT 2023-12-03 07:33:07 +01:00
fs.h io_uring: split out filesystem related operations 2022-07-24 18:39:11 -06:00
io-wq.c io-wq: backoff when retrying worker creation 2025-03-22 12:50:43 -07:00
io-wq.h io_uring: break out of iowq iopoll on teardown 2023-09-07 09:02:27 -06:00
io_uring.c io_uring: fix overflow resched cqe reordering 2025-06-04 14:42:18 +02:00
io_uring.h io_uring/kbuf: use vm_insert_pages() for mmap'ed pbuf ring 2025-03-22 12:50:45 -07:00
kbuf.c io_uring/kbuf: reject zero sized provided buffers 2025-04-25 10:45:26 +02:00
kbuf.h io_uring/kbuf: use vm_insert_pages() for mmap'ed pbuf ring 2025-03-22 12:50:45 -07:00
Makefile io_uring: add zc notification infrastructure 2022-07-24 18:41:06 -06:00
msg_ring.c io_uring: use io_file_from_index in io_msg_grab_file 2023-06-20 09:36:22 -06:00
msg_ring.h io_uring: get rid of double locking 2022-12-07 06:47:13 -07:00
net.c io_uring/net: fix accept multishot handling 2025-04-25 10:45:54 +02:00
net.h io_uring: Add KASAN support for alloc_caches 2023-04-03 07:16:14 -06:00
nop.c io_uring: fail NOP if non-zero op flags is passed in 2024-06-12 11:11:18 +02:00
nop.h io_uring: move nop into its own file 2022-07-24 18:39:11 -06:00
notif.c io_uring/notif: add constant for ubuf_info flags 2023-04-15 14:21:04 -06:00
notif.h io_uring/notif: add constant for ubuf_info flags 2023-04-15 14:21:04 -06:00
opdef.c io_uring: Pass whole sqe to commands 2023-05-04 08:19:05 -06:00
opdef.h io_uring: Split io_issue_def struct 2023-01-29 15:17:41 -07:00
openclose.c io_uring: correct check for O_TMPFILE 2023-08-07 12:34:23 -06:00
openclose.h io_uring: split out fixed file installation and removal 2022-07-24 18:39:16 -06:00
poll.c io_uring/net: don't retry connect operation on EPOLLERR 2025-02-17 09:40:36 +01:00
poll.h io_uring: avoid indirect function calls for the hottest task_work 2023-06-02 08:55:37 -06:00
refs.h io_uring: always do atomic put from iowq 2025-05-02 07:50:57 +02:00
rsrc.c io_uring: unify io_pin_pages() 2025-03-22 12:50:45 -07:00
rsrc.h io_uring: drop any code related to SCM_RIGHTS 2024-03-26 18:19:09 -04:00
rw.c io_uring/rw: commit provided buffer state on async 2025-02-17 09:40:37 +01:00
rw.h io_uring: avoid indirect function calls for the hottest task_work 2023-06-02 08:55:37 -06:00
slist.h io_uring: silence variable ‘prev’ set but not used warning 2023-03-09 10:10:58 -07:00
splice.c io_uring/splice: use fput() directly 2023-08-10 10:24:25 -06:00
splice.h io_uring: split out splice related operations 2022-07-24 18:39:11 -06:00
sqpoll.c io_uring/sqpoll: fix sqpoll error handling races 2025-01-02 10:32:10 +01:00
sqpoll.h io_uring/sqpoll: fix io-wq affinity when IORING_SETUP_SQPOLL is used 2023-08-16 13:40:28 -06:00
statx.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
statx.h io_uring: move statx handling to its own file 2022-07-24 18:39:11 -06:00
sync.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
sync.h io_uring: split out fs related sync/fallocate functions 2022-07-24 18:39:11 -06:00
tctx.c io_uring/tctx: work around xa_store() allocation error issue 2024-12-14 20:00:17 +01:00
tctx.h io_uring: simplify __io_uring_add_tctx_node 2022-10-07 12:25:30 -06:00
timeout.c io_uring/timeout: fix multishot updates 2025-01-17 13:36:19 +01:00
timeout.h io_uring: remove unused return from io_disarm_next 2022-09-21 13:15:01 -06:00
uring_cmd.c io_uring/uring_cmd: use cached cmd_op in io_uring_cmd_sock() 2025-02-08 09:52:33 +01:00
uring_cmd.h io_uring: Remove unnecessary BUILD_BUG_ON 2023-05-04 08:19:05 -06:00
xattr.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
xattr.h io_uring: move xattr related opcodes to its own file 2022-07-24 18:39:11 -06:00