Commit graph

1,887 commits

Author SHA1 Message Date
Joanne Koong
b45aaabc62 fuse: add io-uring buffer pools
Right now, ents and buffers are tightly coupled in fuse io-uring where
each entry has its own dedicated payload buffer, requiring N buffers for
N entries where each buffer must be large enough to accomodate the
maximum payload size. This is suboptimal as most request types (lookup,
open, release, getattr, etc) require vastly less bytes than the maximum
payload size and some requests (unlink, rmdir, fsync, flush, etc) do not
require payload buffers at all.

Instead of requiring a 1:1 coupling between ents and payload buffers,
allow the server to pass in a buffer pool (a contiguous chunk of memory)
that the kernel will use as it wishes for servicing ents/requests.
Entries only reserve a "buffer" from the pool while actively processing
a request that requires a payload buffer. This decoupling and letting
the kernel delegate memory from the pool for requests allows the kernel to
optimize memory usage and reduces the memory usage requirements needed
to use fuse-over-io-uring.

A pool is registered per queue with the new
FUSE_IO_URING_CMD_ADD_BUFPOOL command. The server passes the pool's base
address and length in fuse_uring_cmd_req.bufpool.{uaddr,len}.

Internally, the kernel splits the region into buffers of
ring->max_payload_sz bytes each (nr_bufs = pool len / max_payload_sz). A
queue commits to a payload mode on first use: registering an entry that
carries its own payload selects the legacy per-entry mode, while
ADD_BUFPOOL selects pool mode. The two are mutually exclusive, so
ADD_BUFPOOL must be issued before any payload-carrying entries are
registered on that queue. The queue must have been created before the
bufpool is added, through the FUSE_IO_URING_CMD_ADD_QUEUE command.

The kernel tracks free buffers with a bitmap (a set bit marks a free
buffer). On dispatch, a request that needs a payload claims a free
buffer (find_first_bit + clear). A request that needs none claims
nothing. The buffer's byte offset within the pool is reported to the
server in the new fuse_uring_ent_in_out.offset field so that the server
can locate the payload. On completion the buffer is returned to the pool
or reused directly if the next request on that entry also has a payload.

The FUSE_HAS_IO_URING_BUFPOOL flag advertises kernel support to the
server for bufpools.

Buffer pool request flow
~~~~~~~~~~~~~~~~~~~~~~~~
|  Kernel                                  |  FUSE daemon
|                                          |
|  [request arrives]                       |
|    [claim a free pool buffer]            |
|    >fuse_uring_select_buffer()           |
|    [copy headers to ring]                |
|    [copy payload to buffer]              |
|    [report buffer offset in ent_in_out]  |
|    >io_uring_cmd_done()                  |
|                                          |  [read headers]
|                                          |  [read/write payload at offset]
|                                          |  [process request]
|                                          |  >io_uring_submit()
|                                          |   COMMIT_AND_FETCH
|  >fuse_uring_commit_fetch()              |
|    [copy reply from ring]                |
|    [return buffer to the pool]           |
|    >fuse_uring_recycle_buffer()          |

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Joanne Koong
ebed9ea5b4 fuse: add FUSE_IO_URING_CMD_ADD_QUEUE
fuse-over-io-uring queues are currently created lazily, as a side effect
of the first FUSE_IO_URING_CMD_REGISTER command for a given qid. This
ties queue creation to entry registration.

Add a FUSE_IO_URING_CMD_ADD_QUEUE command so a server can create a queue
explicitly, decoupling queue setup from entry registration. This is
additionally a prerequisite for FUSE_IO_URING_CMD_ADD_BUFPOOL, which
attaches a buffer pool to an existing queue and therefore needs the
queue to have been created first.

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Joanne Koong
6330b1f61e fuse: decouple fuse_ring creation from ent registration
Currently, the connection's fuse_ring is created lazily on the first
FUSE_IO_URING_CMD_REGISTER command. A server registers entries from one
thread per queue (one per CPU) and those threads issue their first
REGISTER command concurrently. They then race to create the single
per-connection fuse_ring, which required open-coded handling in
fuse_uring_create() to detect and protect against concurrent creations.

Decouple fuse_ring creation from ent registration and move it to
FUSE_INIT reply processing after a server has negotiated and set
FUSE_OVER_IO_URING. The ring is published before the connection is
marked initialized. fuse_uring_register() no longer creates the ring and
it instead uses the ring set up at init time.

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Jim Harris
98b4ca2378 fuse: allow larger read requests by setting bdi->io_pages
A FUSE server that advertises a large max_pages and max_write (e.g.
max_pages=256, max_write=1MB) cannot currently obtain matching
FUSE_READ request sizes from the kernel.  Buffered sequential writes
arrive at the server at the negotiated max_write size, but a large
buffered read() is split into several smaller FUSE_READ requests.

For a buffered read, filemap_get_pages() -> page_cache_sync_ra() sizes
the read against ractl_max_pages():

	max_pages = ractl->ra->ra_pages;
	if (req_size > max_pages && bdi->io_pages > max_pages)
		max_pages = min(req_size, bdi->io_pages);

fuse leaves bdi->io_pages at the default VM_READAHEAD_PAGES (128KB), so
a 1MB read() (req_size = 256 pages) is clamped to the readahead window
(128KB, or 256KB for POSIX_FADV_SEQUENTIAL), producing four 256KB
FUSE_READ round-trips instead of one.

Set bdi->io_pages to fc->max_pages after feature negotiation.  As the
code above shows, io_pages only raises the limit when the request size
already exceeds the readahead window, so it enlarges explicitly
requested reads without enlarging the speculative readahead window.
This avoids increasing speculative page-cache readahead on behalf of
an unprivileged server.  NFS does the same, setting io_pages from
rpages while leaving ra_pages at the default.

fc->max_pages is already bounded by fc->max_pages_limit (and, for
virtio-fs, by the virtqueue descriptor count), so io_pages inherits
the same bound.

Suggested-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Jim Harris <jim.harris@nvidia.com>
Assisted-by: Cursor:claude-opus-4.8
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 14:51:24 +02:00
Joanne Koong
42df916e5a fuse: publish io-uring queues with release semantics
fuse_uring_create_queue() initializes a fuse_ring_queue and then
publishes the pointer into ring->queues[qid] with WRITE_ONCE() under the
fch->lock. There are several readers that may concurrently be fetching
that pointer locklessly and then deferencing it.

WRITE_ONCE() doesn't ensure ordering of the queue's field
initialization before the ring->queues[qid] pointer assignment. The
queue must be published with smp_store_release() so the field
initialization is guaranteed to happen before.

Readers in paths where the read may happen concurrently with the store
need to use READ_ONCE() because any race involving a plain access is
undefined.

Fixes: 24fe962c86 ("fuse: {io-uring} Handle SQEs - register commands")
Cc: stable@vger.kernel.org
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 13:06:35 +02:00
Joanne Koong
4ef7c8cc98 fuse: use release/acquire for fch->initialized
fuse_chan_set_initialized() sets values for the connection state and
then sets fch->initialized to true, but lockless readers read
fch->initialized and if true, go to read the connection state values,
without using any barriers.

There are a few instances where this happens (fuse_uring_cmd() before
dispatching register / commit-and-fetch cmds, fuse_dev_do_wriite() for
handling notify retrieves, etc).

To make this as simple as possible, use release/acquire semantics for
writing/reading fch->initialized. Add the missing read barriers.
This is not marked for stable as these are not realistically reachable
on a well-behaved server, and buggy/malicious servers who trigger this
path fail benignly rather than crash or deadlock the kernel.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 13:06:35 +02:00
Joanne Koong
edb310bc27 fuse: fix missing barrier when checking io-uring readiness
fuse_block_alloc() reads fch->initialized and then fch->io_uring.
fch->io_uring is set before fch->initialized, ordered by the smp_wmb()
in fuse_chan_set_intialized(), but fuse_block_alloc() has no matching
read barrier between the two loads.

This may lead a CPU to observe fch->initialized=1 but fch->io_uring=0,
and skip the check that blocks request allocation until the io-uring
queues are ready. This can reintroduce the lock-order inversion deadlock
that commit 3393ff964e prevents.

Add an smp_rmb() barrier to pair with the smp_wmb() in
fuse_chan_set_initialized() to prevent this.

Fixes: 3393ff964e ("fuse: block request allocation until io-uring init is complete")
Cc: stable@vger.kernel.org
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 13:06:35 +02:00
Miklos Szeredi
ed9c881f3b fuse: fix race between interrupt and resend
After commit f8fce75fed ("fuse: clear intr_entry in fuse_resend and
fuse_remove_pending_req") the WARN_ON(!list_empty(&req->intr_entry)) in
fuse_request_free() still triggers due to the following race:

In request_wait_answer()
  if (test_bit(FR_SENT, &req->flags)) -> returns true

In fuse_chan_resend()
  clear_bit(FR_SENT, &req->flags)

In request_wait_answer()
  queue_interrupt(req)

Fix by:

 - move clearing FR_SENT inside fpq->lock

 - move setting FR_PENDING inside fiq->lock

 - recheck FR_SENT after acquiring fiq->lock in fuse_dev_queue_interrupt()

Reported-by: zdi-disclosures@trendmicro.com
Fixes: f8fce75fed ("fuse: clear intr_entry in fuse_resend and fuse_remove_pending_req")
Cc: stable@vger.kernel.org # 6.9
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-09 09:03:47 +02:00
Joanne Koong
16f4be93c6 fuse: use iomap helper to mark folio uptodate
When fuse enables large folios, a large folio will be backed by
iomap_folio_state that keeps track of uptodate and dirty state in an
internal bitmap.

Fuse writethrough and notify store paths currently set folio uptodate
state with folio_mark_uptodate(), which touches only the folio-level
flag, but on an iomap-backed folio, that leaves the uptodate bitmap out
of sync.

Use the iomap_folio_mark_uptodate() helper to update both the folio
uptodate state and the iomap uptodate bitmap.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-08 11:12:48 +02:00
Joanne Koong
cc6f804e78 fuse: don't clear folio uptodate on writethrough errors
In the writethrough path (fuse_send_write_pages()), if the write to the
server failed or was a short write, the uptodate flag on the folios are
cleared.

As explained by Matthew in [1], this is dangerous because the folio may
be mapped into userspace. The mm code has the invariant that a
non-uptodate folio must never be visible to userspace (to avoid
potentially leaking confidental information to userspace) and has checks
in place for this that if violated can bring down the whole machine.

Practically speaking, the effect of this change for the fuse
writethrough error path is that if an application does a write and then
the server fails to persist the data or only services a short write, the
page cache folio keeps the data the application wrote instead of being
reverted to the server's contents on the next read. The failure is still
reported to the application synchronously through the short count /
error return of the write() syscall. Folios that were only partially
written are unaffected since they were never marked uptodate in the
first place (fuse_fill_write_page() only marks a folio as uptodate if
the whole folio was written to).

[1] https://lore.kernel.org/linux-fsdevel/ajtPMgO65FA1TXhi@casper.infradead.org/

Suggested-by: Matthew Wilcox <willy@infradead.org>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-08 11:12:48 +02:00
Miklos Szeredi
6648f54f34 fuse: move "epoch" from dentry.d_time to fuse_dentry.epoch
...in hope of removing d_time one day.

Fixes: 2396356a94 ("fuse: add more control over cache invalidation behaviour")
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-06 15:44:58 +02:00
Joanne Koong
3372eb0384
fuse: call fuse_send_readpages explicitly from fuse_readahead
Move the call to fuse_send_readpages from the iomap ->submit_read method
to the fuse readahead implementation.

fuse_read_folio() does not need to call fuse_send_readpages() because it
always does reads synchronously (the iomap->submit_read method for this
was a no-op since data->ia is always NULL for fuse_read_folio()).

This prepares for an iomap fix that will call ->submit_read after each
iomap.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260629121750.3392300-3-hch@lst.de
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
2026-07-01 15:26:47 +02:00
Linus Torvalds
6edc20078a fuse update for 7.2
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQSQHSd0lITzzeNWNm3h3BK/laaZPAUCajE+HAAKCRDh3BK/laaZ
 POtqAQDLFg02uYFjQ2KTsgLi6fYvQw2w32d+MvNO4f//pO/dPQD9FfH66MSkOW25
 TV/WLbQ9vzsfdPFfe8uXvm4EF01w9wU=
 =qUPT
 -----END PGP SIGNATURE-----

Merge tag 'fuse-update-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse

Pull fuse updates from Miklos Szeredi:

 - Fix lots of bugs, most from the late 6.x era, but some going back
   to 2.6.x

 - Add subsystems (io-uring, passthrough) and respective maintainers
   (Bernd, Joanne and Amir)

 - Separate transport and fs layers (Miklos)

 - Don't block on cat /dev/fuse (Joanne)

 - Perform some refactoring in fuse-uring (Joanne)

 - Don't use bounce-buffer for READDIR reply in virtio-fs (Matthew Ochs)

 - Clean up documentation (Randy)

 - Improve tracing (Amir)

 - Extend page cache invalidation after DIO (Cheng Ding)

 - Invalidate readdir cache on epoch change (Jun Wu)

 - Misc cleanups

* tag 'fuse-update-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse: (81 commits)
  fuse-uring: clear ent->fuse_req in commit_fetch error path
  fuse-uring: use named constants for io-uring iovec indices
  fuse-uring: refactor setting up copy state for payload copying
  fuse-uring: use enum types for header copying
  fuse-uring: refactor io-uring header copying from ring
  fuse-uring: refactor io-uring header copying to ring
  fuse-uring: separate next request fetching from sending logic
  fuse: invalidate readdir cache on epoch bump
  virtio-fs: avoid double-free on failed queue setup
  fuse: invalidate page cache after DIO and async DIO writes
  fuse: set ff->flock only on success
  fuse: clean up interrupt reading
  fuse: remove stray newline in fuse_dev_do_read()
  fuse: use READ_ONCE in fuse_chan_num_background()
  fuse: dax: Move long delayed work on system_dfl_long_wq
  fuse: add fuse_request_sent tracepoint
  fuse: Add SPDX ID lines to some files
  fuse: use QSTR() instead of QSTR_INIT() in fuse_get_dentry
  fuse: convert page array allocation to kcalloc()
  fuse: use current creds for backing files
  ...
2026-06-18 08:50:52 -07:00
Zhenghang Xiao
7d87a5a284 fuse-uring: clear ent->fuse_req in commit_fetch error path
fuse_uring_commit_fetch() error path called fuse_request_end(req) without
clearing ent->fuse_req when fuse_ring_ent_set_commit() fails. The
still-pending fuse_uring_send_in_task() task-work later dereferences the
dangling pointer through fuse_uring_prepare_send(), causing a
use-after-free.

End the request with fuse_uring_req_end(), which handles all conditions
already.

Annotation/edition by Bernd: The UAF should be fixed by other means already
and actually has to be avoided that way.
Just checking for ent->fuse_req == NULL in fuse_uring_send_in_task()
would be prone to race conditions, because if malicious userspace
would commit requests that have passed the NULL check, but are
in doing args copy, it would still trigger a use-after-free.
Setting ent->fuse_req = NULL in fuse_uring_commit_fetch() still
makes sense, though.

Reported-by: Shuvam Pandey <shuvampandey1@gmail.com>
Reported-by: Berkant Koc <me@berkoc.com>
Signed-off-by: Zhenghang Xiao <kipreyyy@gmail.com>
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:19:45 +02:00
Joanne Koong
8bbb2ad1f6 fuse-uring: use named constants for io-uring iovec indices
Replace magic indices 0 and 1 for the iovec array with named constants
FUSE_URING_IOV_HEADERS and FUSE_URING_IOV_PAYLOAD. This makes the usages
self-documenting and prepares for buffer ring support which will also
reference these iovec slots by index.

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:21 +02:00
Joanne Koong
c0f9203732 fuse-uring: refactor setting up copy state for payload copying
Add a new helper function setup_fuse_copy_state() to contain the logic
for setting up the copy state for payload copying.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
b2bbd7dcd2 fuse-uring: use enum types for header copying
Use enum types to identify which part of the header needs to be copied.
This improves the interface and will simplify both kernel-space and
user-space header addresses copying when buffer rings are added.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
ba7d47897f fuse-uring: refactor io-uring header copying from ring
Move header copying from ring logic into a new copy_header_from_ring()
function. This makes the copy_from_user() logic more clear and
centralizes error handling / rate-limited logging.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
6582f8a066 fuse-uring: refactor io-uring header copying to ring
Move header copying to ring logic into a new copy_header_to_ring()
function. This makes the copy_to_user() logic more clear and centralizes
error handling / rate-limited logging.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
6813da0950 fuse-uring: separate next request fetching from sending logic
Simplify the logic for fetching + sending off the next request.

This gets rid of fuse_uring_send_next_to_ring() which contained
duplicated logic from fuse_uring_send(). This decouples request fetching
from the send operation, which makes the control flow clearer and
reduces unnecessary parameter passing.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Jun Wu
0fa8346099 fuse: invalidate readdir cache on epoch bump
FUSE_NOTIFY_INC_EPOCH invalidates dentries, but does not invalidate cached
readdir results. A process with cwd inside a FUSE mount can therefore
observe stale readdir(".") output after an epoch bump.

Fix this by recording epoch in the readdir cache and checking it on reuse.

Minimal reproducer:

- mount a tiny FUSE fs with an empty root directory
- on opendir, enable fi->cache_readdir and fi->keep_cache
- chdir into the mount and call readdir(".") to populate readdir cache
- make the FUSE server report one file in the root directory
- send only FUSE_NOTIFY_INC_EPOCH
- call readdir(".") again; before this change it stays stale, after this
  change it sees the new file

Fixes: 2396356a94 ("fuse: add more control over cache invalidation behaviour")
Signed-off-by: Jun Wu <quark@meta.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Luis Henriques <luis@igalia.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Yung-Tse Cheng
6af3330ec5 virtio-fs: avoid double-free on failed queue setup
virtio_fs_setup_vqs() allocates fs->vqs and fs->mq_map before calling
virtio_find_vqs(). If virtio_find_vqs() fails, the error path frees both
pointers and returns an error to virtio_fs_probe().

virtio_fs_probe() then drops the last kobject reference, and
virtio_fs_ktype_release() frees fs->vqs and fs->mq_map again. This leaves
dangling pointers in struct virtio_fs and can trigger a double-free during
probe failure cleanup.

Set fs->vqs and fs->mq_map to NULL immediately after kfree() in the
virtio_fs_setup_vqs() error path so that the later kobject release sees an
uninitialized state and kfree(NULL) becomes harmless.

This can be reproduced when a broken virtio-fs device advertises more
request queues than the transport actually provides. In that case
virtio_find_vqs() fails while setting up the extra queue, and the probe
path reaches the double-free cleanup sequence.

Signed-off-by: Yung-Tse Cheng <mes900903@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Cheng Ding
2b0408d028 fuse: invalidate page cache after DIO and async DIO writes
This fixe does page cache invalidation after DIO and async DIO writes for
both O_DIRECT and FOPEN_DIRECT_IO cases.

Commit b359af8275 ("fuse: Invalidate the page cache after FOPEN_DIRECT_IO
write") fixed xfstests generic/209 for DIO writes in the FOPEN_DIRECT_IO
path. DIO writes without FOPEN_DIRECT_IO are already handled by
generic_file_direct_write().
However, async DIO writes (xfstests generic/451) remain unhandled.

After this fix:
- Async write with FUSE_ASYNC_DIO:
    invalidate in fuse_aio_invalidate_worker()

- Otherwise (Sync or async write without FUSE_ASYNC_DIO):
    - With FOPEN_DIRECT_IO:
        invalidate in fuse_direct_write_iter()
    - Without FOPEN_DIRECT_IO:
        invalidate in generic_file_direct_write()

Workqueue is required for async write invalidation to prevent deadlock:
calling it directly in the I/O end routine (which is in fuse worker thread
context) can block on a folio lock held by a buffered I/O thread waiting
for the same fuse worker thread.

Co-developed-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Cheng Ding <cding@ddn.com>
Reviewed-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Zhang Tianci
71947173ce fuse: set ff->flock only on success
If FUSE_SETLK fails (e.g., due to EWOULDBLOCK), we shall not set
FUSE_RELEASE_FLOCK_UNLOCK in fuse_file_release().

Reported-by: Li Yichao <liyichao.1@bytedance.com>
Signed-off-by: Zhang Tianci <zhangtianci.1997@bytedance.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
ac7304fa1d fuse: clean up interrupt reading
Clean up interrupt reading logic. Remove passing the pointer to the fuse
request as an arg and make the header initializations more readable.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
1ca4b8b32e fuse: remove stray newline in fuse_dev_do_read()
Remove stray newline that shouldn't be there.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Li Wang
521fed4886 fuse: use READ_ONCE in fuse_chan_num_background()
fuse_chan_num_background() is called without holding fch->bg_lock (for
example from fuse_writepages() to compare against fc->congestion_threshold),
while fch->num_background is updated under bg_lock in dev.c and dev_uring.c.
This is the same locked-write/lockless-read pattern already used for
max_background in fuse_chan_max_background().

Use READ_ONCE() on the read side so that:

- The compiler does not cache or coalesce loads of a value that may change
  concurrently on another CPU.
- Prevent KCSAN from reporting an unexpected race.

Signed-off-by: Li Wang <liwang@kylinos.cn>
Fixes: 670d21c6e1 ("fuse: remove reliance on bdi congestion")
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Marco Crivellari
a9489fbda7 fuse: dax: Move long delayed work on system_dfl_long_wq
Currently the code enqueue work items using {queue|mod}_delayed_work(),
using system_long_wq. This workqueue should be used when long works are
expected and it is a per-cpu workqueue.

The function(s) end up calling __queue_delayed_work(), which set a global
timer that could fire anywhere, enqueuing the work where the timer fired.

Unbound works could benefit from scheduler task placement, to optimize
performance and power consumption. Long work shouldn't stick to a single
CPU.

Recently, a new unbound workqueue specific for long running work has
been added:

  c116737e97 ("workqueue: Add system_dfl_long_wq for long unbound works")

Since the workqueue work doesn't rely on per-cpu variables, there is no
obvious reason that justify the use of a per-cpu workqueue. So change
system_long_wq with system_dfl_long_wq so that the work may benefit from
scheduler task placement.

Signed-off-by: Marco Crivellari <marco.crivellari@suse.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Amir Goldstein
e0dcd3f02c fuse: add fuse_request_sent tracepoint
This new tracepoint complements fuse_request_send (enqueue) and
fuse_request_end (completion).  It fires after the request has been
successfully copied to the daemon's buffer, just before the daemon
can start to process it.

fuse_request_sent does not fire if the copy of the request fails.
It also does not fire for NOTIFY_REPLY, which fires the _end tracepoint
at the end of copy.

This is needed for tools tracking the in-flight state of user initiated
fuse requests.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Tim Bird
9f3e7166aa fuse: Add SPDX ID lines to some files
Some fuse source files are missing SPDX-License-Identifier
lines. Add appropriate IDs to these files, and remove old
license references from the headers.

Signed-off-by: Tim Bird <tim.bird@sony.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Thorsten Blum
5ac67ab433 fuse: use QSTR() instead of QSTR_INIT() in fuse_get_dentry
Drop the hard-coded length argument and use the simpler QSTR(). Inline
the code and drop the local variable.

Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
William Theesfeld
03728af4ae fuse: convert page array allocation to kcalloc()
fuse_get_user_pages() allocates the temporary pages[] array used by
iov_iter_extract_pages() with the open-coded kzalloc(n * sizeof(*p),
...) form.  max_pages is derived from the inbound iov_iter and is not
bounded at compile time, so the multiplication can overflow on
sufficiently large iter counts; the resulting too-small allocation
would then be written past by iov_iter_extract_pages().

Switch to kcalloc(), which carries the same zero-on-allocation
semantics and adds the standard size_mul overflow check.  No
functional change for non-overflow inputs.

Signed-off-by: William Theesfeld <william@theesfeld.net>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
GuoHan Zhao
c51248524a fuse: use current creds for backing files
FUSE backing files only need a stable snapshot of the current credentials
for later backing-file I/O. prepare_creds() allocates a mutable copy and
can fail, but this code never modifies or commits the result.

Use get_current_cred() instead and store it as a const pointer. This
matches the rest of the backing-file helpers and avoids an unnecessary
allocation and failure path.

Signed-off-by: GuoHan Zhao <zhaoguohan@kylinos.cn>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Acked-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Joanne Koong
6bd421b499 fuse: remove redundant buffer size checks for interrupt and forget requests
In fuse_dev_do_read(), there is already logic that ensures the buffer is
a minimum of at least FUSE_MIN_READ_BUFFER (8k) bytes.

This makes the buffer size checks for interrupt and forget requests
redundant as sizeof(struct fuse_in_header) + sizeof(struct
fuse_interrupt_in) and sizeof(struct fuse_in_header) + sizeof(struct
fuse_forget_in) are both less than FUSE_MIN_READ_BUFFER.

We can get rid of these checks.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Li Wang
d3aa89c9bf fuse: drop redundant check in fuse_sync_bucket_alloc()
kzalloc_obj with __GFP_NOFAIL is documented to never return failure,
and checking for NULL is redundant (__GFP_NOFAIL in gfp_types.h).

Signed-off-by: Li Wang <liwang@kylinos.cn>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Konrad Sztyber
fde04c3d6f fuse: reduce attributes invalidated on directory change
When the contents of a directory is modified, some of its attributes may
also change, so they need to be invalidated.  But this isn't the case
for every attribute.  For instance, unlinking or creating a file doesn't
change the uid/gid of its parent directory.

This can cause unnecessary FUSE_GETATTRs to be sent to user-space.  For
example, fuse_permission() checks if mode, uid, and gid are valid and
will issue a FUSE_GETATTR if they're not, which results in an extra
FUSE_GETATTR request for every FUSE_UNLINK when removing files in the
same directory.

Signed-off-by: Konrad Sztyber <ksztyber@nvidia.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Li Wang
8c72b1f0e2 fuse: drop redundant err assignment in fuse_create_open()
In fuse_create_open(), err is initialized to -ENOMEM immediately before
the fuse_alloc_forget() NULL check. If forget allocation fails,
it branches to out_err with that value. If it succeeds, it falls through
without modifying err, so err is still -ENOMEM at the point where
fuse_file_alloc() is called. The second err = -ENOMEM before
fuse_file_alloc() therefore is redundant.

Signed-off-by: Li Wang <liwang@kylinos.cn>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Randy Dunlap
567820f02a fuse: fuse_i.h: clean up kernel-doc comments
Convert many comments to kernel-doc format to eliminate around 20
kernel-doc warnings like these:

Warning: fs/fuse/fuse_i.h:374 This comment starts with '/**', but isn't a kernel-doc comment. Refer to Documentation/doc-guide/kernel-doc.rst
 * A Fuse connection.
Warning: fs/fuse/fuse_i.h:817 This comment starts with '/**', but isn't a kernel-doc comment. Refer to Documentation/doc-guide/kernel-doc.rst
 * Get a filled in inode
Warning: fs/fuse/fuse_i.h:859 This comment starts with '/**', but isn't a kernel-doc comment. Refer to Documentation/doc-guide/kernel-doc.rst
 * Send RELEASE or RELEASEDIR request
and more like this.

Also add struct member and function parameter descriptions to avoid
these warnings:

Warning: fs/fuse/fuse_i.h:1071 struct member 'epoch_work' not described in 'fuse_conn'
Warning: fs/fuse/fuse_i.h:1071 struct member 'rcu' not described in 'fuse_conn'
Warning: fs/fuse/fuse_i.h:1423 function parameter 'fc' not described in 'fuse_reverse_inval_inode'
Warning: fs/fuse/fuse_i.h:1423 function parameter 'nodeid' not described in 'fuse_reverse_inval_inode'
Warning: fs/fuse/fuse_i.h:1423 function parameter 'offset' not described in 'fuse_reverse_inval_inode'
Warning: fs/fuse/fuse_i.h:1423 function parameter 'len' not described in 'fuse_reverse_inval_inode'
Warning: fs/fuse/fuse_i.h:1436 function parameter 'fc' not described in 'fuse_reverse_inval_entry'
Warning: fs/fuse/fuse_i.h:1436 function parameter 'parent_nodeid' not described in 'fuse_reverse_inval_entry'
Warning: fs/fuse/fuse_i.h:1436 function parameter 'child_nodeid' not described in 'fuse_reverse_inval_entry'
Warning: fs/fuse/fuse_i.h:1436 function parameter 'name' not described in 'fuse_reverse_inval_entry'
Warning: fs/fuse/fuse_i.h:1436 function parameter 'flags' not described in 'fuse_reverse_inval_entry'

Convert struct fuse_file, struct fuse_submount_lookup, struct fuse_inode,
and struct fuse_conn to kernel-doc.

Convert these to plain comments:
Warning: fs/fuse/fuse_i.h:1423 expecting prototype for File(). Prototype was for fuse_reverse_inval_inode() instead
Warning: fs/fuse/fuse_i.h:1436 expecting prototype for File(). Prototype was for fuse_reverse_inval_entry() instead

Change some "/**" to "/*" since they are not kernel-doc comments.

The changes above fix most kernel-doc warnings in this file but
these warnings are not fixed and still remain:
Warning: fs/fuse/fuse_i.h:1428 No description found for return value of 'fuse_fill_super_common'
Warning: fs/fuse/fuse_i.h:1455 No description found for return value of 'fuse_ctl_add_conn'

Binary build output is the same before and after these changes.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Randy Dunlap
eafb65976d fuse: fuse_dev_i.h: clean up kernel-doc warnings
Change some "/**" to "/*" since they are not kernel-doc comments:

Warning: fs/fuse/fuse_dev_i.h:25 This comment starts with '/**', but isn't a kernel-doc comment. Refer to Documentation/doc-guide/kernel-doc.rst
 * Request flags
Warning: fs/fuse/fuse_dev_i.h:58 This comment starts with '/**', but isn't a kernel-doc comment. Refer to Documentation/doc-guide/kernel-doc.rst
 * A request to the client
Warning: fs/fuse/fuse_dev_i.h:117 This comment starts with '/**', but isn't a kernel-doc comment. Refer to Documentation/doc-guide/kernel-doc.rst
 * Input queue callbacks
Warning: fs/fuse/fuse_dev_i.h:289 This comment starts with '/**', but isn't a kernel-doc comment. Refer to Documentation/doc-guide/kernel-doc.rst
 * Fuse device instance
and more like this.

Convert enum fuse_req_flag to kernel-doc format.
Convert struct fuse_req, struct fuse_iqueue_ops, and struct fuse_dev
to kernel-doc format.

These warnings remain:
Warning: fs/fuse/fuse_dev_i.h:115 struct member 'ring_entry' not described in 'fuse_req'
Warning: fs/fuse/fuse_dev_i.h:115 struct member 'ring_queue' not described in 'fuse_req'

Binary build output is the same before and after these changes.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Randy Dunlap
288241c802 fuse-uring: drop kernel-doc notation for a comment
Use regular C comment syntax for a non-kernel-doc comment to avoid
a kernel-doc warning:

Warning: fs/fuse/dev_uring_i.h:104 This comment starts with '/**', but
 isn't a kernel-doc comment.
 * Describes if uring is for communication and holds alls the data needed

Binary build output is the same before and after this change.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Miklos Szeredi
a636077231 fuse: simplify fuse_dev_ioctl_clone()
Don't need to check if the new device file is already initialized, since
fuse_dev_install_with_pq() will do that anyway.

Make fuse_dev_install_with_pq() return a boolean value indicating success so
that fuse_dev_ioctl_clone() can return an error in case of failure.

Move aborting the connection (setting fc->connected to zero) to
fuse_dev_install(), because it is not needed when the clone ioctl fails.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Miklos Szeredi
48649c0603 fuse: alloc pqueue before installing fch in fuse_dev
Prior to this patchset, fuse_dev (containing fuse_pqueue) was allocated on
mount.  But now fuse_dev is allocated when opening /dev/fuse, even though
the queues are not needed at that time.

Delay allocation of the pqueue (4k worth of list_head) just before mounting
or cloning a device.

Various distributions (e.g. Debian/Fedora) configure /dev/fuse as world
writable, so the pqueue allocation should be deferred to a privileged
operation (mount) to prevent unprivileged userspace from consuming pinned
kernel memory.

[Li Wang: fix kernel NULL pointer dereference in fuse_uring_add_to_pq()]
[Fix race in fuse_dev_release()]

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Miklos Szeredi
c0f817320d fuse: remove #include "fuse_i.h" from dev.c and dev_uring.c
Move a couple of function declarations from fuse_i.h to dev.h and
fuse_dev_i.h.

Add fuse_conn_get_id() helper that retrieves the connection ID (s_dev) from
fuse_conn.

With the exception of cuse.c, virtio_fs.c and trace.c source files now
either include fuse_i.h or fuse_dev_i/dev_uring_i.h but not both.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Miklos Szeredi
b03404ea3a fuse: change ring->fc to ring->chan
Store pointer to struct fuse_chan instead of struct fuse_conn in fuse_ring.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Miklos Szeredi
e582371be4 fuse: remove fuse_mutex protection from fuse_dev_ioctl_sync_init()
In normal use ioctl(FUSE_DEV_IOC_SYNC_INIT) comes before the mount() or
fsconfig() syscalls, they are executed strictly serially.

If ioctl and mount are performed in parallel, the behavior is
nondeterministic.  Removing the mutex does not change this.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Miklos Szeredi
ff572265f2 fuse: set params in fuse_chan_set_initialized()
Set minor, max_write and max_pages in the fuse_chan.  These match the same
fields in fuse_conn but are needed in both layers.

[Dongyang Jin: Pointers should use NULL instead of explicit '0']

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:17 +02:00
Miklos Szeredi
ca46a4aca6 fuse: create notify.c
Move FUSE_NOTIFY_* handling into a separate source file.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:17 +02:00
Miklos Szeredi
b8b072cbaf fuse: create poll.c
Move f_op->poll related functions to the new source file.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:17 +02:00
Miklos Szeredi
994d807943 fuse: change fud->fc to fud->chan
Store pointer to struct fuse_chan instead of struct fuse_conn in fuse_dev.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:17 +02:00
Miklos Szeredi
794e811d14 fuse: split out filesystem part of request sending
Create a new source file: req.c and add the request sending entry
functions:

  __fuse_simple_request()
  fuse_simple_background()
  fuse_simple_notify_reply()

Introduce transport layer sending functions that are called by the
respective fs layer function:

  fuse_chan_send()
  fuse_chan_send_bg()
  fuse_chan_send_notify_reply()

Move calculation of request header fields uid, gid and pid from
fuse_get_req() and fuse_force_creads() to a new helper: fuse_fill_creds().

These fileds are now passed to the transport layer via struct fuse_args.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:17 +02:00