Commit graph

1,900 commits

Author SHA1 Message Date
Baokun Li
d1dbc59200 fuse: wake one waiter per freed slot when raising max_background
fuse_get_req() parks background allocations on fch->blocked_waitq via
wait_event_state_exclusive(), so each wakeup releases exactly one
waiter.  fuse_chan_max_background_set() clears fch->blocked when the
new limit exceeds num_background, but the accompanying wake_up()
releases a single waiter regardless of how many slots just became
available.  Raising max_background from 10 to 100 therefore admits one
request instead of ninety.

The remaining waiters are not permanently stranded — the "else if
(!fch->blocked)" branch in fuse_request_end() wakes one more per
completion — but that only helps while requests keep completing.
Consider a fixed pool of threads doing readahead or async direct I/O
with the quota exhausted: every thread is either in flight or parked,
and each completion wakes one waiter while freeing one slot, a net
change of zero.  num_background oscillates around the old limit and
the added quota is never taken up.

Waking one waiter per freed slot also preserves submission order:
once fch->blocked is clear, new callers of fuse_get_req() skip the
waitqueue entirely, overtaking waiters that parked before the limit
was raised.

Use wake_up_nr() with the number of slots that just became available.
Since the wakeup is guarded by !fch->blocked, num_background is
strictly below max_background, so the count is at least 1 and never
degenerates into wake_up_all().

Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Reviewed-By: Horst Birthelmer <hbirthelmer@ddn.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 15:20:29 +02:00
Sang-Heon Jeon
b773346540 fuse: use min_not_zero() in fuse_init_server_timeout()
fuse_init_server_timeout() limits timeout to fuse_max_req_timeout with
the same logic as min_not_zero(), and returns early exactly when the
computed timeout would be zero.

So use min_not_zero() instead and return when the computed timeout is
zero.

No functional change.

Signed-off-by: Sang-Heon Jeon <ekffu200098@gmail.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 14:35:23 +02:00
Xiang Mei
fd10f40af3 fuse: copy request headers via a stack buffer for io-uring
The fuse-io-uring transport copies req->in.h out to the ring in
fuse_uring_copy_to_ring() and req->out.h back in fuse_uring_commit().
Both headers live inside the fuse_request slab object, whose cache
(fuse_req_cachep) is created without a usercopy whitelist, so copying
them directly to/from userspace trips CONFIG_HARDENED_USERCOPY and
panics:

  usercopy: Kernel memory exposure attempt detected from SLUB object
  'fuse_request' (offset 56, size 40)!
  kernel BUG at mm/usercopy.c:102!
  Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
  RIP: 0010:usercopy_abort (mm/usercopy.c:90)
  Call Trace:
   __check_heap_object (mm/slub.c:8268)
   __check_object_size (mm/usercopy.c:197 mm/usercopy.c:258 mm/usercopy.c:223)
   copy_header_to_ring (fs/fuse/dev_uring.c:618)
   fuse_uring_prepare_send (fs/fuse/dev_uring.c:776 fs/fuse/dev_uring.c:785)
   fuse_uring_send_in_task (fs/fuse/dev_uring.c:1306)
   tctx_task_work_run (io_uring/tw.c:96)
   task_work_run (kernel/task_work.c:233)
   io_run_task_work (io_uring/tw.h:84)
   io_cqring_wait (io_uring/wait.c:278)
   __do_sys_io_uring_enter (io_uring/io_uring.c:2685)
   entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)

Bounce both headers through an on-stack copy so the usercopy touches
stack memory, not the slab object.

Fixes: c090c8abae ("fuse: Add io-uring sqe commit and fetch support")
Cc: stable@vger.kernel.org
Reported-by: Weiming Shi <bestswngs@gmail.com>
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Xiang Mei <xmei5@asu.edu>
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 13:50:31 +02:00
Xuewen Yan
4332cf75e4 fuse: give wakeup hints to the scheduler for synchronous requests
When a synchronous FUSE request is sent, the in-kernel client queues it
on fiq->pending and wakes the userspace daemon sleeping in
fuse_dev_do_read()->wait_event_interruptible_exclusive(fiq->waitq, ...).
The client then blocks in request_wait_answer() waiting for the reply,
so the waker is about to go to sleep: this is exactly the pattern that
WF_SYNC is meant to optimise.
As Peter Zijlstra explained in the earlier discussion [1],
WF_SYNC is a hint that the waker is about to sleep and the waker and
wakee share data, so stacking the woken thread on the current CPU
is beneficial for cache locality instead of searching for an idle one.

Add a wake_up_sync() wrapper for task on the synchronous request path.

Performance:

  On an Android big.LITTLE device where the FUSE daemon (MediaProvider)
  runs as a background service on the little cores while foreground
  applications run on the big cores, the synchronous wakeup hint lets
  the scheduler pull the daemon thread onto the big core that is
  issuing the request, where the request data is cache-hot.  Measured
  by qixiaoyu [2] on a 2000-picture zip decompression to /sdcard:

    ------------------------------------------
    | Default   | patched     | Improvement |
    ------------------------------------------
    | 13.0 s    | 7.0 s       | 46%         |
    ------------------------------------------

    Server thread wall duration: 3583 ms -> 1276 ms
    Server runs on big core:       5% -> 79%

  The original 4K-file copy/compress/decompress workload [1] on the
  same kind of device showed a ~28% improvement (13.8s -> 9.9s).

  Note: Miklos reported [2] that on his test box he could not observe
  an actual migration from wake_up_interruptible_sync(); the benefit
  appears to be most visible on asymmetric topologies (big.LITTLE,
  where the daemon normally lives on a little core) and on workloads
  dominated by small synchronous requests.  No regression was
  reported on the symmetric- SMP test setups tried.

The earlier version of this change [1] added a `bool sync` argument
to all three hooks of `struct fuse_iqueue_ops` and threaded it
through virtio_fs as well.  Miklos questioned the interface churn,
and the patch has been stalled since.

Re-work it so the exported interface is left alone.  The hint is
carried in a new FR_SYNC_WAKEUP bit of the existing `fuse_req->flags`
bitfield (an `unsigned long`, so no layout change):

  - __fuse_request_send() sets the flag before fuse_send_one().
  - fuse_dev_queue_req() consumes it with test_and_clear_bit() and
    forwards the result to fuse_dev_wake_and_unlock(), which then
    picks wake_up_sync() or wake_up().
  - The forget, interrupt and resend paths pass `false` explicitly,
    preserving their original wake_up() behaviour.

Only /dev/fuse ever wakes fiq->waitq; virtio_fs and fuse_uring
dispatch through their own transport and never call wake_up(), so
threading `sync` through their ops would just add an unused argument.
test_and_clear_bit() makes the flag a one-shot hint that cannot leak
into a future requeue, and no extra cleanup is needed in
fuse_request_end()/fuse_put_request().

[1] https://lore.kernel.org/lkml/1638780405-38026-1-git-send-email-quic_pragalla@quicinc.com/
[2] https://lore.kernel.org/lkml/20221222093407.GA1141@mi-HP-ProDesk-680-G4-MT/

This work is based on "Pradeep P V K <quic_pragalla@quicinc.com>"  and
"Pavankumar Kondeti <quic_pkondeti@quicinc.com>"

Assisted-by: TRAE:GLM-5.2
Signed-off-by: Xuewen Yan <xuewen.yan@unisoc.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 11:52:56 +02:00
Baokun Li
928f659a3e fuse: check for NULL root inode in fuse_fill_super_submount
fuse_iget() can return NULL when its inode allocation fails, but
fuse_fill_super_submount() passed the result straight to get_fuse_inode()
and decremented fi->nlookup without checking it:

        root = fuse_iget(sb, parent_fi->nodeid, ...);
        fi = get_fuse_inode(root);
        fi->nlookup--;

Inside fuse_iget() the inode allocation can fail and return NULL.  The
submount root takes the iget5_locked() path, whose alloc_inode() can fail
under memory pressure (the auto-submount branch can fail the same way in
new_inode() or fuse_alloc_submount_lookup()):

        inode = iget5_locked(sb, nodeid, fuse_inode_eq, fuse_inode_set,
                             &nodeid);
        if (!inode)
                return NULL;

A NULL root makes get_fuse_inode() a container_of() on NULL and the
nlookup decrement a write to a bogus address, oopsing the mount.  With
CONFIG_KASAN the following null pointer dereference is reported when the
root inode allocation of an auto-submount fails (e.g. under memory
pressure):

==================================================================
BUG: KASAN: null-ptr-deref in fuse_get_tree_submount+0x656/0x8b0
Read of size 8 at addr 00000000000002b0 by task ls/942
CPU: 0 PID: 942 Comm: ls Tainted: G W 6.6 #15
Call Trace:
 <TASK>
 fuse_get_tree_submount+0x656/0x8b0
 vfs_get_tree+0x48/0x140
 fc_mount+0x13/0x50
 fuse_dentry_automount+0x7a/0xb0
 __traverse_mounts+0xca/0x330
 step_into+0x339/0xac0
 path_lookupat+0xc5/0x2f0
 filename_lookup+0x163/0x2a0
 vfs_statx+0xd5/0x200
 do_statx+0x83/0xd0
 __x64_sys_statx+0xa0/0xc0
 do_syscall_64+0x37/0x90
 entry_SYSCALL_64_after_hwframe+0x78/0xe2
 </TASK>
==================================================================

Return -ENOMEM instead; the caller tears down the partially built
superblock on error, matching the other error returns in this
function.

Fixes: 1866d779d5 ("fuse: Allow fuse_fill_super_common() for submounts")
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Reviewed-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 11:21:05 +02:00
Baokun Li
ed97699493 fuse: reject a duplicate fd= mount option
fuse_opt_fd() stored the fuse device in ctx->fud and bumped its refcount
unconditionally:

        ctx->fud = fuse_dev_grab(file);

If fd= is given twice (two fsconfig FSCONFIG_SET_FD calls), the second
call overwrites ctx->fud and grabs the new device, while the reference
taken on the first device is never released - a permanent refcount leak
that pins the first fuse_dev until reboot.

Reject a second fd= outright.  ctx is zeroed on allocation, so a non-NULL
ctx->fud reliably means the option was already processed.

Fixes: d42eb23b2e ("fuse: don't require /dev/fuse fd to be kept open during mount")
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Reviewed-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 10:58:56 +02:00
Baokun Li
4deb3edead cuse: wait for pending RCU callbacks on module exit
Since commit 053fc4f755 ("fuse: fix UAF in rcu pathwalks"),
fuse_conn_put() frees the fuse_conn through call_rcu() rather than
synchronously.  For cuse, fc->release is cuse_fc_release(), which
lives in the cuse module.  If the module is removed before the RCU
grace period ends, the callback jumps into freed module memory:

      userspace / module unload      |        RCU softirq
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 close(/dev/cuse)                    |
  cuse_channel_release()             |
   fuse_dev_release()                |
    fuse_conn_put(fch->conn)         |
     call_rcu(delayed_release) ------+---> callback queued
                                     |
 rmmod cuse                          |
  cuse_exit()                        |
   cuse_channel_destroy()            |
   ...                               |
   return                            |
                                     |
 <module text freed>                 |
                                     |  rcu_do_batch()
                                     |   delayed_release()
                                     |    fc->release()
                                     |     -> cuse_fc_release()
                                     |        ^^^ freed text!

The freed module text is unmapped by vfree(), so the jump into the
stale callback triggers a page-fault Oops.  If the virtual address
is subsequently reused, the callback could execute unrelated code
(undefined behaviour).

Fix this by calling rcu_barrier() in cuse_exit() so that any pending
fuse_conn release callback completes before the module is removed.

Fixes: 053fc4f755 ("fuse: fix UAF in rcu pathwalks")
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 10:30:50 +02:00
Baokun Li
a927f1867e fuse: fix invalidate lock leak on open O_TRUNC DAX failure
fuse_open() takes filemap_invalidate_lock() for a DAX truncate
(dax_truncate = true) and releases it before the out_inode_unlock
label.  But when fuse_dax_break_layouts() fails, the goto
out_inode_unlock skips the unlock and leaks the rwsem, so any later
fault or truncate on the file stalls on the stale lock.

fuse_dax_break_layouts() can fail with -ERESTARTSYS when a signal
interrupts the wait for busy DAX pages to drain:

  open("file", O_RDWR | O_TRUNC)
  └─ fuse_open()
     ├─ filemap_invalidate_lock()        # dax_truncate
     └─ fuse_dax_break_layouts()
        └─ dax_break_layout()
           └─ wait_page_idle()           # TASK_INTERRUPTIBLE
              └─ fuse_wait_dax_page()    # unlock, schedule, re-lock
                 └─ signal → -ERESTARTSYS
     goto out_inode_unlock               # <- lock leaked

Fix this by moving filemap_invalidate_unlock() below the label so
that all error paths release the lock, and rename the label to
out_unlock as it now covers more than just the inode lock.

Fixes: 2fdbb8dd01 ("fuse: fix deadlock between atomic O_TRUNC and page invalidation")
Cc: stable@vger.kernel.org # v6.0+
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 10:26:12 +02:00
Baokun Li
9afeca0d56 fuse: fix invalidate lock leak on setattr writeback failure
fuse_do_setattr() takes filemap_invalidate_lock() for a DAX truncate
(fault_blocked = true) and releases it at the out:/error: labels.  But
when a writeback flush is also needed, a write_inode_now() failure
returns directly and leaks the lock, so any later fault or truncate on
the file stalls on the stale rwsem.

For example, truncate(2) on a setuid file reaches fuse_do_setattr()
with both ATTR_SIZE and ATTR_MODE set:

  truncate(2)
  └─ do_truncate()
     ├─ dentry_needs_remove_privs()         # S_ISUID
     └─ notify_change()                     # KILL_SUID -> ATTR_MODE
        └─ fuse_setattr()                   # no killpriv:
           │                                #   ia_valid |= ATTR_MODE
           └─ fuse_do_setattr()
              ├─ filemap_invalidate_lock()  # IS_DAX && is_truncate
              └─ write_inode_now()          # is_wb && ATTR_MODE
                 └─ if (err)                # e.g. daemon -> -EIO
                    return err              # <- lock leaked

Fix this by adding an unlock label that releases the lock before
returning the error, and use it for the fuse_dax_break_layouts()
failure path as well.

Fixes: 6ae330cad6 ("virtiofs: serialize truncate/punch_hole and dax fault path")
Cc: stable@vger.kernel.org # v5.10+
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-18 10:26:12 +02:00
Rochan Avlur
64b0b5cacb fuse: wait for FR_FINISHED on abort_on_kill to prevent use-after-free
The abort_on_kill path in request_wait_answer() calls fuse_abort_conn()
and returns without waiting for FR_FINISHED.  If fuse_dev_do_write() is
concurrently processing the same request (FR_LOCKED set), the caller
frees req->args while it is still being accessed, causing a
use-after-free.

Fix this by jumping to the existing wait_event(FR_FINISHED) instead of
returning early.  The wait will not hang because fuse_abort_conn()
ensures all requests are ended.

Reported-by: syzbot+d6540a3fa1626e11360d@syzkaller.appspotmail.com
Fixes: 204aa22a68 ("fuse: abort on fatal signal during sync init")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Rochan Avlur <rochan.avlur@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:55:28 +02:00
Ben Dooks
6e64df0f73 fuse: make dentry_tree_work static
The dentry_tree_work is not exported, so make it static
to remove the followign sparse warning:
fs/fuse/dir.c:37:21: warning: symbol 'dentry_tree_work' was not declared. Should it be static?

Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:22:10 +02:00
Joanne Koong
43f8343858 fuse: add zero-copy over io-uring
Implement zero-copy in fuse io-uring to eliminate memory copies between
the application, kernel, and server for read/write operations. The
server can directly access client pages or page cache folios without
copying data through an intermediary buffer. When a fuse request arrives,
the kernel registers the relevant pages into a sparse slot in the
server's io_uring registered buffer table. The server can then operate
on these pages directly using io-uring fixed buffer operations (eg
read_fixed/write_fixed) and the kernel unregisters these pages when the
request completes. Non-page-backed args (eg op out headers) will go
through the payload buffer as normal. The server can specify which open
files should have their reads/writes go through zero-copy, by setting
the FOPEN_IO_URING_ZERO_COPY flag when servicing opens.

This requires CAP_SYS_ADMIN and bufpools. This is gated behind
CAP_SYS_ADMIN because zero-copy allows the server direct access to the
client's underlying pages, rather than operating on an intermediary
buffer that the contents of the client's pages were copied into or on
page cache folios.

The request flow for the zero-copy direct-io write path (client writes
data, server reads it) is as follows:
=======================================================================
|  Kernel                                   |  FUSE server
|                                           |
|  "write(fd, buf, 1MB)"                    |
|                                           |
|  >sys_write()                             |
|    >fuse_file_write_iter()                |
|      >fuse_send_one()                     |
|        [req->args->in_pages = true]       |
|        [folios hold client write data]    |
|                                           |
|  >fuse_uring_copy_to_ring()               |
|    >copy_header_to_ring(IN_OUT)           |
|      [memcpy fuse_in_header]              |
|    >copy_header_to_ring(OP)               |
|      [memcpy write_in header]             |
|                                           |
|    >fuse_uring_args_to_ring()             |
|      >setup_fuse_copy_state()             |
|        [skip_folio_copy = true]           |
|                                           |
|      >fuse_uring_set_up_zero_copy()       |
|        [folio_get for each client folio]  |
|        [build bio_vec array from folios]  |
|        >io_buffer_register_bvec()         |
|          [register pages at
                 ent->zero_copy_index]      |
|        [ent->zero_copied = true]          |
|                                           |
|      >fuse_copy_args()                    |
|        [skip_folio_copy => return 0       |
|         for page arg, skip data copy]     |
|                                           |
|    >copy_header_to_ring(RING_ENT)         |
|      [memcpy ent_in_out]                  |
|    >io_uring_cmd_done()                   |
|                                           |
|                                           | [CQE received]
|                                           |
|                                           | [issue io_uring READ at
|                                           |  ent->zero_copy_index]
|                                           | [reads directly from
|                                           |client's pages (ZERO_COPY)]
|                                           |
|                                           | [write data to backing
|                                           | store]
|                                           |  [submit COMMIT AND FETCH]
|                                           |
|  >fuse_uring_commit_fetch()               |
|    >fuse_uring_commit()                   |
|      >fuse_uring_copy_from_ring()         |
|    >fuse_uring_req_end()                  |
|      >io_buffer_unregister(ent->zero_copy_index) |
|        [unregister pages from index]      |
|      >fuse_zero_copy_release()            |
|        [folio_put for each folio]         |
|      [ent->zero_copied = false]           |
|      >fuse_request_end()                  |
|        [wake up client]                   |

The zero-copy read path is analogous.

Some requests may have both page-backed args and non-page-backed args.
For these requests, the page-backed args are zero-copied while the
non-page-backed args are copied to the buffer selected from the buffer
pool:
    zero-copy: pages registered via io_buffer_register_bvec()
    non-page-backed: copied to payload buffer via fuse_copy_args()

For a request whose payload is zero-copied, the
registration/unregistration path looks like:

    register:  fuse_uring_set_up_zero_copy()
                 folio_get() for each folio
                 io_buffer_register_bvec(ent->zero_copy_index)

    unregister: fuse_uring_req_end()
                  io_buffer_unregister(ent->zero_copy_index)
                  -> fuse_zero_copy_release() callback
                     folio_put() for each folio

Please note that on abort for in-flight zero-copied requests that have
been sent to userspace, the registered bvec slot remains occupied and
its folios remain pinned until the io-uring ring is destroyed, at which
point io-uring unregisters all buffers and the fuse_zero_copy_release()
callback drops the folio references. Unregistering at teardown would
require operating on the ring context directly, whose validity is hard
to ascertain; this is deemed not worth the complexity for the abort
race, since everything is freed when the ring is torn down.

The throughput improvement from zero-copy depends on how much of the
per-request latency is spent on data copying vs backing I/O. The gain
comes from eliminating the payload-buffer memcpy,  but accessing the
zero-copied pages requires the server to issue the read/write as an
IORING_OP_READ/WRITE_FIXED operation. The benefit is largest when the
mempcy is a meaningful fraction of per-request latency while backing i/o
is still noticable enough that the extra io-uring op's overhead doesn't
dominate.

Benchmarked with passthrough_hp (--nopassthrough, q_depth=8) on a
2-socket Intel Xeon Gold 6138 (40 cores / 80 threads), using fio (sync
engine, bs=1M, O_DIRECT, numjobs=2, 30s run + 10s ramp, 3 runs) where
direct-I/O throughput is against a RAM-backed (tmpfs) source (backing
I/O is not the bottleneck):

		baseline   registered-buf   zero-copy   (zc vs base)
direct read     ~5.1 GB/s  ~5.4 GB/s        ~8.9 GB/s   (+75%)
direct write    ~3.4 GB/s  ~4.8 GB/s        ~5.1 GB/s   (+50%)

Reads end up higher than writes because the backing store reads faster
than it writes (the baseline shows the same read>write gap, and the raw
device does too). On a device-bound NVMe (~2 GB/s reads) the read gain
shrinks to ~10-16% (and no measurable gains for writes), as backing I/O
rather than the eliminated copy dominates latency.

The benefit overall scales with how much of the
per-request latency is the data copy versus backing I/O.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Joanne Koong
96caf2496e fuse: support registered buffer pools in io-uring
Allow servers to use a buffer pool that is also registered through
io-uring. When the server registers a buffer pool with io-uring, the
pages backing the pool are pinned upfront. This eliminates the overhead
of pinning/unpinning user pages and translating virtual addresses per
i/o request. This also allows servers to use the same registered memory
for subsequent backing store I/O (eg read_fixed/write_fixed), keeping
data in the same pinned pages without additional pinning or mapping
overhead required.

To use this, the server needs to set the FUSE_URING_REGISTERED_BUFPOOL
flag when adding a bufpool through the FUSE_IO_URING_CMD_ADD_BUFPOOL
cmd. For every sqe submitted (including the one for adding the bufpool),
it should set sqe->uring_cmd_flags to include IORING_URING_CMD_FIXED,
and pass in the index where the registered bufpool resides to
sqe->buf_index.

Benchmarked with passthrough_hp (--nopassthrough, q_depth=8) on a
2-socket Intel Xeon Gold 6138 (40 cores / 80 threads), using fio (sync
engine, bs=1M, O_DIRECT, numjobs=2, 30s run + 10s ramp, 3 runs) where
direct-I/O throughput is against a RAM-backed (tmpfs) source (backing
I/O is not the bottleneck):

		    baseline      registered buffers
  direct read       ~5.1 GB/s     ~5.4 GB/s   (+~5%)
  direct write      ~3.4 GB/s     ~4.8 GB/s   (+~45%)

Registered buffers bring up the write path speed up closer to speed of
reads. There isn't much improvement for reads because it is already fast
enough where it's at the copy-bound ceiling (surpassing that requires
doing zero-copy). On a device-bound NVMe though, the differences are
within noise, as backing I/O dominates per-request latency.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Joanne Koong
b45aaabc62 fuse: add io-uring buffer pools
Right now, ents and buffers are tightly coupled in fuse io-uring where
each entry has its own dedicated payload buffer, requiring N buffers for
N entries where each buffer must be large enough to accomodate the
maximum payload size. This is suboptimal as most request types (lookup,
open, release, getattr, etc) require vastly less bytes than the maximum
payload size and some requests (unlink, rmdir, fsync, flush, etc) do not
require payload buffers at all.

Instead of requiring a 1:1 coupling between ents and payload buffers,
allow the server to pass in a buffer pool (a contiguous chunk of memory)
that the kernel will use as it wishes for servicing ents/requests.
Entries only reserve a "buffer" from the pool while actively processing
a request that requires a payload buffer. This decoupling and letting
the kernel delegate memory from the pool for requests allows the kernel to
optimize memory usage and reduces the memory usage requirements needed
to use fuse-over-io-uring.

A pool is registered per queue with the new
FUSE_IO_URING_CMD_ADD_BUFPOOL command. The server passes the pool's base
address and length in fuse_uring_cmd_req.bufpool.{uaddr,len}.

Internally, the kernel splits the region into buffers of
ring->max_payload_sz bytes each (nr_bufs = pool len / max_payload_sz). A
queue commits to a payload mode on first use: registering an entry that
carries its own payload selects the legacy per-entry mode, while
ADD_BUFPOOL selects pool mode. The two are mutually exclusive, so
ADD_BUFPOOL must be issued before any payload-carrying entries are
registered on that queue. The queue must have been created before the
bufpool is added, through the FUSE_IO_URING_CMD_ADD_QUEUE command.

The kernel tracks free buffers with a bitmap (a set bit marks a free
buffer). On dispatch, a request that needs a payload claims a free
buffer (find_first_bit + clear). A request that needs none claims
nothing. The buffer's byte offset within the pool is reported to the
server in the new fuse_uring_ent_in_out.offset field so that the server
can locate the payload. On completion the buffer is returned to the pool
or reused directly if the next request on that entry also has a payload.

The FUSE_HAS_IO_URING_BUFPOOL flag advertises kernel support to the
server for bufpools.

Buffer pool request flow
~~~~~~~~~~~~~~~~~~~~~~~~
|  Kernel                                  |  FUSE daemon
|                                          |
|  [request arrives]                       |
|    [claim a free pool buffer]            |
|    >fuse_uring_select_buffer()           |
|    [copy headers to ring]                |
|    [copy payload to buffer]              |
|    [report buffer offset in ent_in_out]  |
|    >io_uring_cmd_done()                  |
|                                          |  [read headers]
|                                          |  [read/write payload at offset]
|                                          |  [process request]
|                                          |  >io_uring_submit()
|                                          |   COMMIT_AND_FETCH
|  >fuse_uring_commit_fetch()              |
|    [copy reply from ring]                |
|    [return buffer to the pool]           |
|    >fuse_uring_recycle_buffer()          |

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Joanne Koong
ebed9ea5b4 fuse: add FUSE_IO_URING_CMD_ADD_QUEUE
fuse-over-io-uring queues are currently created lazily, as a side effect
of the first FUSE_IO_URING_CMD_REGISTER command for a given qid. This
ties queue creation to entry registration.

Add a FUSE_IO_URING_CMD_ADD_QUEUE command so a server can create a queue
explicitly, decoupling queue setup from entry registration. This is
additionally a prerequisite for FUSE_IO_URING_CMD_ADD_BUFPOOL, which
attaches a buffer pool to an existing queue and therefore needs the
queue to have been created first.

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Joanne Koong
6330b1f61e fuse: decouple fuse_ring creation from ent registration
Currently, the connection's fuse_ring is created lazily on the first
FUSE_IO_URING_CMD_REGISTER command. A server registers entries from one
thread per queue (one per CPU) and those threads issue their first
REGISTER command concurrently. They then race to create the single
per-connection fuse_ring, which required open-coded handling in
fuse_uring_create() to detect and protect against concurrent creations.

Decouple fuse_ring creation from ent registration and move it to
FUSE_INIT reply processing after a server has negotiated and set
FUSE_OVER_IO_URING. The ring is published before the connection is
marked initialized. fuse_uring_register() no longer creates the ring and
it instead uses the ring set up at init time.

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-08-17 17:02:37 +02:00
Jim Harris
98b4ca2378 fuse: allow larger read requests by setting bdi->io_pages
A FUSE server that advertises a large max_pages and max_write (e.g.
max_pages=256, max_write=1MB) cannot currently obtain matching
FUSE_READ request sizes from the kernel.  Buffered sequential writes
arrive at the server at the negotiated max_write size, but a large
buffered read() is split into several smaller FUSE_READ requests.

For a buffered read, filemap_get_pages() -> page_cache_sync_ra() sizes
the read against ractl_max_pages():

	max_pages = ractl->ra->ra_pages;
	if (req_size > max_pages && bdi->io_pages > max_pages)
		max_pages = min(req_size, bdi->io_pages);

fuse leaves bdi->io_pages at the default VM_READAHEAD_PAGES (128KB), so
a 1MB read() (req_size = 256 pages) is clamped to the readahead window
(128KB, or 256KB for POSIX_FADV_SEQUENTIAL), producing four 256KB
FUSE_READ round-trips instead of one.

Set bdi->io_pages to fc->max_pages after feature negotiation.  As the
code above shows, io_pages only raises the limit when the request size
already exceeds the readahead window, so it enlarges explicitly
requested reads without enlarging the speculative readahead window.
This avoids increasing speculative page-cache readahead on behalf of
an unprivileged server.  NFS does the same, setting io_pages from
rpages while leaving ra_pages at the default.

fc->max_pages is already bounded by fc->max_pages_limit (and, for
virtio-fs, by the virtqueue descriptor count), so io_pages inherits
the same bound.

Suggested-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Jim Harris <jim.harris@nvidia.com>
Assisted-by: Cursor:claude-opus-4.8
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 14:51:24 +02:00
Joanne Koong
42df916e5a fuse: publish io-uring queues with release semantics
fuse_uring_create_queue() initializes a fuse_ring_queue and then
publishes the pointer into ring->queues[qid] with WRITE_ONCE() under the
fch->lock. There are several readers that may concurrently be fetching
that pointer locklessly and then deferencing it.

WRITE_ONCE() doesn't ensure ordering of the queue's field
initialization before the ring->queues[qid] pointer assignment. The
queue must be published with smp_store_release() so the field
initialization is guaranteed to happen before.

Readers in paths where the read may happen concurrently with the store
need to use READ_ONCE() because any race involving a plain access is
undefined.

Fixes: 24fe962c86 ("fuse: {io-uring} Handle SQEs - register commands")
Cc: stable@vger.kernel.org
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 13:06:35 +02:00
Joanne Koong
4ef7c8cc98 fuse: use release/acquire for fch->initialized
fuse_chan_set_initialized() sets values for the connection state and
then sets fch->initialized to true, but lockless readers read
fch->initialized and if true, go to read the connection state values,
without using any barriers.

There are a few instances where this happens (fuse_uring_cmd() before
dispatching register / commit-and-fetch cmds, fuse_dev_do_wriite() for
handling notify retrieves, etc).

To make this as simple as possible, use release/acquire semantics for
writing/reading fch->initialized. Add the missing read barriers.
This is not marked for stable as these are not realistically reachable
on a well-behaved server, and buggy/malicious servers who trigger this
path fail benignly rather than crash or deadlock the kernel.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 13:06:35 +02:00
Joanne Koong
edb310bc27 fuse: fix missing barrier when checking io-uring readiness
fuse_block_alloc() reads fch->initialized and then fch->io_uring.
fch->io_uring is set before fch->initialized, ordered by the smp_wmb()
in fuse_chan_set_intialized(), but fuse_block_alloc() has no matching
read barrier between the two loads.

This may lead a CPU to observe fch->initialized=1 but fch->io_uring=0,
and skip the check that blocks request allocation until the io-uring
queues are ready. This can reintroduce the lock-order inversion deadlock
that commit 3393ff964e prevents.

Add an smp_rmb() barrier to pair with the smp_wmb() in
fuse_chan_set_initialized() to prevent this.

Fixes: 3393ff964e ("fuse: block request allocation until io-uring init is complete")
Cc: stable@vger.kernel.org
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-17 13:06:35 +02:00
Miklos Szeredi
ed9c881f3b fuse: fix race between interrupt and resend
After commit f8fce75fed ("fuse: clear intr_entry in fuse_resend and
fuse_remove_pending_req") the WARN_ON(!list_empty(&req->intr_entry)) in
fuse_request_free() still triggers due to the following race:

In request_wait_answer()
  if (test_bit(FR_SENT, &req->flags)) -> returns true

In fuse_chan_resend()
  clear_bit(FR_SENT, &req->flags)

In request_wait_answer()
  queue_interrupt(req)

Fix by:

 - move clearing FR_SENT inside fpq->lock

 - move setting FR_PENDING inside fiq->lock

 - recheck FR_SENT after acquiring fiq->lock in fuse_dev_queue_interrupt()

Reported-by: zdi-disclosures@trendmicro.com
Fixes: f8fce75fed ("fuse: clear intr_entry in fuse_resend and fuse_remove_pending_req")
Cc: stable@vger.kernel.org # 6.9
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-09 09:03:47 +02:00
Joanne Koong
16f4be93c6 fuse: use iomap helper to mark folio uptodate
When fuse enables large folios, a large folio will be backed by
iomap_folio_state that keeps track of uptodate and dirty state in an
internal bitmap.

Fuse writethrough and notify store paths currently set folio uptodate
state with folio_mark_uptodate(), which touches only the folio-level
flag, but on an iomap-backed folio, that leaves the uptodate bitmap out
of sync.

Use the iomap_folio_mark_uptodate() helper to update both the folio
uptodate state and the iomap uptodate bitmap.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-08 11:12:48 +02:00
Joanne Koong
cc6f804e78 fuse: don't clear folio uptodate on writethrough errors
In the writethrough path (fuse_send_write_pages()), if the write to the
server failed or was a short write, the uptodate flag on the folios are
cleared.

As explained by Matthew in [1], this is dangerous because the folio may
be mapped into userspace. The mm code has the invariant that a
non-uptodate folio must never be visible to userspace (to avoid
potentially leaking confidental information to userspace) and has checks
in place for this that if violated can bring down the whole machine.

Practically speaking, the effect of this change for the fuse
writethrough error path is that if an application does a write and then
the server fails to persist the data or only services a short write, the
page cache folio keeps the data the application wrote instead of being
reverted to the server's contents on the next read. The failure is still
reported to the application synchronously through the short count /
error return of the write() syscall. Folios that were only partially
written are unaffected since they were never marked uptodate in the
first place (fuse_fill_write_page() only marks a folio as uptodate if
the whole folio was written to).

[1] https://lore.kernel.org/linux-fsdevel/ajtPMgO65FA1TXhi@casper.infradead.org/

Suggested-by: Matthew Wilcox <willy@infradead.org>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-08 11:12:48 +02:00
Miklos Szeredi
6648f54f34 fuse: move "epoch" from dentry.d_time to fuse_dentry.epoch
...in hope of removing d_time one day.

Fixes: 2396356a94 ("fuse: add more control over cache invalidation behaviour")
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-07-06 15:44:58 +02:00
Joanne Koong
3372eb0384
fuse: call fuse_send_readpages explicitly from fuse_readahead
Move the call to fuse_send_readpages from the iomap ->submit_read method
to the fuse readahead implementation.

fuse_read_folio() does not need to call fuse_send_readpages() because it
always does reads synchronously (the iomap->submit_read method for this
was a no-op since data->ia is always NULL for fuse_read_folio()).

This prepares for an iomap fix that will call ->submit_read after each
iomap.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260629121750.3392300-3-hch@lst.de
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
2026-07-01 15:26:47 +02:00
Linus Torvalds
6edc20078a fuse update for 7.2
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQSQHSd0lITzzeNWNm3h3BK/laaZPAUCajE+HAAKCRDh3BK/laaZ
 POtqAQDLFg02uYFjQ2KTsgLi6fYvQw2w32d+MvNO4f//pO/dPQD9FfH66MSkOW25
 TV/WLbQ9vzsfdPFfe8uXvm4EF01w9wU=
 =qUPT
 -----END PGP SIGNATURE-----

Merge tag 'fuse-update-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse

Pull fuse updates from Miklos Szeredi:

 - Fix lots of bugs, most from the late 6.x era, but some going back
   to 2.6.x

 - Add subsystems (io-uring, passthrough) and respective maintainers
   (Bernd, Joanne and Amir)

 - Separate transport and fs layers (Miklos)

 - Don't block on cat /dev/fuse (Joanne)

 - Perform some refactoring in fuse-uring (Joanne)

 - Don't use bounce-buffer for READDIR reply in virtio-fs (Matthew Ochs)

 - Clean up documentation (Randy)

 - Improve tracing (Amir)

 - Extend page cache invalidation after DIO (Cheng Ding)

 - Invalidate readdir cache on epoch change (Jun Wu)

 - Misc cleanups

* tag 'fuse-update-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse: (81 commits)
  fuse-uring: clear ent->fuse_req in commit_fetch error path
  fuse-uring: use named constants for io-uring iovec indices
  fuse-uring: refactor setting up copy state for payload copying
  fuse-uring: use enum types for header copying
  fuse-uring: refactor io-uring header copying from ring
  fuse-uring: refactor io-uring header copying to ring
  fuse-uring: separate next request fetching from sending logic
  fuse: invalidate readdir cache on epoch bump
  virtio-fs: avoid double-free on failed queue setup
  fuse: invalidate page cache after DIO and async DIO writes
  fuse: set ff->flock only on success
  fuse: clean up interrupt reading
  fuse: remove stray newline in fuse_dev_do_read()
  fuse: use READ_ONCE in fuse_chan_num_background()
  fuse: dax: Move long delayed work on system_dfl_long_wq
  fuse: add fuse_request_sent tracepoint
  fuse: Add SPDX ID lines to some files
  fuse: use QSTR() instead of QSTR_INIT() in fuse_get_dentry
  fuse: convert page array allocation to kcalloc()
  fuse: use current creds for backing files
  ...
2026-06-18 08:50:52 -07:00
Zhenghang Xiao
7d87a5a284 fuse-uring: clear ent->fuse_req in commit_fetch error path
fuse_uring_commit_fetch() error path called fuse_request_end(req) without
clearing ent->fuse_req when fuse_ring_ent_set_commit() fails. The
still-pending fuse_uring_send_in_task() task-work later dereferences the
dangling pointer through fuse_uring_prepare_send(), causing a
use-after-free.

End the request with fuse_uring_req_end(), which handles all conditions
already.

Annotation/edition by Bernd: The UAF should be fixed by other means already
and actually has to be avoided that way.
Just checking for ent->fuse_req == NULL in fuse_uring_send_in_task()
would be prone to race conditions, because if malicious userspace
would commit requests that have passed the NULL check, but are
in doing args copy, it would still trigger a use-after-free.
Setting ent->fuse_req = NULL in fuse_uring_commit_fetch() still
makes sense, though.

Reported-by: Shuvam Pandey <shuvampandey1@gmail.com>
Reported-by: Berkant Koc <me@berkoc.com>
Signed-off-by: Zhenghang Xiao <kipreyyy@gmail.com>
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:19:45 +02:00
Joanne Koong
8bbb2ad1f6 fuse-uring: use named constants for io-uring iovec indices
Replace magic indices 0 and 1 for the iovec array with named constants
FUSE_URING_IOV_HEADERS and FUSE_URING_IOV_PAYLOAD. This makes the usages
self-documenting and prepares for buffer ring support which will also
reference these iovec slots by index.

Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:21 +02:00
Joanne Koong
c0f9203732 fuse-uring: refactor setting up copy state for payload copying
Add a new helper function setup_fuse_copy_state() to contain the logic
for setting up the copy state for payload copying.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
b2bbd7dcd2 fuse-uring: use enum types for header copying
Use enum types to identify which part of the header needs to be copied.
This improves the interface and will simplify both kernel-space and
user-space header addresses copying when buffer rings are added.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
ba7d47897f fuse-uring: refactor io-uring header copying from ring
Move header copying from ring logic into a new copy_header_from_ring()
function. This makes the copy_from_user() logic more clear and
centralizes error handling / rate-limited logging.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
6582f8a066 fuse-uring: refactor io-uring header copying to ring
Move header copying to ring logic into a new copy_header_to_ring()
function. This makes the copy_to_user() logic more clear and centralizes
error handling / rate-limited logging.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
6813da0950 fuse-uring: separate next request fetching from sending logic
Simplify the logic for fetching + sending off the next request.

This gets rid of fuse_uring_send_next_to_ring() which contained
duplicated logic from fuse_uring_send(). This decouples request fetching
from the send operation, which makes the control flow clearer and
reduces unnecessary parameter passing.

Reviewed-by: Bernd Schubert <bschubert@ddn.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Baokun Li <libaokun@linux.alibaba.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Jun Wu
0fa8346099 fuse: invalidate readdir cache on epoch bump
FUSE_NOTIFY_INC_EPOCH invalidates dentries, but does not invalidate cached
readdir results. A process with cwd inside a FUSE mount can therefore
observe stale readdir(".") output after an epoch bump.

Fix this by recording epoch in the readdir cache and checking it on reuse.

Minimal reproducer:

- mount a tiny FUSE fs with an empty root directory
- on opendir, enable fi->cache_readdir and fi->keep_cache
- chdir into the mount and call readdir(".") to populate readdir cache
- make the FUSE server report one file in the root directory
- send only FUSE_NOTIFY_INC_EPOCH
- call readdir(".") again; before this change it stays stale, after this
  change it sees the new file

Fixes: 2396356a94 ("fuse: add more control over cache invalidation behaviour")
Signed-off-by: Jun Wu <quark@meta.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Luis Henriques <luis@igalia.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Yung-Tse Cheng
6af3330ec5 virtio-fs: avoid double-free on failed queue setup
virtio_fs_setup_vqs() allocates fs->vqs and fs->mq_map before calling
virtio_find_vqs(). If virtio_find_vqs() fails, the error path frees both
pointers and returns an error to virtio_fs_probe().

virtio_fs_probe() then drops the last kobject reference, and
virtio_fs_ktype_release() frees fs->vqs and fs->mq_map again. This leaves
dangling pointers in struct virtio_fs and can trigger a double-free during
probe failure cleanup.

Set fs->vqs and fs->mq_map to NULL immediately after kfree() in the
virtio_fs_setup_vqs() error path so that the later kobject release sees an
uninitialized state and kfree(NULL) becomes harmless.

This can be reproduced when a broken virtio-fs device advertises more
request queues than the transport actually provides. In that case
virtio_find_vqs() fails while setting up the extra queue, and the probe
path reaches the double-free cleanup sequence.

Signed-off-by: Yung-Tse Cheng <mes900903@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Cheng Ding
2b0408d028 fuse: invalidate page cache after DIO and async DIO writes
This fixe does page cache invalidation after DIO and async DIO writes for
both O_DIRECT and FOPEN_DIRECT_IO cases.

Commit b359af8275 ("fuse: Invalidate the page cache after FOPEN_DIRECT_IO
write") fixed xfstests generic/209 for DIO writes in the FOPEN_DIRECT_IO
path. DIO writes without FOPEN_DIRECT_IO are already handled by
generic_file_direct_write().
However, async DIO writes (xfstests generic/451) remain unhandled.

After this fix:
- Async write with FUSE_ASYNC_DIO:
    invalidate in fuse_aio_invalidate_worker()

- Otherwise (Sync or async write without FUSE_ASYNC_DIO):
    - With FOPEN_DIRECT_IO:
        invalidate in fuse_direct_write_iter()
    - Without FOPEN_DIRECT_IO:
        invalidate in generic_file_direct_write()

Workqueue is required for async write invalidation to prevent deadlock:
calling it directly in the I/O end routine (which is in fuse worker thread
context) can block on a folio lock held by a buffered I/O thread waiting
for the same fuse worker thread.

Co-developed-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Cheng Ding <cding@ddn.com>
Reviewed-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Zhang Tianci
71947173ce fuse: set ff->flock only on success
If FUSE_SETLK fails (e.g., due to EWOULDBLOCK), we shall not set
FUSE_RELEASE_FLOCK_UNLOCK in fuse_file_release().

Reported-by: Li Yichao <liyichao.1@bytedance.com>
Signed-off-by: Zhang Tianci <zhangtianci.1997@bytedance.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
ac7304fa1d fuse: clean up interrupt reading
Clean up interrupt reading logic. Remove passing the pointer to the fuse
request as an arg and make the header initializations more readable.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Joanne Koong
1ca4b8b32e fuse: remove stray newline in fuse_dev_do_read()
Remove stray newline that shouldn't be there.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:20 +02:00
Li Wang
521fed4886 fuse: use READ_ONCE in fuse_chan_num_background()
fuse_chan_num_background() is called without holding fch->bg_lock (for
example from fuse_writepages() to compare against fc->congestion_threshold),
while fch->num_background is updated under bg_lock in dev.c and dev_uring.c.
This is the same locked-write/lockless-read pattern already used for
max_background in fuse_chan_max_background().

Use READ_ONCE() on the read side so that:

- The compiler does not cache or coalesce loads of a value that may change
  concurrently on another CPU.
- Prevent KCSAN from reporting an unexpected race.

Signed-off-by: Li Wang <liwang@kylinos.cn>
Fixes: 670d21c6e1 ("fuse: remove reliance on bdi congestion")
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Marco Crivellari
a9489fbda7 fuse: dax: Move long delayed work on system_dfl_long_wq
Currently the code enqueue work items using {queue|mod}_delayed_work(),
using system_long_wq. This workqueue should be used when long works are
expected and it is a per-cpu workqueue.

The function(s) end up calling __queue_delayed_work(), which set a global
timer that could fire anywhere, enqueuing the work where the timer fired.

Unbound works could benefit from scheduler task placement, to optimize
performance and power consumption. Long work shouldn't stick to a single
CPU.

Recently, a new unbound workqueue specific for long running work has
been added:

  c116737e97 ("workqueue: Add system_dfl_long_wq for long unbound works")

Since the workqueue work doesn't rely on per-cpu variables, there is no
obvious reason that justify the use of a per-cpu workqueue. So change
system_long_wq with system_dfl_long_wq so that the work may benefit from
scheduler task placement.

Signed-off-by: Marco Crivellari <marco.crivellari@suse.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Amir Goldstein
e0dcd3f02c fuse: add fuse_request_sent tracepoint
This new tracepoint complements fuse_request_send (enqueue) and
fuse_request_end (completion).  It fires after the request has been
successfully copied to the daemon's buffer, just before the daemon
can start to process it.

fuse_request_sent does not fire if the copy of the request fails.
It also does not fire for NOTIFY_REPLY, which fires the _end tracepoint
at the end of copy.

This is needed for tools tracking the in-flight state of user initiated
fuse requests.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Tim Bird
9f3e7166aa fuse: Add SPDX ID lines to some files
Some fuse source files are missing SPDX-License-Identifier
lines. Add appropriate IDs to these files, and remove old
license references from the headers.

Signed-off-by: Tim Bird <tim.bird@sony.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Thorsten Blum
5ac67ab433 fuse: use QSTR() instead of QSTR_INIT() in fuse_get_dentry
Drop the hard-coded length argument and use the simpler QSTR(). Inline
the code and drop the local variable.

Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
William Theesfeld
03728af4ae fuse: convert page array allocation to kcalloc()
fuse_get_user_pages() allocates the temporary pages[] array used by
iov_iter_extract_pages() with the open-coded kzalloc(n * sizeof(*p),
...) form.  max_pages is derived from the inbound iov_iter and is not
bounded at compile time, so the multiplication can overflow on
sufficiently large iter counts; the resulting too-small allocation
would then be written past by iov_iter_extract_pages().

Switch to kcalloc(), which carries the same zero-on-allocation
semantics and adds the standard size_mul overflow check.  No
functional change for non-overflow inputs.

Signed-off-by: William Theesfeld <william@theesfeld.net>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
GuoHan Zhao
c51248524a fuse: use current creds for backing files
FUSE backing files only need a stable snapshot of the current credentials
for later backing-file I/O. prepare_creds() allocates a mutable copy and
can fail, but this code never modifies or commits the result.

Use get_current_cred() instead and store it as a const pointer. This
matches the rest of the backing-file helpers and avoids an unnecessary
allocation and failure path.

Signed-off-by: GuoHan Zhao <zhaoguohan@kylinos.cn>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Acked-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Joanne Koong
6bd421b499 fuse: remove redundant buffer size checks for interrupt and forget requests
In fuse_dev_do_read(), there is already logic that ensures the buffer is
a minimum of at least FUSE_MIN_READ_BUFFER (8k) bytes.

This makes the buffer size checks for interrupt and forget requests
redundant as sizeof(struct fuse_in_header) + sizeof(struct
fuse_interrupt_in) and sizeof(struct fuse_in_header) + sizeof(struct
fuse_forget_in) are both less than FUSE_MIN_READ_BUFFER.

We can get rid of these checks.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:19 +02:00
Li Wang
d3aa89c9bf fuse: drop redundant check in fuse_sync_bucket_alloc()
kzalloc_obj with __GFP_NOFAIL is documented to never return failure,
and checking for NULL is redundant (__GFP_NOFAIL in gfp_types.h).

Signed-off-by: Li Wang <liwang@kylinos.cn>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Konrad Sztyber
fde04c3d6f fuse: reduce attributes invalidated on directory change
When the contents of a directory is modified, some of its attributes may
also change, so they need to be invalidated.  But this isn't the case
for every attribute.  For instance, unlinking or creating a file doesn't
change the uid/gid of its parent directory.

This can cause unnecessary FUSE_GETATTRs to be sent to user-space.  For
example, fuse_permission() checks if mode, uid, and gid are valid and
will issue a FUSE_GETATTR if they're not, which results in an extra
FUSE_GETATTR request for every FUSE_UNLINK when removing files in the
same directory.

Signed-off-by: Konrad Sztyber <ksztyber@nvidia.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00
Li Wang
8c72b1f0e2 fuse: drop redundant err assignment in fuse_create_open()
In fuse_create_open(), err is initialized to -ENOMEM immediately before
the fuse_alloc_forget() NULL check. If forget allocation fails,
it branches to out_err with that value. If it succeeds, it falls through
without modifying err, so err is still -ENOMEM at the point where
fuse_file_alloc() is called. The second err = -ENOMEM before
fuse_file_alloc() therefore is redundant.

Signed-off-by: Li Wang <liwang@kylinos.cn>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2026-06-15 14:06:18 +02:00