ceph: use iov_iter_extract_bvecs() for direct I/O

Browse source

ceph_direct_read_write() builds its bio_vec array with
iov_iter_get_pages2(), which takes a reference on each page rather than
pinning it. As the page is not pinned, fork() does not copy it early for
the child and page migration is free to move it. If a process forks
while a direct read is in flight and the parent then writes to the page,
the parent gets a fresh copy while the read completes into the original,
which now belongs to the child. The read reports success and the parent
never sees the data.

Switch to iov_iter_extract_bvecs(), which pins user pages with FOLL_PIN
and borrows kernel pages without taking a reference. This is the same
conversion the block layer did in commit a7e689dd1c ("block: Convert
bio_iov_iter_get_pages to use iov_iter_extract_pages") and fuse did in
commit 738adade96 ("fuse: Fix missing FOLL_PIN for direct-io").

Since iov_iter_extract_bvecs() coalesces contiguous ranges within a
folio into a single bio_vec, release per folio in put_bvecs() with
unpin_user_folio(). Unpinning is done based on
iov_iter_extract_will_pin().

The bio_vec array no longer needs to be zeroed. put_bvecs() used to walk
all npages entries and skip those with a NULL bv_page.
iov_iter_extract_bvecs() counts the entries it fills, so return that
count instead and don't go past it. The array is still sized by npages,
as that is the worst case of one bio_vec per page. Cap npages at
USHRT_MAX to match the type taken by iov_iter_extract_bvecs().

This also replaces the deprecated set_page_dirty_lock() with
folio_mark_dirty_lock() and drops the on-stack page array.

Tested against a Ceph 19.2.3 cluster with KASAN, lockdep and DEBUG_VM
enabled. A reproducer issues an 8 MiB O_DIRECT read (libaio and blocking
pread, with and without THP), forks while the read is held in flight by
pausing the OSDs, has the parent write to every page of the buffer,
and verifies the result. Without this change the parent loses the read
data in every variant, and with it the data is intact. The AIO and
direct I/O xfstests in the generic group pass as well.

Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Reviewed-by: Alex Markuze <amarkuze@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

...

This commit is contained in:

Tal Zussman 2026-09-05 19:30:16 -04:00 • committed by Ilya Dryomov

parent 77ff2ba463

commit 25f66aa5fc

1 changed files with 33 additions and 48 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

81

fs/ceph/file.c

View file