2019-04-30 14:42:43 -04:00
|
|
|
|
2006-01-11 12:17:46 -08:00
|
|
|
|
2019-11-28 15:48:10 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2011-05-26 16:00:52 -04:00
|
|
|
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 17:04:11 +09:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2006-01-08 01:02:50 -08:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2011-09-16 02:31:11 -04:00
|
|
|
|
2006-03-23 20:00:26 +01:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
2016-12-24 11:46:01 -08:00
|
|
|
|
block: implement async io_uring discard cmd
io_uring allows implementing custom file specific asynchronous
operations via the fops->uring_cmd callback, a.k.a. IORING_OP_URING_CMD
requests or just io_uring commands. Use it to add support for async
discards.
Normally, it first tries to queue up bios in a non-blocking context,
and if that fails, we'd retry from a blocking context by returning
-EAGAIN to the core io_uring. We always get the result from bios
asynchronously by setting a custom bi_end_io callback, at which point
we drag the request into the task context to either reissue or complete
it and post a completion to the user.
Unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only
do a best effort attempt to invalidate page cache, and it can race with
any writes and reads and leave page cache stale. It's the same kind of
races we allow to direct writes.
Also, apart from cases where discarding is not allowed at all, e.g.
discards are not supported or the file/device is read only, the user
should assume that the sector range on disk is not valid anymore, even
when an error was returned to the user.
Suggested-by: Conrad Meyer <conradmeyer@meta.com>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/2b5210443e4fa0257934f73dfafcc18a77cd0e09.1726072086.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2024-09-11 17:34:41 +01:00
|
|
|
|
|
|
|
|
|
2025-06-30 14:35:48 +05:30
|
|
|
|
block: implement async io_uring discard cmd
io_uring allows implementing custom file specific asynchronous
operations via the fops->uring_cmd callback, a.k.a. IORING_OP_URING_CMD
requests or just io_uring commands. Use it to add support for async
discards.
Normally, it first tries to queue up bios in a non-blocking context,
and if that fails, we'd retry from a blocking context by returning
-EAGAIN to the core io_uring. We always get the result from bios
asynchronously by setting a custom bi_end_io callback, at which point
we drag the request into the task context to either reissue or complete
it and post a completion to the user.
Unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only
do a best effort attempt to invalidate page cache, and it can race with
any writes and reads and leave page cache stale. It's the same kind of
races we allow to direct writes.
Also, apart from cases where discarding is not allowed at all, e.g.
discards are not supported or the file/device is read only, the user
should assume that the sector range on disk is not valid anymore, even
when an error was returned to the user.
Suggested-by: Conrad Meyer <conradmeyer@meta.com>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/2b5210443e4fa0257934f73dfafcc18a77cd0e09.1726072086.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2024-09-11 17:34:41 +01:00
|
|
|
|
2020-03-25 16:48:41 +01:00
|
|
|
|
blk-crypto: add ioctls to create and prepare hardware-wrapped keys
Until this point, the kernel can use hardware-wrapped keys to do
encryption if userspace provides one -- specifically a key in
ephemerally-wrapped form. However, no generic way has been provided for
userspace to get such a key in the first place.
Getting such a key is a two-step process. First, the key needs to be
imported from a raw key or generated by the hardware, producing a key in
long-term wrapped form. This happens once in the whole lifetime of the
key. Second, the long-term wrapped key needs to be converted into
ephemerally-wrapped form. This happens each time the key is "unlocked".
In Android, these operations are supported in a generic way through
KeyMint, a userspace abstraction layer. However, that method is
Android-specific and can't be used on other Linux systems, may rely on
proprietary libraries, and also misleads people into supporting KeyMint
features like rollback resistance that make sense for other KeyMint keys
but don't make sense for hardware-wrapped inline encryption keys.
Therefore, this patch provides a generic kernel interface for these
operations by introducing new block device ioctls:
- BLKCRYPTOIMPORTKEY: convert a raw key to long-term wrapped form.
- BLKCRYPTOGENERATEKEY: have the hardware generate a new key, then
return it in long-term wrapped form.
- BLKCRYPTOPREPAREKEY: convert a key from long-term wrapped form to
ephemerally-wrapped form.
These ioctls are implemented using new operations in blk_crypto_ll_ops.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Tested-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org> # sm8650
Link: https://lore.kernel.org/r/20250204060041.409950-4-ebiggers@kernel.org
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-02-03 22:00:37 -08:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2019-11-28 11:28:41 +01:00
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2021-08-10 17:45:10 +02:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2024-03-05 11:21:32 +08:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-28 11:28:41 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2020-09-03 07:40:57 +02:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2008-08-25 19:30:16 +09:00
|
|
|
|
2020-04-14 09:28:53 +02:00
|
|
|
|
|
|
|
|
|
2008-09-03 09:03:02 +02:00
|
|
|
|
2020-04-14 09:28:53 +02:00
|
|
|
|
2021-08-10 17:45:11 +02:00
|
|
|
|
2008-09-03 09:03:02 +02:00
|
|
|
|
2024-05-07 03:53:49 +00:00
|
|
|
|
2023-06-29 14:25:17 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-04-14 09:28:53 +02:00
|
|
|
|
|
|
|
|
|
2024-03-05 11:21:32 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-09-03 09:03:02 +02:00
|
|
|
|
2020-04-14 09:28:53 +02:00
|
|
|
|
|
|
|
|
|
2021-08-10 17:45:10 +02:00
|
|
|
|
2020-04-14 09:28:53 +02:00
|
|
|
|
2021-08-10 17:45:12 +02:00
|
|
|
|
2020-04-14 09:28:53 +02:00
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-28 11:28:41 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-11 17:34:40 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
2024-09-11 17:34:40 +01:00
|
|
|
|
2024-05-06 06:20:27 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-09-04 10:58:52 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-08-11 14:17:49 -07:00
|
|
|
|
2024-09-11 17:34:40 +01:00
|
|
|
|
|
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
2024-09-11 17:34:40 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-09-04 10:58:52 +02:00
|
|
|
|
block: fix race between set_blocksize and read paths
With the new large sector size support, it's now the case that
set_blocksize can change i_blksize and the folio order in a manner that
conflicts with a concurrent reader and causes a kernel crash.
Specifically, let's say that udev-worker calls libblkid to detect the
labels on a block device. The read call can create an order-0 folio to
read the first 4096 bytes from the disk. But then udev is preempted.
Next, someone tries to mount an 8k-sectorsize filesystem from the same
block device. The filesystem calls set_blksize, which sets i_blksize to
8192 and the minimum folio order to 1.
Now udev resumes, still holding the order-0 folio it allocated. It then
tries to schedule a read bio and do_mpage_readahead tries to create
bufferheads for the folio. Unfortunately, blocks_per_folio == 0 because
the page size is 4096 but the blocksize is 8192 so no bufferheads are
attached and the bh walk never sets bdev. We then submit the bio with a
NULL block device and crash.
Therefore, truncate the page cache after flushing but before updating
i_blksize. However, that's not enough -- we also need to lock out file
IO and page faults during the update. Take both the i_rwsem and the
invalidate_lock in exclusive mode for invalidations, and in shared mode
for read/write operations.
I don't know if this is the correct fix, but xfs/259 found it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/174543795699.4139148.2086129139322431423.stgit@frogsfrogsfrogs
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-04-23 12:53:42 -07:00
|
|
|
|
2024-04-11 15:53:41 +01:00
|
|
|
|
2024-09-11 17:34:40 +01:00
|
|
|
|
2020-09-04 10:58:52 +02:00
|
|
|
|
2021-11-09 19:47:22 +09:00
|
|
|
|
2024-05-06 06:20:27 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-11-09 19:47:22 +09:00
|
|
|
|
2024-04-11 15:53:41 +01:00
|
|
|
|
block: fix race between set_blocksize and read paths
With the new large sector size support, it's now the case that
set_blocksize can change i_blksize and the folio order in a manner that
conflicts with a concurrent reader and causes a kernel crash.
Specifically, let's say that udev-worker calls libblkid to detect the
labels on a block device. The read call can create an order-0 folio to
read the first 4096 bytes from the disk. But then udev is preempted.
Next, someone tries to mount an 8k-sectorsize filesystem from the same
block device. The filesystem calls set_blksize, which sets i_blksize to
8192 and the minimum folio order to 1.
Now udev resumes, still holding the order-0 folio it allocated. It then
tries to schedule a read bio and do_mpage_readahead tries to create
bufferheads for the folio. Unfortunately, blocks_per_folio == 0 because
the page size is 4096 but the blocksize is 8192 so no bufferheads are
attached and the bh walk never sets bdev. We then submit the bio with a
NULL block device and crash.
Therefore, truncate the page cache after flushing but before updating
i_blksize. However, that's not enough -- we also need to lock out file
IO and page faults during the update. Take both the i_rwsem and the
invalidate_lock in exclusive mode for invalidations, and in shared mode
for read/write operations.
I don't know if this is the correct fix, but xfs/259 found it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/174543795699.4139148.2086129139322431423.stgit@frogsfrogsfrogs
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-04-23 12:53:42 -07:00
|
|
|
|
2021-11-09 19:47:22 +09:00
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
|
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
|
|
|
|
|
2024-09-03 22:48:19 +03:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-03 22:48:19 +03:00
|
|
|
|
|
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
|
|
|
|
|
block: fix race between set_blocksize and read paths
With the new large sector size support, it's now the case that
set_blocksize can change i_blksize and the folio order in a manner that
conflicts with a concurrent reader and causes a kernel crash.
Specifically, let's say that udev-worker calls libblkid to detect the
labels on a block device. The read call can create an order-0 folio to
read the first 4096 bytes from the disk. But then udev is preempted.
Next, someone tries to mount an 8k-sectorsize filesystem from the same
block device. The filesystem calls set_blksize, which sets i_blksize to
8192 and the minimum folio order to 1.
Now udev resumes, still holding the order-0 folio it allocated. It then
tries to schedule a read bio and do_mpage_readahead tries to create
bufferheads for the folio. Unfortunately, blocks_per_folio == 0 because
the page size is 4096 but the blocksize is 8192 so no bufferheads are
attached and the bh walk never sets bdev. We then submit the bio with a
NULL block device and crash.
Therefore, truncate the page cache after flushing but before updating
i_blksize. However, that's not enough -- we also need to lock out file
IO and page faults during the update. Take both the i_rwsem and the
invalidate_lock in exclusive mode for invalidations, and in shared mode
for read/write operations.
I don't know if this is the correct fix, but xfs/259 found it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/174543795699.4139148.2086129139322431423.stgit@frogsfrogsfrogs
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-04-23 12:53:42 -07:00
|
|
|
|
2024-04-11 15:53:37 +01:00
|
|
|
|
2024-09-03 22:48:19 +03:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-04-11 15:53:37 +01:00
|
|
|
|
block: fix race between set_blocksize and read paths
With the new large sector size support, it's now the case that
set_blocksize can change i_blksize and the folio order in a manner that
conflicts with a concurrent reader and causes a kernel crash.
Specifically, let's say that udev-worker calls libblkid to detect the
labels on a block device. The read call can create an order-0 folio to
read the first 4096 bytes from the disk. But then udev is preempted.
Next, someone tries to mount an 8k-sectorsize filesystem from the same
block device. The filesystem calls set_blksize, which sets i_blksize to
8192 and the minimum folio order to 1.
Now udev resumes, still holding the order-0 folio it allocated. It then
tries to schedule a read bio and do_mpage_readahead tries to create
bufferheads for the folio. Unfortunately, blocks_per_folio == 0 because
the page size is 4096 but the blocksize is 8192 so no bufferheads are
attached and the bh walk never sets bdev. We then submit the bio with a
NULL block device and crash.
Therefore, truncate the page cache after flushing but before updating
i_blksize. However, that's not enough -- we also need to lock out file
IO and page faults during the update. Take both the i_rwsem and the
invalidate_lock in exclusive mode for invalidations, and in shared mode
for read/write operations.
I don't know if this is the correct fix, but xfs/259 found it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/174543795699.4139148.2086129139322431423.stgit@frogsfrogsfrogs
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-04-23 12:53:42 -07:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2012-09-18 12:19:29 -04:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2016-10-11 13:51:05 -07:00
|
|
|
|
2020-09-04 10:58:52 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-10-11 13:51:05 -07:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2012-09-18 12:19:29 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-19 08:20:22 +02:00
|
|
|
|
2016-10-11 13:51:05 -07:00
|
|
|
|
|
|
|
|
|
2012-09-18 12:19:29 -04:00
|
|
|
|
|
|
|
|
|
2016-10-11 13:51:05 -07:00
|
|
|
|
block: fix race between set_blocksize and read paths
With the new large sector size support, it's now the case that
set_blocksize can change i_blksize and the folio order in a manner that
conflicts with a concurrent reader and causes a kernel crash.
Specifically, let's say that udev-worker calls libblkid to detect the
labels on a block device. The read call can create an order-0 folio to
read the first 4096 bytes from the disk. But then udev is preempted.
Next, someone tries to mount an 8k-sectorsize filesystem from the same
block device. The filesystem calls set_blksize, which sets i_blksize to
8192 and the minimum folio order to 1.
Now udev resumes, still holding the order-0 folio it allocated. It then
tries to schedule a read bio and do_mpage_readahead tries to create
bufferheads for the folio. Unfortunately, blocks_per_folio == 0 because
the page size is 4096 but the blocksize is 8192 so no bufferheads are
attached and the bh walk never sets bdev. We then submit the bio with a
NULL block device and crash.
Therefore, truncate the page cache after flushing but before updating
i_blksize. However, that's not enough -- we also need to lock out file
IO and page faults during the update. Take both the i_rwsem and the
invalidate_lock in exclusive mode for invalidations, and in shared mode
for read/write operations.
I don't know if this is the correct fix, but xfs/259 found it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/174543795699.4139148.2086129139322431423.stgit@frogsfrogsfrogs
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-04-23 12:53:42 -07:00
|
|
|
|
2024-04-11 15:53:41 +01:00
|
|
|
|
2020-09-04 10:58:52 +02:00
|
|
|
|
|
|
|
|
|
2021-11-09 19:47:23 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-07-01 18:51:20 +02:00
|
|
|
|
2016-10-11 13:51:05 -07:00
|
|
|
|
2021-11-09 19:47:23 +09:00
|
|
|
|
2024-04-11 15:53:41 +01:00
|
|
|
|
block: fix race between set_blocksize and read paths
With the new large sector size support, it's now the case that
set_blocksize can change i_blksize and the folio order in a manner that
conflicts with a concurrent reader and causes a kernel crash.
Specifically, let's say that udev-worker calls libblkid to detect the
labels on a block device. The read call can create an order-0 folio to
read the first 4096 bytes from the disk. But then udev is preempted.
Next, someone tries to mount an 8k-sectorsize filesystem from the same
block device. The filesystem calls set_blksize, which sets i_blksize to
8192 and the minimum folio order to 1.
Now udev resumes, still holding the order-0 folio it allocated. It then
tries to schedule a read bio and do_mpage_readahead tries to create
bufferheads for the folio. Unfortunately, blocks_per_folio == 0 because
the page size is 4096 but the blocksize is 8192 so no bufferheads are
attached and the bh walk never sets bdev. We then submit the bio with a
NULL block device and crash.
Therefore, truncate the page cache after flushing but before updating
i_blksize. However, that's not enough -- we also need to lock out file
IO and page faults during the update. Take both the i_rwsem and the
invalidate_lock in exclusive mode for invalidations, and in shared mode
for read/write operations.
I don't know if this is the correct fix, but xfs/259 found it.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/174543795699.4139148.2086129139322431423.stgit@frogsfrogsfrogs
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-04-23 12:53:42 -07:00
|
|
|
|
2021-11-09 19:47:23 +09:00
|
|
|
|
2012-09-18 12:19:29 -04:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-10-03 20:52:01 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-10-03 20:52:01 +02:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2020-05-18 21:07:34 -07:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
2020-05-18 21:07:34 -07:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-28 15:48:10 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2019-11-28 15:48:10 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2023-06-13 16:40:07 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:07 +08:00
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-08 13:02:54 +02:00
|
|
|
|
|
|
|
|
|
2005-06-23 00:10:15 -07:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
2023-08-11 12:08:26 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-10-18 17:29:24 +02:00
|
|
|
|
|
|
|
|
|
2023-08-11 12:08:26 +02:00
|
|
|
|
2023-10-18 17:29:24 +02:00
|
|
|
|
2023-08-11 12:08:26 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
2023-06-08 13:02:54 +02:00
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
2017-10-18 14:38:38 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
2020-11-03 11:00:11 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-04-12 01:15:55 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-05-21 22:19:55 -04:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-09-18 12:19:29 -04:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-05-21 22:19:55 -04:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2024-04-18 00:34:31 -04:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
2024-04-18 00:34:31 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2012-09-18 12:19:29 -04:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-09-18 12:19:29 -04:00
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2024-04-18 00:34:31 -04:00
|
|
|
|
2008-08-11 15:58:42 +01:00
|
|
|
|
2024-04-18 00:34:31 -04:00
|
|
|
|
|
|
|
|
|
2020-09-21 09:19:47 +02:00
|
|
|
|
2024-04-18 00:34:31 -04:00
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
2006-01-08 01:02:50 -08:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2023-02-17 10:21:59 +08:00
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-08 13:02:54 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2023-06-08 13:02:54 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
2022-04-15 06:52:57 +02:00
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
|
|
|
|
|
2021-07-13 01:05:27 +02:00
|
|
|
|
|
|
|
|
|
2016-10-18 15:40:35 +09:00
|
|
|
|
2023-06-08 13:02:54 +02:00
|
|
|
|
2016-10-18 15:40:35 +09:00
|
|
|
|
2019-10-27 23:05:46 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-10-12 19:08:45 +09:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2018-10-12 19:08:46 +09:00
|
|
|
|
2022-07-06 09:03:45 +02:00
|
|
|
|
2008-09-18 15:53:24 -04:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-10-03 20:52:01 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-10-03 20:52:01 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-10-03 20:52:01 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-10-03 20:52:01 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-10-03 20:52:01 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2009-12-03 09:24:48 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2008-09-18 15:53:24 -04:00
|
|
|
|
2014-05-25 21:43:33 +09:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2012-01-11 16:29:31 +01:00
|
|
|
|
2022-04-15 06:52:42 +02:00
|
|
|
|
2008-09-18 15:53:24 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-08-09 16:17:44 +02:00
|
|
|
|
2008-09-18 15:53:24 -04:00
|
|
|
|
|
|
|
|
|
2021-11-22 14:06:16 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-04-17 16:47:43 +02:00
|
|
|
|
|
|
|
|
|
2008-09-18 15:53:24 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:47 +02:00
|
|
|
|
blk-crypto: add ioctls to create and prepare hardware-wrapped keys
Until this point, the kernel can use hardware-wrapped keys to do
encryption if userspace provides one -- specifically a key in
ephemerally-wrapped form. However, no generic way has been provided for
userspace to get such a key in the first place.
Getting such a key is a two-step process. First, the key needs to be
imported from a raw key or generated by the hardware, producing a key in
long-term wrapped form. This happens once in the whole lifetime of the
key. Second, the long-term wrapped key needs to be converted into
ephemerally-wrapped form. This happens each time the key is "unlocked".
In Android, these operations are supported in a generic way through
KeyMint, a userspace abstraction layer. However, that method is
Android-specific and can't be used on other Linux systems, may rely on
proprietary libraries, and also misleads people into supporting KeyMint
features like rollback resistance that make sense for other KeyMint keys
but don't make sense for hardware-wrapped inline encryption keys.
Therefore, this patch provides a generic kernel interface for these
operations by introducing new block device ioctls:
- BLKCRYPTOIMPORTKEY: convert a raw key to long-term wrapped form.
- BLKCRYPTOGENERATEKEY: have the hardware generate a new key, then
return it in long-term wrapped form.
- BLKCRYPTOPREPAREKEY: convert a key from long-term wrapped form to
ephemerally-wrapped form.
These ioctls are implemented using new operations in blk_crypto_ll_ops.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Tested-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org> # sm8650
Link: https://lore.kernel.org/r/20250204060041.409950-4-ebiggers@kernel.org
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-02-03 22:00:37 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2015-10-15 14:10:48 +02:00
|
|
|
|
2023-06-13 16:40:08 +08:00
|
|
|
|
2008-09-18 15:53:24 -04:00
|
|
|
|
2025-07-28 15:12:00 -07:00
|
|
|
|
2008-09-18 15:53:24 -04:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-12 12:44:50 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2021-10-12 12:44:50 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2021-10-12 12:44:50 +02:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-08-09 16:17:44 +02:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2021-10-19 08:20:22 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2021-10-19 08:20:22 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-04-18 00:34:31 -04:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2021-10-19 08:20:22 +02:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-02-17 10:21:59 +08:00
|
|
|
|
2020-11-03 11:00:18 +01:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2020-11-03 11:00:18 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
2020-11-23 13:38:40 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2023-06-08 13:02:55 +02:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2021-08-09 16:17:44 +02:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2022-04-14 15:40:56 -07:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2021-10-19 08:20:22 +02:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-04-18 00:34:31 -04:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2021-10-19 08:20:22 +02:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
2023-02-17 10:21:59 +08:00
|
|
|
|
2019-11-29 11:48:31 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-29 11:45:30 +01:00
|
|
|
|
|
|
|
|
|
block: implement async io_uring discard cmd
io_uring allows implementing custom file specific asynchronous
operations via the fops->uring_cmd callback, a.k.a. IORING_OP_URING_CMD
requests or just io_uring commands. Use it to add support for async
discards.
Normally, it first tries to queue up bios in a non-blocking context,
and if that fails, we'd retry from a blocking context by returning
-EAGAIN to the core io_uring. We always get the result from bios
asynchronously by setting a custom bi_end_io callback, at which point
we drag the request into the task context to either reissue or complete
it and post a completion to the user.
Unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only
do a best effort attempt to invalidate page cache, and it can race with
any writes and reads and leave page cache stale. It's the same kind of
races we allow to direct writes.
Also, apart from cases where discarding is not allowed at all, e.g.
discards are not supported or the file/device is read only, the user
should assume that the sector range on disk is not valid anymore, even
when an error was returned to the user.
Suggested-by: Conrad Meyer <conradmeyer@meta.com>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/2b5210443e4fa0257934f73dfafcc18a77cd0e09.1726072086.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2024-09-11 17:34:41 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-09-22 11:02:31 -06:00
|
|
|
|
block: implement async io_uring discard cmd
io_uring allows implementing custom file specific asynchronous
operations via the fops->uring_cmd callback, a.k.a. IORING_OP_URING_CMD
requests or just io_uring commands. Use it to add support for async
discards.
Normally, it first tries to queue up bios in a non-blocking context,
and if that fails, we'd retry from a blocking context by returning
-EAGAIN to the core io_uring. We always get the result from bios
asynchronously by setting a custom bi_end_io callback, at which point
we drag the request into the task context to either reissue or complete
it and post a completion to the user.
Unlike ioctl(BLKDISCARD) with stronger guarantees against races, we only
do a best effort attempt to invalidate page cache, and it can race with
any writes and reads and leave page cache stale. It's the same kind of
races we allow to direct writes.
Also, apart from cases where discarding is not allowed at all, e.g.
discards are not supported or the file/device is read only, the user
should assume that the sector range on disk is not valid anymore, even
when an error was returned to the user.
Suggested-by: Conrad Meyer <conradmeyer@meta.com>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/2b5210443e4fa0257934f73dfafcc18a77cd0e09.1726072086.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2024-09-11 17:34:41 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|