Add the core formatting logic. The initialization path is updated to
read the geometry block (block 0 on the storage device). If the block
is entirely zeroed, the device is treated as unformatted and
vdo_format() is called. Otherwise, the existing geometry is parsed
and the VDO is loaded as before.
The vdo_format() function initializes the volume geometry and super
block, and marks the VDO as needing it's layout saved to disk.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Add vdo_submit_metadata_vio_wait(), a synchronous I/O submission
helper that blocks until completion. This is needed for I/O during
early initialization before work queues are available.
Refactor read_geometry_block() to use it.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Introduce a vdo_geometry_block structure, containing a vio and buffer,
mirroring the existing vdo_super_block structure. Both are now
initialized at VDO startup and freed at shutdown, establishing the
infrastructure needed to read and write the geometry block using the
same mechanisms as the super block.
Refactor read_geometry_block() to use the new structure.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Add vdo_encode_volume_geometry() to write the geometry block into a
buffer so that it can be written to disk. The corresponding decode
path already exists.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Add a validation check that the logical size passed via the table line
does not exceed MAXIMUM_VDO_LOGICAL_BLOCKS.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Extend the dm table line with three new optional parameters:
indexMemory (UDS index memory size), indexSparse (dense vs sparse
index), and slabSize (blocks per allocation slab). These values are
parsed, validated, and stored in the device configuration for use
during formatting.
Rework the slab size constants from the single MAX_VDO_SLAB_BITS into
explicit MIN_VDO_SLAB_BLOCKS, MAX_VDO_SLAB_BLOCKS, and
DEFAULT_VDO_SLAB_BLOCKS values.
Bump the target version from 9.1.0 to 9.2.0 to reflect this table
line change.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Add vdo_initialize_component_states() to populate the super block,
computing the space required for the main VDO components on disk.
Those include the slab depot, block map, and recovery journal.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Add vdo_initialize_volume_geometry() to populate the geometry block,
computing the space required for the two main regions on disk.
Add uds_compute_index_size() to calculate the space required for the
UDS indexer from the UDS configuration.
Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Since all implementations of crypt_iv_operations::wipe now return 0,
change the return type to void.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Simplify and optimize dm-crypt's implementation of Bitlocker's "elephant
diffuser" to use the AES library instead of an "ecb(aes)"
crypto_skcipher.
Note: struct aes_enckey is fixed-size, so it could be embedded directly
in struct iv_elephant_private. But I kept it as a separate allocation
so that the size of struct crypt_config doesn't increase. The elephant
diffuser is rarely used in dm-crypt.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Currently FEC logs a warning message if at least one error was
corrected, or an error message if there were uncorrectable errors.
However, it doesn't log anything if there were no errors.
"No errors" is actually unexpected, though, considering that dm-verity
calls verity_fec_decode() only when a block's digest doesn't match.
If there were to ever be a bug where verity_fec_decode() is called on
blocks with the correct digest, then there would be no indication in the
log that FEC is running and degrading performance.
Therefore, let's log the warning message even when there were no errors.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The command "dmsetup remove_all" may take a long time (a minute for
removing 1000 devices), so make it interruptible with fatal signals.
For better readability, the bool arguments were changed to flags.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
If dm_hash_remove_all was called from dm_deferred_remove, it would write
a warning "remove_all left %d open device(s)" if there are some other
devices active.
The warning is bogus, so let's disable it in this case.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Reported-by: Zdenek Kabelac <zkabelac@redhat.com>
Cc: stable@vger.kernel.org
Fixes: 2c140a246d ("dm: allow remove to be deferred")
The early_lookup_bdev() function returns successfully when the disk
device is present but not necessarily its partitions. In this situation,
dm_early_create() fails as the partition block device does not exist
yet.
In my case, this phenomenon occurs quite often because the device is
an SD card with slow reading times, on which kernel takes time to
enumerate available partitions.
Fortunately, the underlying device is back to "probing" state while
enumerating partitions. Waiting for all probing to end is enough to fix
this issue.
That's also the reason why this problem never occurs with rootwait=
parameter: the while loop inside wait_for_root() explicitly waits for
probing to be done and then the function calls async_synchronize_full().
These lines were omitted in 035641b, even though the commit says it's
based on the rootwait logic...
Anyway, calling wait_for_device_probe() after our while loop does the
job (it both waits for probing and calls async_synchronize_full).
Fixes: 035641b01e ("dm init: add dm-mod.waitfor to wait for asynchronously probed block devices")
Signed-off-by: Guillaume Gonnet <ggonnet.linux@gmail.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Update the comments in and above fec_read_bufs() to more clearly
describe what it does.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The log message for a FEC error or correction includes the data device
name and index_in_region as the context. Although the result of FEC
(for a particular dm-verity instance) is expected to be the same for a
given index_in_region, index_in_region does not uniquely identify the
actual target block that is being corrected. Since that value
(target_block) is likely more useful, log it instead.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
fec_decode_bufs() returns the number of errors corrected or a negative
errno value. However, the caller just checks for an errno value and
doesn't do anything with the number of errors corrected. Simplify the
code by just returning 0 instead of the number of errors corrected.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Replace 'rsb', which is a byte index, with 'index_in_region' which is a
block index. The block index is slightly easier to compute, it matches
what fec_read_bufs() wants, and it avoids the mismatch between the name
and the units of the variable. ('rsb' stood for "Reed-Solomon block",
but its units were bytes, not blocks.)
fec_decode_bufs() does want it as a byte index when computing
parity_block, but that's easily handled locally.
As long as the parameters to the log messages are being adjusted, also
eliminate the unnecessary casts to 'unsigned long long'. %llu is the
correct way to print a u64 in the Linux kernel, as documented in
printk-formats.rst. There's no PRIu64 macro like there is in userspace.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Instead of determining the target block's region by checking which block
of the k blocks being iterated over in fec_read_bufs() is equal to the
target block, instead just directly use the quotient of the division of
target_block by region_blocks.
This is the same value, just derived in a more straightforward way.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
verity_fec_decode() computes (offset, rsb) from the target block index
and calls fec_decode_rsb() with these parameters. Move this computation
into fec_decode_rsb(), and rename fec_decode_rsb() to fec_decode().
This ends up being simpler and enables further refactoring, specifically
making use of the quotient from the division more easily. The function
renaming also eliminates a reference to the ambiguous term "rsb".
This change does mean the same div64_u64_rem() can now be executed twice
per block, since verity_fec_decode() calls fec_decode() up to twice per
block. However, this cost is negligible compared to the rest of FEC.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The current position in the output block buffer is called 'pos' in
fec_decode_rsb(), and 'block_offset' in fec_read_bufs() and
fec_decode_bufs(). These names aren't very clear, especially
'block_offset' which is easily confused with the offset of a message or
parity block or the position in the current parity block.
Rename it to 'out_pos'.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Since fec_read_bufs() deinterleaves the bytes from 'bbuf' sequentially
starting from 'block_offset', it can just do simple increments instead
of the more complex fec_buffer_rs_index() computation.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
fec_read_bufs() just iterates over a sequence of message blocks with
step size region_blocks. At each step, 'ileaved' is just the offset (in
bytes) to one of these blocks. Compute it in the straightforward way,
eliminating fec_interleave().
In more detail, previously the code computed
'ileaved = (n / k) + (n % k) * (region_blocks * block_size)'
where n = rsb * k + i and 0 <= i < k. Substituting 'n' gives:
ileaved = ((rsb * k + i) / k) + ((rsb * k + i) % k) * region_blocks * block_size
= rsb + (i * region_blocks * block_size)
The result is more efficient and easier to understand.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
To compute 'rsb', verity_fec_decode() divides 'offset' by
'v->fec->region_blocks << v->data_dev_block_bits', then subtracts the
quotient times that divisor. That's simply the long way to do a modulo
operation, i.e. a - b * floor(a / b) instead of just a % b. Use
div64_u64_rem() to get the remainder more concisely.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
It's hard to reconcile the value stored in dm_verity_fec::rounds with
its name and documentation. Most likely "rounds" is being used as an
alias for what is more commonly called the interleaving degree or
"number of ways". But the interleaving is done at the byte level,
whereas the units of "rounds" are blocks. So it's not really that.
In practice, the reason the code needs this value is that it expresses
the number of blocks in each "region" of the message data, where each
region contains the bytes from a particular index in the RS codewords.
Rename it to region_blocks to make the code a bit more understandable.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
dm-verity's FEC implementation assumes that data_block_size ==
hash_block_size, and it accesses the FEC device in units of the same
size. Many places in the code want that size and compute it on-demand
as '1 << v->data_dev_block_bits'. However, it's actually already
available in v->fec->io_size. Rename that field to block_size,
initialize it a bit earlier, and use it in the appropriate places.
Note that while these sizes could in principle be different, that case
is not supported. So there's no need to complicate the code for it.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The literature refers to the unit of a Reed-Solomon (RS) code as either
a "block" or a "codeword".
dm-verity's source code uses "RS block". Unfortunately, that's really
confusing because "block" already means something else in dm-verity.
Especially problematic is the fact that dm-verity sometimes uses "RS
block" to mean an RS codeword and sometimes to mean some dm-verity block
that's related to the RS decoding process, for example one of the blocks
that shares its RS codewords with the target block.
Let's use "RS codeword" instead, or "RS message" when referring to just
the message part of the codeword. Update some comments, function names,
macro names, and variable names accordingly. No functional change.
There are still some remaining comments where "RS block" refers to a
dm-verity block. Later commits will handle these cases.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
"RS(n, k)" is by far the most common and standard notation for
describing Reed-Solomon codes. Each RS codeword consists of 'n'
symbols, divided into 'k' message symbols and 'n - k' parity symbols.
'n - k' is also the number of roots of the generator polynomial.
dm-verity uses "RS(M, N)" instead. I haven't been able to find any
other source that uses this convention. This quirk makes the code
harder to understand than necessary, especially due to dm-verity's 'N'
meaning something different from the standard 'n'.
Therefore, update dm-verity-fec.c and dm-verity-fec.h to use the
standard parameter names. No functional changes.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Every time DM_VERITY_FEC_{MAX,MIN}_RSN are used, they are subtracted
from DM_VERITY_FEC_RSM to get the bounds on the number of roots.
Therefore, replace these with {MIN,MAX}_ROOTS constants which are more
directly useful. (Note the inversion, where MAX_RSN maps to MIN_ROOTS
and MIN_RSN maps to MAX_ROOTS.) No functional change.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Update verity.rst to add a dedicated section about FEC and improve the
documentation for the FEC-related parameters.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Rename hash_blocks to hash_end to reflect what it actually is.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
fec_decode_bufs() assumes that the parity bytes of the first RS codeword
it decodes are never split across parity blocks.
This assumption is false. Consider v->fec->block_size == 4096 &&
v->fec->roots == 17 && fio->nbufs == 1, for example. In that case, each
call to fec_decode_bufs() consumes v->fec->roots * (fio->nbufs <<
DM_VERITY_FEC_BUF_RS_BITS) = 272 parity bytes.
Considering that the parity data for each message block starts on a
block boundary, the byte alignment in the parity data will iterate
through 272*i mod 4096 until the 3 parity blocks have been consumed. On
the 16th call (i=15), the alignment will be 4080 bytes into the first
block. Only 16 bytes remain in that block, but 17 parity bytes will be
needed. The code reads out-of-bounds from the parity block buffer.
Fortunately this doesn't normally happen, since it can occur only for
certain non-default values of fec_roots *and* when the maximum number of
buffers couldn't be allocated due to low memory. For example with
block_size=4096 only the following cases are affected:
fec_roots=17: nbufs in [1, 3, 5, 15]
fec_roots=19: nbufs in [1, 229]
fec_roots=21: nbufs in [1, 3, 5, 13, 15, 39, 65, 195]
fec_roots=23: nbufs in [1, 89]
Regardless, fix it by refactoring how the parity blocks are read.
Fixes: 6df90c02ba ("dm-verity FEC: Fix RS FEC repair for roots unaligned to block size (take 2)")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
At most 25 entries in dm_verity_fec_io::erasures are used: the maximum
number of FEC roots plus one. Therefore, set the array size
accordingly. This reduces the size of dm_verity_fec_io by 912 bytes.
Note: a later commit introduces a constant DM_VERITY_FEC_MAX_ROOTS,
which allows the size to be more clearly expressed as
DM_VERITY_FEC_MAX_ROOTS + 1. This commit just fixes the size first.
Fixes: a739ff3f54 ("dm verity: add support for forward error correction")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
dm_verity_fec::corrected seems to have been intended to count the number
of corrected blocks. However, it actually counted the number of calls
to fec_decode_bufs() that corrected at least one error. That's not the
same thing. For example, in low-memory situations correcting a single
block can require many calls to fec_decode_bufs().
Fix it to count corrected blocks instead.
Fixes: ae97648e14 ("dm verity fec: Expose corrected block count via status")
Cc: Shubhankar Mishra <shubhankarm@google.com>
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fix verity_fec_ctr() to reject too-small hash devices by correctly
taking hash_start into account.
Note that this is necessary because dm-verity doesn't call
dm_bufio_set_sector_offset() on the hash device's bufio client
(v->bufio). Thus, dm_bufio_get_device_size(v->bufio) returns a size
relative to 0 rather than hash_start. An alternative fix would be to
call dm_bufio_set_sector_offset() on v->bufio, but then all the code
that reads from the hash device would have to be adjusted accordingly.
Fixes: a739ff3f54 ("dm verity: add support for forward error correction")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fix verity_fec_ctr() to reject too-small FEC devices by correctly
computing the number of parity blocks as 'f->rounds * f->roots'.
Previously it incorrectly used 'div64_u64(f->rounds * f->roots,
v->fec->roots << SECTOR_SHIFT)' which is a much smaller value.
Note that the units of 'rounds' are blocks, not bytes. This matches the
units of the value returned by dm_bufio_get_device_size(), which are
also blocks. A later commit will give 'rounds' a clearer name.
Fixes: a739ff3f54 ("dm verity: add support for forward error correction")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The local variable region_count in create_log_context() is declared as
unsigned int (32-bit), but dm_sector_div_up() returns sector_t (64-bit).
When a device-mapper target has a sufficiently large ti->len with a small
region_size, the division result can exceed UINT_MAX. The truncated
value is then used to calculate bitset_size, causing clean_bits,
sync_bits, and recovering_bits to be allocated far smaller than needed
for the actual number of regions.
Subsequent log operations (log_set_bit, log_clear_bit, log_test_bit) use
region indices derived from the full untruncated region space, causing
out-of-bounds writes to kernel heap memory allocated by vmalloc.
This can be reproduced by creating a mirror target whose region_count
overflows 32 bits:
dmsetup create bigzero --table '0 8589934594 zero'
dmsetup create mymirror --table '0 8589934594 mirror \
core 2 2 nosync 2 /dev/mapper/bigzero 0 \
/dev/mapper/bigzero 0'
The status output confirms the truncation (sync_count=1 instead of
4294967297, because 0x100000001 was truncated to 1):
$ dmsetup status mymirror
0 8589934594 mirror 2 254:1 254:1 1/4294967297 ...
This leads to a kernel crash in core_in_sync:
BUG: scheduling while atomic: (udev-worker)/9150/0x00000000
RIP: 0010:core_in_sync+0x14/0x30 [dm_log]
CR2: 0000000000000008
Fixing recursive fault but reboot is needed!
Fix by widening the local region_count to sector_t and adding an
explicit overflow check before the value is assigned to lc->region_count.
Fixes: 1da177e4c3 ("Linux-2.6.12-rc2")
Reported-by: Yuhao Jiang <danisjiang@gmail.com>
Signed-off-by: Junrui Luo <moonafterrain@outlook.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
When failing to acquire the root_lock in dm_cache_metadata_abort because
the block_manager is read-only, the temporary block_manager created
outside the root_lock is not properly released, causing a memory leak.
Reproduce steps:
This can be reproduced by reloading a new table while the metadata
is read-only. While the second call to dm_cache_metadata_abort is
caused by lack of support for table preload in dm-cache, mentioned
in commit 9b1cc9f251 ("dm cache: share cache-metadata object across
inactive and active DM tables"), it exposes the memory leak in
dm_cache_metadata_abort when the function is called multiple times.
Specifically, dm-cache fails to sync the new cache object's mode during
preresume, creating the reproducer condition.
This issue could also occur through concurrent metadata_operation_failed
calls due to races in cache mode updates, but the table preload scenario
below provides a reliable reproducer.
1. Create a cache device with some faulty trailing metadata blocks
dmsetup create cmeta <<EOF
0 200 linear /dev/sdc 0
200 7992 error
EOF
dmsetup create cdata --table "0 131072 linear /dev/sdc 8192"
dmsetup create corig --table "0 262144 linear /dev/sdc 262144"
dd if=/dev/zero of=/dev/mapper/cmeta bs=4k count=1 oflag=direct
dmsetup create cache --table "0 131072 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 1 writethrough smq 0"
2. Suspend and resume the cache to start a new metadata transaction and
trigger metadata io errors on the next metadata commit.
dmsetup suspend cache
dmsetup resume cache
3. Write to the cache device to update metadata
fio --filename=/dev/mapper/cache --name test --rw=randwrite --bs=4k \
--randrepeat=0 --direct=1 --size 64k
4. Preload the same table
dmsetup reload cache --table "$(dmsetup table cache)"
5. Resume the new table. This triggers the memory leak.
dmsetup suspend cache
dmsetup resume cache
kmemleak logs:
<snip>
unreferenced object 0xffff8880080c2010 (size 16):
comm "dmsetup", pid 132, jiffies 4294982580
hex dump (first 16 bytes):
00 38 b9 07 80 88 ff ff 6a 6b 6b 6b 6b 6b 6b a5 ...
backtrace (crc 3118f31c):
kmemleak_alloc+0x28/0x40
__kmalloc_cache_noprof+0x3d9/0x510
dm_block_manager_create+0x51/0x140
dm_cache_metadata_abort+0x85/0x320
metadata_operation_failed+0x103/0x1e0
cache_preresume+0xacd/0xe70
dm_table_resume_targets+0xd3/0x320
__dm_resume+0x1b/0xf0
dm_resume+0x127/0x170
<snip>
Fixes: 352b837a55 ("dm cache: Fix ABBA deadlock between shrink_slab and dm_cache_metadata_abort")
Signed-off-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The argument count calculation in create_dirty_log() performs
`*args_used = 2 + param_count` before validating against argc. When a
user provides a param_count close to UINT_MAX via the device mapper
table string, this unsigned addition wraps around to a small value,
causing the subsequent `argc < *args_used` check to be bypassed.
The overflowed param_count is then passed as argc to dm_dirty_log_create(),
where it can cause out-of-bounds reads on the argv array.
Fix by comparing param_count against argc - 2 before performing the
addition, following the same pattern used by parse_features() in the
same file. Since argc >= 2 is already guaranteed, the subtraction is
safe.
Fixes: 1da177e4c3 ("Linux-2.6.12-rc2")
Cc: stable@vger.kernel.org
Reported-by: Yuhao Jiang <danisjiang@gmail.com>
Signed-off-by: Junrui Luo <moonafterrain@outlook.com>
Reviewed-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
This attribute allows the compiler to refine compile-time diagnostics
and run-time sanitizer features with information about the size of the
flexible arrays.
Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
We can infer the type needed from the supplied pointer argument. A
couple invocation sites needed fixing to supply the proper type of
pointer.
Use overflow.h's size_mul, and we can remove the __vdo_do_allocation
wrapper which did the same overflow check.
Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
All of VDO's "extended" allocations use a flexible array field at the
end of the allocated structure. We can infer the struct type from the
supplied pointer. Replacing the array field type with the field name
lets us use struct_size from overflow.h to compute the size instead of
the local __vdo_do_allocation version.
One allocation of bio structures doesn't conform to this pattern,
since the removal of bi_inline_vecs; directly compute the total size
for that case.
Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Make dm_ima_measure_on_table_load() use the SHA-256 library API instead
of crypto_shash to calculate the SHA-256 hash value that it needs. This
is simpler and more efficient. It also ensures that SHA-256 is actually
available and doesn't fail due to the unreliable loading by name.
While doing this, also use kasprintf() to simplify building the string
version of the digest.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Commit 5c977f1023 ("dm-mpath: Don't grab work_mutex while probing
paths"), added code to make multipath quit probing paths early, if it
was trying to suspend. This isn't necessary. It was just an optimization
to try to keep path probing from delaying a suspend. However it causes
problems with the intended user of this code, qemu. The path probing
code was added because failed ioctls to multipath devices don't cause
paths to fail in cases where a regular IO failure would.
If an ioctl to a path failed because the path was down, and the
multipath device had passed presuspend, the M_MPATH_PROBE_PATHS ioctl
would exit early, without probing the path. The caller would then retry
the original ioctl, hoping to use a different path. But if there was
only one path in the pathgroup, it would pick the same non-working path
again, even if there were working paths in other pathgroups.
ioctls to a suspended dm device will return -EAGAIN, notifying the
caller that the device is suspended, but ioctls to a device that is just
preparing to suspend won't (and in general, shouldn't). This means that
the caller (qemu in this case) would get into a tight loop where it
would issue an ioctl that failed, skip probing the paths because the
device had already passed presuspend, and start over issuing the ioctl
again. This would continue until the multipath device finally fully
suspended, or the caller gave up and failed the ioctl.
multipath's path probing code could return -EAGAIN in this case, and the
caller could delay a bit before retrying, but the whole purpose of
skipping the probe after presuspend was to speed things up, and that
would just slow them down. Instead, remove the is_suspending flag, and
check dm_suspended() instead to decide whether to exit the probing code
early. This means that when the probing code exits early, future ioctls
will also be delayed, because the device is fully suspended.
Fixes: 5c977f1023 ("dm-mpath: Don't grab work_mutex while probing paths")
Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
Reviewed-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
dm-cache assumes all cache blocks are dirty when it recovers from an
unclean shutdown. Given that the passthrough mode doesn't handle dirty
blocks, we should not load a cache in passthrough mode if it was not
cleanly shut down; or we'll risk data loss while updating an actually
dirty block.
Also bump the target version to 2.4.0 to mark completion of passthrough
mode fixes.
Reproduce steps:
1. Create a writeback cache with zero migration_threshold to produce
dirty blocks.
dmsetup create cmeta --table "0 8192 linear /dev/sdc 0"
dmsetup create cdata --table "0 131072 linear /dev/sdc 8192"
dmsetup create corig --table "0 262144 linear /dev/sdc 262144"
dd if=/dev/zero of=/dev/mapper/cmeta bs=4k count=1 oflag=direct
dmsetup create cache --table "0 262144 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 writeback smq \
2 migration_threshold 0"
2. Write the first cache block dirty
fio --filename=/dev/mapper/cache --name=populate --rw=write --bs=4k \
--direct=1 --size=64k
3. Ensure the number of dirty blocks is 1. This status query triggers
metadata commit without flushing the dirty bitset, setting up the
unclean shutdown state.
dmsetup status cache | awk '{print $14}'
4. Force reboot, leaving the cache uncleanly shutdown.
echo b > /proc/sysrq-trigger
5. Activate the above cache components, and verify the first data block
remains dirty.
dmsetup create cmeta --table "0 8192 linear /dev/sdc 0"
dmsetup create cdata --table "0 131072 linear /dev/sdc 8192"
dmsetup create corig --table "0 262144 linear /dev/sdc 262144"
dd if=/dev/mapper/cdata of=/tmp/cb0.bin bs=64k count=1
dd if=/dev/mapper/corig of=/tmp/ob0.bin bs=64k count=1
md5sum /tmp/cb0.bin /tmp/ob0.bin # expected to be different
6. Try bringing up the cache in passthrough mode. It succeeds, while the
first cache block was loaded dirty due to unclean shutdown, violates
the passthrough mode's constraints.
dmsetup create cache --table "0 262144 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 passthrough smq 0"
dmsetup status cache | awk '{print $14}'
7. (Optional) Demonstrate the integrity issue: invalidating the dirty
block in passthrough mode doesn't write back the dirty data, causing
data loss.
fio --filename=/dev/mapper/cache --name=invalidate --rw=write --bs=4k \
--direct=1 --size=4k # overwrite the first 4k to trigger invalidation
dmsetup remove cache
dd if=/dev/mapper/corig of=/tmp/ob0new.bin bs=64k count=1
cb0sum=$(dd if=/tmp/cb0.bin bs=4k count=15 skip=1 | md5sum | \
awk '{print $1}')
ob0newsum=$(dd if=/tmp/ob0new.bin bs=4k count=15 skip=1 | md5sum | \
awk '{print $1}')
echo "$cb0sum, $ob0newsum" # remaining 60k should differ (data loss)
Signed-off-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
As mentioned in commit 9b1cc9f251 ("dm cache: share cache-metadata
object across inactive and active DM tables"), dm-cache assumed table
reload occurs after suspension, while LVM's table preload breaks this
assumption. The dirty mapping check for passthrough mode was designed
around this assumption and is performed during table creation, causing
the check to fail with preload while metadata updates are ongoing. This
risks loading dirty mappings into passthrough mode, resulting in data
loss.
Reproduce steps:
1. Create a writeback cache with zero migration_threshold to produce
dirty mappings
dmsetup create cmeta --table "0 8192 linear /dev/sdc 0"
dmsetup create cdata --table "0 131072 linear /dev/sdc 8192"
dmsetup create corig --table "0 262144 linear /dev/sdc 262144"
dd if=/dev/zero of=/dev/mapper/cmeta bs=4k count=1 oflag=direct
dmsetup create cache --table "0 262144 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 writeback smq \
2 migration_threshold 0"
2. Preload a table in passthrough mode
dmsetup reload cache --table "0 262144 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 passthrough smq 0"
3. Write to the first cache block to make it dirty
fio --filename=/dev/mapper/cache --name=populate --rw=write --bs=4k \
--direct=1 --size=64k
4. Resume the inactive table. Now it's possible to load the dirty block
into passthrough mode.
dmsetup resume cache
Fix by moving the checks to the preresume phase to support table
preloading. Also remove the unused function dm_cache_metadata_all_clean.
Fixes: 2ee57d5873 ("dm cache: add passthrough mode")
Signed-off-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
In passthrough mode, the policy invalidate_mapping operation is called
simultaneously from multiple workers, thus it should be protected by a
lock. Otherwise, we might end up with data races on the allocated blocks
counter, or even use-after-free issues with internal data structures
when doing concurrent writes.
Note that the existing FIXME in smq_invalidate_mapping() doesn't affect
passthrough mode since migration tasks don't exist there, but would need
attention if supporting fast device shrinking via suspend/resume without
target reloading.
Reproduce steps:
1. Create a cache device consisting of 1024 cache entries
dmsetup create cmeta --table "0 8192 linear /dev/sdc 0"
dmsetup create cdata --table "0 131072 linear /dev/sdc 8192"
dmsetup create corig --table "0 262144 linear /dev/sdc 262144"
dd if=/dev/zero of=/dev/mapper/cmeta bs=4k count=1 oflag=direct
dmsetup create cache --table "0 262144 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 writethrough smq 0"
2. Populate the cache, and record the number of cached blocks
fio --name=populate --filename=/dev/mapper/cache --rw=randwrite --bs=4k \
--size=64m --direct=1
nr_cached=$(dmsetup status cache | awk '{split($7, a, "/"); print a[1]}')
3. Reload the cache into passthrough mode
dmsetup suspend cache
dmsetup reload cache --table "0 262144 cache /dev/mapper/cmeta \
/dev/mapper/cdata /dev/mapper/corig 128 2 metadata2 passthrough smq 0"
dmsetup resume cache
4. Write to the passthrough cache. By setting multiple jobs with I/O
size equal to the cache block size, cache blocks are invalidated
concurrently from different workers.
fio --filename=/dev/mapper/cache --name=test --rw=randwrite --bs=64k \
--direct=1 --numjobs=2 --randrepeat=0 --size=64m
5. Check if demoted matches cached block count. These numbers should
match but may differ due to the data race.
nr_demoted=$(dmsetup status cache | awk '{print $12}')
echo "$nr_cached, $nr_demoted"
Fixes: b29d4986d0 ("dm cache: significant rework to leverage dm-bio-prison-v2")
Signed-off-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
The invalidate_remove() function has incomplete logic for handling write
hit bios after cache invalidation. It sets up the remapping for the
overwrite_bio but then drops it immediately without submission, causing
write operations to hang.
Fix by adding a new invalidate_committed() continuation that submits
the remapped writes to the cache origin after metadata commit completes,
while using the overwrite_endio hook to ensure proper completion
sequencing. This maintains existing coherency. Also improve error
handling in invalidate_complete() to preserve the original error status
instead of using bio_io_error() unconditionally.
Fixes: b29d4986d0 ("dm cache: significant rework to leverage dm-bio-prison-v2")
Signed-off-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
