This is the counterpart to ex-set_bit_hook (now btrfs_set_delalloc_extent),
similar to what was done before remove clear_bit_hook and rename the
function. No functional changes.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This callback is used to properly account delalloc extents for data
inodes (ordinary file inodes and freespace v1 inodes). Those can be
easily identified since they have their extent_io trees ->private_data
member point to the inode. Let's exploit this fact to remove the
needless indirection through extent_io_hooks and directly call the
function. Also give the function a name which reflects its purpose -
btrfs_set_delalloc_extent.
This patch also modified test_find_delalloc so that the extent_io_tree
used for testing doesn't have its ->private_data set which would have
caused a crash in btrfs_set_delalloc_extent due to the btrfs_inode->root
member not being initialised. The old version of the code also didn't
call set_bit_hook since the extent_io ops weren't set for the inode. No
functional changes.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This callback was only used in debug builds by btrfs_leak_debug_check.
A better approach is to move its implementation in
btrfs_leak_debug_check and ensure the latter is only executed for extent
tree which have ->private_data set i.e. relate to a data node and not
the btree one. No functional changes.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This callback is ony ever called for data page writeout so there is no
need to actually abstract it via extent_io_ops. Lets just export it,
remove the definition of the callback and call it directly in the
functions that invoke the callback. Also rename the function to
btrfs_writepage_endio_finish_ordered since what it really does is
account finished io in the ordered extent data structures. No
functional changes.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This hook is called only from __extent_writepage_io which is already
called only from the data page writeout path. So there is no need to
make an indirect call via extent_io_ops. This patch just removes the
callback definition, exports the callback function and calls it directly
at the only call site. Also give the function a more descriptive name.
No functional changes.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This callback is called only from writepage_delalloc which in turn is
guaranteed to be called from the data page writeout path. In the end
there is no reason to have the call to this function to be indrected via
the extent_io_ops structure. This patch removes the callback definition,
exports the function and calls it directly. No functional changes.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
[ rename to btrfs_run_delalloc_range ]
Signed-off-by: David Sterba <dsterba@suse.com>
This will be used in future patches that remove the optional
extent_io_ops callbacks.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add extra dev extent end check against device boundary.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance btrfs_verify_dev_extents() to remember previous checked dev
extents, so it can verify no dev extents can overlap.
Analysis from Hans:
"Imagine allocating a DATA|DUP chunk.
In the chunk allocator, we first set...
max_stripe_size = SZ_1G;
max_chunk_size = BTRFS_MAX_DATA_CHUNK_SIZE
... which is 10GiB.
Then...
/* we don't want a chunk larger than 10% of writeable space */
max_chunk_size = min(div_factor(fs_devices->total_rw_bytes, 1),
max_chunk_size);
Imagine we only have one 7880MiB block device in this filesystem. Now
max_chunk_size is down to 788MiB.
The next step in the code is to search for max_stripe_size * dev_stripes
amount of free space on the device, which is in our example 1GiB * 2 =
2GiB. Imagine the device has exactly 1578MiB free in one contiguous
piece. This amount of bytes will be put in devices_info[ndevs - 1].max_avail
Next we recalculate the stripe_size (which is actually the device extent
length), based on the actual maximum amount of available raw disk space:
stripe_size = div_u64(devices_info[ndevs - 1].max_avail, dev_stripes);
stripe_size is now 789MiB
Next we do...
data_stripes = num_stripes / ncopies
...where data_stripes ends up as 1, because num_stripes is 2 (the amount
of device extents we're going to have), and DUP has ncopies 2.
Next there's a check...
if (stripe_size * data_stripes > max_chunk_size)
...which matches because 789MiB * 1 > 788MiB.
We go into the if code, and next is...
stripe_size = div_u64(max_chunk_size, data_stripes);
...which resets stripe_size to max_chunk_size: 788MiB
Next is a fun one...
/* bump the answer up to a 16MB boundary */
stripe_size = round_up(stripe_size, SZ_16M);
...which changes stripe_size from 788MiB to 800MiB.
We're not done changing stripe_size yet...
/* But don't go higher than the limits we found while searching
* for free extents
*/
stripe_size = min(devices_info[ndevs - 1].max_avail,
stripe_size);
This is bad. max_avail is twice the stripe_size (we need to fit 2 device
extents on the same device for DUP).
The result here is that 800MiB < 1578MiB, so it's unchanged. However,
the resulting DUP chunk will need 1600MiB disk space, which isn't there,
and the second dev_extent might extend into the next thing (next
dev_extent? end of device?) for 22MiB.
The last shown line of code relies on a situation where there's twice
the value of stripe_size present as value for the variable stripe_size
when it's DUP. This was actually the case before commit 92e222df7b
"btrfs: alloc_chunk: fix DUP stripe size handling", from which I quote:
"[...] in the meantime there's a check to see if the stripe_size does
not exceed max_chunk_size. Since during this check stripe_size is twice
the amount as intended, the check will reduce the stripe_size to
max_chunk_size if the actual correct to be used stripe_size is more than
half the amount of max_chunk_size."
In the previous version of the code, the 16MiB alignment (why is this
done, by the way?) would result in a 50% chance that it would actually
do an 8MiB alignment for the individual dev_extents, since it was
operating on double the size. Does this matter?
Does it matter that stripe_size can be set to anything which is not
16MiB aligned because of the amount of remaining available disk space
which is just taken?
What is the main purpose of this round_up?
The most straightforward thing to do seems something like...
stripe_size = min(
div_u64(devices_info[ndevs - 1].max_avail, dev_stripes),
stripe_size
)
..just putting half of the max_avail into stripe_size."
Link: https://lore.kernel.org/linux-btrfs/b3461a38-e5f8-f41d-c67c-2efac8129054@mendix.com/
Reported-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ add analysis from report ]
Signed-off-by: David Sterba <dsterba@suse.com>
We have a complex loop design for find_free_extent(), that has different
behavior for each loop, some even includes new chunk allocation.
Instead of putting such a long code into find_free_extent() and makes it
harder to read, just extract them into find_free_extent_update_loop().
With all the cleanups, the main find_free_extent() should be pretty
barebone:
find_free_extent()
|- Iterate through all block groups
| |- Get a valid block group
| |- Try to do clustered allocation in that block group
| |- Try to do unclustered allocation in that block group
| |- Check if the result is valid
| | |- If valid, then exit
| |- Jump to next block group
|
|- Push harder to find free extents
|- If not found, re-iterate all block groups
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
[ copy callchain from changelog to function comment ]
Signed-off-by: David Sterba <dsterba@suse.com>
This patch will extract unclsutered extent allocation code into
find_free_extent_unclustered().
And this helper function will use return value to indicate what to do
next.
This should make find_free_extent() a little easier to read.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
[Update merge conflict with fb5c39d7a8 ("btrfs: don't use ctl->free_space for max_extent_size")]
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We have two main methods to find free extents inside a block group:
1) clustered allocation
2) unclustered allocation
This patch will extract the clustered allocation into
find_free_extent_clustered() to make it a little easier to read.
Instead of jumping between different labels in find_free_extent(), the
helper function will use return value to indicate different behavior.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Instead of tons of different local variables in find_free_extent(),
extract them into find_free_extent_ctl structure, and add better
explanation for them.
Some modification may looks redundant, but will later greatly simplify
function parameter list during find_free_extent() refactor.
Also add two comments to co-operate with fb5c39d7a8 ("btrfs: don't use
ctl->free_space for max_extent_size"), to make ffe_ctl->max_extent_size
update more reader-friendly.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new wrapper update_bytes_pinned to replace open coded
bytes_pinned modifiers. Now the underflows of space_info::bytes_pinned
get detected and reported.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Although we have space_info::bytes_may_use underflow detection in
btrfs_free_reserved_data_space_noquota(), we have more callers who are
subtracting number from space_info::bytes_may_use.
So instead of doing underflow detection for every caller, introduce a
new wrapper update_bytes_may_use() to replace open coded bytes_may_use
modifiers.
This also introduce a macro to declare more wrappers, but currently
space_info::bytes_may_use is the mostly interesting one.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Tracking pending ordered extents per transaction was introduced in commit
50d9aa99bd ("Btrfs: make sure logged extents complete in the current
transaction V3") and later updated in commit 161c3549b4 ("Btrfs: change
how we wait for pending ordered extents").
However now that on fsync we always wait for ordered extents to complete
before logging, done in commit 5636cf7d6d ("btrfs: remove the logged
extents infrastructure"), we no longer need the stuff to track for pending
ordered extents, which was not completely removed in the mentioned commit.
So remove the remaining of the pending ordered extents infrastructure.
Reviewed-by: Liu Bo <bo.liu@linux.alibaba.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The logged_start and logged_end variables, at btrfs_log_changed_extents,
were added in commit 8c6c592831 ("btrfs: log csums for all modified
extents"). However since the recent simplification for fsync, which makes
us wait for all ordered extents to complete before logging extents, we
no longer need those variables. Commit a2120a473a ("btrfs: clean up the
left over logged_list usage") forgot to remove them.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Typedefing structs is not encouraged in the kernel.
Signed-off-by: Dominic Braun <inf.braun@fau.de>
Signed-off-by: Tobias Büttner <tobias.buettner@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>