2019-05-31 01:09:56 -07:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2008-01-31 10:31:39 -06:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-02-27 17:23:27 -05:00
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
2009-10-02 11:54:39 +01:00
|
|
|
|
2015-12-24 11:09:40 -05:00
|
|
|
|
2025-07-16 14:12:07 +01:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-02-27 17:23:27 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-02-27 17:23:27 -05:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2011-06-15 10:29:37 +01:00
|
|
|
|
2019-05-02 14:17:40 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2014-11-13 20:42:04 -06:00
|
|
|
|
|
|
|
|
|
2011-08-02 13:09:36 +01:00
|
|
|
|
|
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2021-05-24 11:51:26 -05:00
|
|
|
|
|
|
|
|
|
2015-03-16 11:52:05 -05:00
|
|
|
|
|
|
|
|
|
2011-08-02 13:09:36 +01:00
|
|
|
|
2025-08-05 18:22:51 +01:00
|
|
|
|
2021-05-24 11:51:26 -05:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2011-08-02 13:09:36 +01:00
|
|
|
|
2021-05-24 11:51:26 -05:00
|
|
|
|
2025-08-05 23:32:39 +02:00
|
|
|
|
2011-08-02 13:09:36 +01:00
|
|
|
|
|
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2011-09-07 10:33:25 +01:00
|
|
|
|
2021-03-30 17:44:29 +01:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-26 17:09:33 -05:00
|
|
|
|
|
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2011-09-07 10:33:25 +01:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2011-09-07 10:33:25 +01:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2026-03-20 16:43:15 +01:00
|
|
|
|
2011-03-11 11:52:25 +00:00
|
|
|
|
2013-07-26 17:09:33 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2011-09-07 10:33:25 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-08-02 13:09:36 +01:00
|
|
|
|
2011-09-07 10:33:25 +01:00
|
|
|
|
2007-09-03 11:01:33 +01:00
|
|
|
|
2013-07-26 17:09:33 -05:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2012-10-15 10:57:02 +01:00
|
|
|
|
2011-03-11 11:52:25 +00:00
|
|
|
|
2026-03-20 16:43:15 +01:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2021-01-29 16:45:33 +01:00
|
|
|
|
2023-04-21 15:07:07 -04:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2021-01-29 16:45:33 +01:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2021-01-29 16:45:33 +01:00
|
|
|
|
2019-11-13 13:47:02 -06:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-03-20 16:43:15 +01:00
|
|
|
|
2019-11-13 13:47:02 -06:00
|
|
|
|
|
|
|
|
|
2026-03-20 16:43:15 +01:00
|
|
|
|
2019-11-13 13:47:02 -06:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
2019-11-13 13:47:02 -06:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2021-01-29 16:45:33 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-04-21 15:07:09 -04:00
|
|
|
|
|
|
|
|
|
2021-01-29 16:45:33 +01:00
|
|
|
|
2023-04-21 15:07:09 -04:00
|
|
|
|
2021-01-29 16:45:33 +01:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2021-01-29 16:45:33 +01:00
|
|
|
|
2019-11-13 13:47:02 -06:00
|
|
|
|
2023-04-21 15:07:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-04-21 15:07:07 -04:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
|
|
|
|
|
2011-09-07 10:33:25 +01:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
gfs2: Per-revoke accounting in transactions
In the log, revokes are stored as a revoke descriptor (struct
gfs2_log_descriptor), followed by zero or more additional revoke blocks
(struct gfs2_meta_header). On filesystems with a blocksize of 4k, the
revoke descriptor contains up to 503 revokes, and the metadata blocks
contain up to 509 revokes each. We've so far been reserving space for
revokes in transactions in block granularity, so a lot more space than
necessary was being allocated and then released again.
This patch switches to assigning revokes to transactions individually
instead. Initially, space for the revoke descriptor is reserved and
handed out to transactions. When more revokes than that are reserved,
additional revoke blocks are added. When the log is flushed, the space
for the additional revoke blocks is released, but we keep the space for
the revoke descriptor block allocated.
Transactions may still reserve more revokes than they will actually need
in the end, but now we won't overshoot the target as much, and by only
returning the space for excess revokes at log flush time, we further
reduce the amount of contention between processes.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2020-12-17 16:14:30 +01:00
|
|
|
|
2011-04-14 09:54:02 +01:00
|
|
|
|
|
|
|
|
|
gfs2: Per-revoke accounting in transactions
In the log, revokes are stored as a revoke descriptor (struct
gfs2_log_descriptor), followed by zero or more additional revoke blocks
(struct gfs2_meta_header). On filesystems with a blocksize of 4k, the
revoke descriptor contains up to 503 revokes, and the metadata blocks
contain up to 509 revokes each. We've so far been reserving space for
revokes in transactions in block granularity, so a lot more space than
necessary was being allocated and then released again.
This patch switches to assigning revokes to transactions individually
instead. Initially, space for the revoke descriptor is reserved and
handed out to transactions. When more revokes than that are reserved,
additional revoke blocks are added. When the log is flushed, the space
for the additional revoke blocks is released, but we keep the space for
the revoke descriptor block allocated.
Transactions may still reserve more revokes than they will actually need
in the end, but now we won't overshoot the target as much, and by only
returning the space for excess revokes at log flush time, we further
reduce the amount of contention between processes.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2020-12-17 16:14:30 +01:00
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2018-01-08 10:34:17 -05:00
|
|
|
|
|
|
|
|
|
2006-10-03 11:10:41 -04:00
|
|
|
|
2006-07-26 11:27:10 -04:00
|
|
|
|
2020-10-27 12:29:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2025-04-06 00:31:37 +02:00
|
|
|
|
2020-10-27 12:29:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-08-06 17:30:40 +02:00
|
|
|
|
2020-10-27 12:29:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-07-26 11:27:10 -04:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-30 07:55:08 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
gfs2: Rework how rgrp buffer_heads are managed
Before this patch, the rgrp code had a serious problem related to
how it managed buffer_heads for resource groups. The problem caused
file system corruption, especially in cases of journal replay.
When an rgrp glock was demoted to transfer ownership to a
different cluster node, do_xmote() first calls rgrp_go_sync and then
rgrp_go_inval, as expected. When it calls rgrp_go_sync, that called
gfs2_rgrp_brelse() that dropped the buffer_head reference count.
In most cases, the reference count went to zero, which is right.
However, there were other places where the buffers are handled
differently.
After rgrp_go_sync, do_xmote called rgrp_go_inval which called
gfs2_rgrp_brelse a second time, then rgrp_go_inval's call to
truncate_inode_pages_range would get rid of the pages in memory,
but only if the reference count drops to 0.
Unfortunately, gfs2_rgrp_brelse was setting bi->bi_bh = NULL.
So when rgrp_go_sync called gfs2_rgrp_brelse, it lost the pointer
to the buffer_heads in cases where the reference count was still 1.
Therefore, when rgrp_go_inval called gfs2_rgrp_brelse a second time,
it failed the check for "if (bi->bi_bh)" and thus failed to call
brelse a second time. Because of that, the reference count on those
buffers sometimes failed to drop from 1 to 0. And that caused
function truncate_inode_pages_range to keep the pages in page cache
rather than freeing them.
The next time the rgrp glock was acquired, the metadata read of
the rgrp buffers re-used the pages in memory, which were now
wrong because they were likely modified by the other node who
acquired the glock in EX (which is why we demoted the glock).
This re-use of the page cache caused corruption because changes
made by the other nodes were never seen, so the bitmaps were
inaccurate.
For some reason, the problem became most apparent when journal
replay forced the replay of rgrps in memory, which caused newer
rgrp data to be overwritten by the older in-core pages.
A big part of the problem was that the rgrp buffer were released
in multiple places: The go_unlock function would release them when
the glock was released rather than when the glock is demoted,
which is clearly wrong because our intent was to cache them until
the glock is demoted from SH or EX.
This patch attempts to clean up the mess and make one consistent
and centralized mechanism for managing the rgrp buffer_heads by
implementing several changes:
1. It eliminates the call to gfs2_rgrp_brelse() from rgrp_go_sync.
We don't want to release the buffers or zero the pointers when
syncing for the reasons stated above. It only makes sense to
release them when the glock is actually invalidated (go_inval).
And when we do, then we set the bh pointers to NULL.
2. The go_unlock function (which was only used for rgrps) is
eliminated, as we've talked about doing many times before.
The go_unlock function was called too early in the glock dq
process, and should not happen until the glock is invalidated.
3. It also eliminates the call to rgrp_brelse in gfs2_clear_rgrpd.
That will now happen automatically when the rgrp glocks are
demoted, and shouldn't happen any sooner or later than that.
Instead, function gfs2_clear_rgrpd has been modified to demote
the rgrp glocks, and therefore, free those pages, before the
remaining glocks are culled by gfs2_gl_hash_clear. This
prevents the gl_object from hanging around when the glocks are
culled.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
Reviewed-by: Andreas Gruenbacher <agruenba@redhat.com>
2019-11-13 11:50:30 -06:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2007-01-22 12:15:34 -05:00
|
|
|
|
2023-01-24 14:55:18 -05:00
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
2012-10-15 10:57:02 +01:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2018-01-08 10:34:17 -05:00
|
|
|
|
|
|
|
|
|
2020-10-27 12:29:37 -05:00
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
|
|
|
|
|
2020-10-15 11:07:26 -05:00
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2025-04-06 00:31:37 +02:00
|
|
|
|
2017-06-30 07:55:08 -05:00
|
|
|
|
2020-10-15 11:07:26 -05:00
|
|
|
|
2023-01-24 14:55:18 -05:00
|
|
|
|
2015-06-05 08:38:57 -05:00
|
|
|
|
2023-01-24 14:55:18 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-10-15 11:07:26 -05:00
|
|
|
|
2012-10-15 10:57:02 +01:00
|
|
|
|
2025-08-09 22:04:00 +02:00
|
|
|
|
2020-10-15 11:07:26 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2023-06-21 22:32:06 +02:00
|
|
|
|
2020-10-07 12:30:58 +01:00
|
|
|
|
|
|
|
|
|
2020-11-22 18:10:24 -05:00
|
|
|
|
2020-10-07 12:30:58 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-30 07:55:08 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-01-22 12:15:34 -05:00
|
|
|
|
2020-10-27 12:29:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2020-10-27 12:29:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-01-22 12:15:34 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
2007-01-22 12:15:34 -05:00
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
|
|
|
|
|
2009-12-08 12:12:13 +00:00
|
|
|
|
2020-05-08 09:18:03 -05:00
|
|
|
|
2007-11-02 08:39:34 +00:00
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
2013-12-19 11:04:14 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
2007-01-22 12:15:34 -05:00
|
|
|
|
2012-10-15 10:57:02 +01:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2018-01-08 10:34:17 -05:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-01-22 12:15:34 -05:00
|
|
|
|
2020-10-27 12:29:37 -05:00
|
|
|
|
2020-05-08 09:18:03 -05:00
|
|
|
|
|
|
|
|
|
2023-04-21 15:07:07 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-04-20 08:58:45 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-03-17 18:06:10 +01:00
|
|
|
|
2009-04-20 08:58:45 +01:00
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-11-13 14:09:28 -06:00
|
|
|
|
2007-01-22 12:15:34 -05:00
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-06-29 12:21:39 +02:00
|
|
|
|
|
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2025-08-09 22:04:00 +02:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2009-12-08 12:12:13 +00:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2009-10-02 11:54:39 +01:00
|
|
|
|
gfs2: fix GL_SKIP node_scope problems
Before this patch, when a glock was locked, the very first holder on the
queue would unlock the lockref and call the go_instantiate glops function
(if one existed), unless GL_SKIP was specified. When we introduced the new
node-scope concept, we allowed multiple holders to lock glocks in EX mode
and share the lock.
But node-scope introduced a new problem: if the first holder has GL_SKIP
and the next one does NOT, since it is not the first holder on the queue,
the go_instantiate op was not called. Eventually the GL_SKIP holder may
call the instantiate sub-function (e.g. gfs2_rgrp_bh_get) but there was
still a window of time in which another non-GL_SKIP holder assumes the
instantiate function had been called by the first holder. In the case of
rgrp glocks, this led to a NULL pointer dereference on the buffer_heads.
This patch tries to fix the problem by introducing two new glock flags:
GLF_INSTANTIATE_NEEDED, which keeps track of when the instantiate function
needs to be called to "fill in" or "read in" the object before it is
referenced.
GLF_INSTANTIATE_IN_PROG which is used to determine when a process is
in the process of reading in the object. Whenever a function needs to
reference the object, it checks the GLF_INSTANTIATE_NEEDED flag, and if
set, it sets GLF_INSTANTIATE_IN_PROG and calls the glops "go_instantiate"
function.
As before, the gl_lockref spin_lock is unlocked during the IO operation,
which may take a relatively long amount of time to complete. While
unlocked, if another process determines go_instantiate is still needed,
it sees GLF_INSTANTIATE_IN_PROG is set, and waits for the go_instantiate
glop operation to be completed. Once GLF_INSTANTIATE_IN_PROG is cleared,
it needs to check GLF_INSTANTIATE_NEEDED again because the other process's
go_instantiate operation may not have been successful.
Functions that previously called the instantiate sub-functions now call
directly into gfs2_instantiate so the new bits are managed properly.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2021-10-06 09:29:18 -05:00
|
|
|
|
2009-10-02 11:54:39 +01:00
|
|
|
|
2015-12-24 11:09:40 -05:00
|
|
|
|
2011-06-15 10:29:37 +01:00
|
|
|
|
2009-10-02 11:54:39 +01:00
|
|
|
|
2006-11-23 10:51:34 -05:00
|
|
|
|
|
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
|
|
|
|
|
2018-01-08 10:34:17 -05:00
|
|
|
|
|
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2011-06-13 14:27:40 -05:00
|
|
|
|
2007-10-15 15:40:33 +01:00
|
|
|
|
2006-11-23 10:51:34 -05:00
|
|
|
|
2017-06-30 07:47:15 -05:00
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
|
|
|
|
|
2023-03-28 00:43:16 +02:00
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
2023-10-04 14:52:25 -04:00
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
2021-02-12 13:22:38 -05:00
|
|
|
|
2022-12-04 16:50:41 +01:00
|
|
|
|
2025-10-09 09:59:24 +02:00
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
2024-01-11 17:42:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
2022-12-04 16:50:41 +01:00
|
|
|
|
2021-02-12 13:22:38 -05:00
|
|
|
|
2022-12-04 16:50:41 +01:00
|
|
|
|
2021-02-12 13:22:38 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-12-04 16:50:41 +01:00
|
|
|
|
|
|
|
|
|
2021-02-12 13:22:38 -05:00
|
|
|
|
|
|
|
|
|
2019-10-04 10:55:29 -05:00
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
2022-12-04 16:50:41 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
|
|
|
|
|
2023-10-04 14:52:25 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-07-05 15:01:12 -04:00
|
|
|
|
|
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-06-16 14:06:55 +01:00
|
|
|
|
|
|
|
|
|
2022-12-04 16:50:41 +01:00
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
2024-01-11 17:42:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-01-11 17:42:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-07-16 14:12:07 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-03-26 22:56:26 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-01-11 17:42:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-12-04 16:50:41 +01:00
|
|
|
|
|
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-16 14:02:41 +02:00
|
|
|
|
2011-05-09 13:49:59 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2021-09-29 15:06:21 -05:00
|
|
|
|
2023-11-13 16:54:59 +01:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-06-10 12:06:06 +02:00
|
|
|
|
2022-06-10 11:42:33 +02:00
|
|
|
|
|
|
|
|
|
2024-09-16 15:42:57 +02:00
|
|
|
|
|
|
|
|
|
2022-06-10 11:42:33 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-16 15:42:57 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-06-10 11:42:33 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-02-27 17:23:27 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
gfs2: fix GL_SKIP node_scope problems
Before this patch, when a glock was locked, the very first holder on the
queue would unlock the lockref and call the go_instantiate glops function
(if one existed), unless GL_SKIP was specified. When we introduced the new
node-scope concept, we allowed multiple holders to lock glocks in EX mode
and share the lock.
But node-scope introduced a new problem: if the first holder has GL_SKIP
and the next one does NOT, since it is not the first holder on the queue,
the go_instantiate op was not called. Eventually the GL_SKIP holder may
call the instantiate sub-function (e.g. gfs2_rgrp_bh_get) but there was
still a window of time in which another non-GL_SKIP holder assumes the
instantiate function had been called by the first holder. In the case of
rgrp glocks, this led to a NULL pointer dereference on the buffer_heads.
This patch tries to fix the problem by introducing two new glock flags:
GLF_INSTANTIATE_NEEDED, which keeps track of when the instantiate function
needs to be called to "fill in" or "read in" the object before it is
referenced.
GLF_INSTANTIATE_IN_PROG which is used to determine when a process is
in the process of reading in the object. Whenever a function needs to
reference the object, it checks the GLF_INSTANTIATE_NEEDED flag, and if
set, it sets GLF_INSTANTIATE_IN_PROG and calls the glops "go_instantiate"
function.
As before, the gl_lockref spin_lock is unlocked during the IO operation,
which may take a relatively long amount of time to complete. While
unlocked, if another process determines go_instantiate is still needed,
it sees GLF_INSTANTIATE_IN_PROG is set, and waits for the go_instantiate
glop operation to be completed. Once GLF_INSTANTIATE_IN_PROG is cleared,
it needs to check GLF_INSTANTIATE_NEEDED again because the other process's
go_instantiate operation may not have been successful.
Functions that previously called the instantiate sub-functions now call
directly into gfs2_instantiate so the new bits are managed properly.
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2021-10-06 09:29:18 -05:00
|
|
|
|
2022-06-10 11:42:33 +02:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2013-12-19 11:04:14 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-04 10:05:22 +00:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2022-06-02 15:15:02 -05:00
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-03-30 17:44:29 +01:00
|
|
|
|
2019-05-09 09:21:48 -05:00
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-21 22:32:06 +02:00
|
|
|
|
2019-05-09 09:21:48 -05:00
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
2018-04-18 12:05:01 -07:00
|
|
|
|
2023-06-21 22:32:06 +02:00
|
|
|
|
2018-04-18 12:05:01 -07:00
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
2014-01-16 10:31:13 +00:00
|
|
|
|
2018-04-18 12:05:01 -07:00
|
|
|
|
2019-05-09 09:21:48 -05:00
|
|
|
|
|
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
|
|
|
|
|
2023-06-21 22:32:06 +02:00
|
|
|
|
2008-11-10 10:10:12 +00:00
|
|
|
|
2023-06-21 22:32:06 +02:00
|
|
|
|
|
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-09-11 20:00:28 +02:00
|
|
|
|
|
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2020-11-18 08:54:31 -05:00
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
2023-09-11 20:00:28 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-11-18 08:54:31 -05:00
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
GFS2: remove transaction glock
GFS2 has a transaction glock, which must be grabbed for every
transaction, whose purpose is to deal with freezing the filesystem.
Aside from this involving a large amount of locking, it is very easy to
make the current fsfreeze code hang on unfreezing.
This patch rewrites how gfs2 handles freezing the filesystem. The
transaction glock is removed. In it's place is a freeze glock, which is
cached (but not held) in a shared state by every node in the cluster
when the filesystem is mounted. This lock only needs to be grabbed on
freezing, and actions which need to be safe from freezing, like
recovery.
When a node wants to freeze the filesystem, it grabs this glock
exclusively. When the freeze glock state changes on the nodes (either
from shared to unlocked, or shared to exclusive), the filesystem does a
special log flush. gfs2_log_flush() does all the work for flushing out
the and shutting down the incore log, and then it tries to grab the
freeze glock in a shared state again. Since the filesystem is stuck in
gfs2_log_flush, no new transaction can start, and nothing can be written
to disk. Unfreezing the filesytem simply involes dropping the freeze
glock, allowing gfs2_log_flush() to grab and then release the shared
lock, so it is cached for next time.
However, in order for the unfreezing ioctl to occur, gfs2 needs to get a
shared lock on the filesystem root directory inode to check permissions.
If that glock has already been grabbed exclusively, fsfreeze will be
unable to get the shared lock and unfreeze the filesystem.
In order to allow the unfreeze, this patch makes gfs2 grab a shared lock
on the filesystem root directory during the freeze, and hold it until it
unfreezes the filesystem. The functions which need to grab a shared
lock in order to allow the unfreeze ioctl to be issued now use the lock
grabbed by the freeze code instead.
The freeze and unfreeze code take care to make sure that this shared
lock will not be dropped while another process is using it.
Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Steven Whitehouse <swhiteho@redhat.com>
2014-05-01 22:26:55 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2021-03-19 07:56:44 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2006-06-14 15:32:57 -04:00
|
|
|
|
2006-02-27 17:23:27 -05:00
|
|
|
|
2006-10-13 21:47:13 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
2006-11-20 10:37:45 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2025-05-16 02:58:02 +02:00
|
|
|
|
2025-08-05 23:32:39 +02:00
|
|
|
|
2021-06-01 09:41:40 -05:00
|
|
|
|
2025-08-05 23:32:39 +02:00
|
|
|
|
|
|
|
|
|
2021-06-01 09:41:40 -05:00
|
|
|
|
2025-05-16 05:34:20 +02:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2009-07-23 18:52:34 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-03-30 17:44:29 +01:00
|
|
|
|
2009-07-23 18:52:34 -05:00
|
|
|
|
2015-10-29 10:58:09 -05:00
|
|
|
|
2009-07-23 18:52:34 -05:00
|
|
|
|
2013-04-10 10:26:55 +01:00
|
|
|
|
2009-07-23 18:52:34 -05:00
|
|
|
|
2017-06-30 07:55:08 -05:00
|
|
|
|
2026-01-23 14:45:46 +01:00
|
|
|
|
2011-03-30 14:17:51 +01:00
|
|
|
|
2025-11-19 12:14:24 +00:00
|
|
|
|
2011-03-30 14:17:51 +01:00
|
|
|
|
2009-07-23 18:52:34 -05:00
|
|
|
|
|
|
|
|
|
2009-12-08 12:12:13 +00:00
|
|
|
|
2013-10-15 15:18:08 +01:00
|
|
|
|
2022-12-21 00:52:51 +01:00
|
|
|
|
2013-10-15 15:18:08 +01:00
|
|
|
|
2009-07-23 18:52:34 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2012-10-24 14:41:05 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
2021-09-29 15:06:21 -05:00
|
|
|
|
2022-06-10 11:42:33 +02:00
|
|
|
|
2008-05-21 17:03:22 +01:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2024-03-30 05:08:32 +01:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2012-10-24 14:41:05 -04:00
|
|
|
|
2009-03-09 09:03:51 +00:00
|
|
|
|
2021-09-29 15:06:21 -05:00
|
|
|
|
2020-10-07 12:30:58 +01:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2013-12-06 16:19:54 +00:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
GFS2: remove transaction glock
GFS2 has a transaction glock, which must be grabbed for every
transaction, whose purpose is to deal with freezing the filesystem.
Aside from this involving a large amount of locking, it is very easy to
make the current fsfreeze code hang on unfreezing.
This patch rewrites how gfs2 handles freezing the filesystem. The
transaction glock is removed. In it's place is a freeze glock, which is
cached (but not held) in a shared state by every node in the cluster
when the filesystem is mounted. This lock only needs to be grabbed on
freezing, and actions which need to be safe from freezing, like
recovery.
When a node wants to freeze the filesystem, it grabs this glock
exclusively. When the freeze glock state changes on the nodes (either
from shared to unlocked, or shared to exclusive), the filesystem does a
special log flush. gfs2_log_flush() does all the work for flushing out
the and shutting down the incore log, and then it tries to grab the
freeze glock in a shared state again. Since the filesystem is stuck in
gfs2_log_flush, no new transaction can start, and nothing can be written
to disk. Unfreezing the filesytem simply involes dropping the freeze
glock, allowing gfs2_log_flush() to grab and then release the shared
lock, so it is cached for next time.
However, in order for the unfreezing ioctl to occur, gfs2 needs to get a
shared lock on the filesystem root directory inode to check permissions.
If that glock has already been grabbed exclusively, fsfreeze will be
unable to get the shared lock and unfreeze the filesystem.
In order to allow the unfreeze, this patch makes gfs2 grab a shared lock
on the filesystem root directory during the freeze, and hold it until it
unfreezes the filesystem. The functions which need to grab a shared
lock in order to allow the unfreeze ioctl to be issued now use the lock
grabbed by the freeze code instead.
The freeze and unfreeze code take care to make sure that this shared
lock will not be dropped while another process is using it.
Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Steven Whitehouse <swhiteho@redhat.com>
2014-05-01 22:26:55 -05:00
|
|
|
|
|
|
|
|
|
gfs2: Rework freeze / thaw logic
So far, at mount time, gfs2 would take the freeze glock in shared mode
and then immediately drop it again, turning it into a cached glock that
can be reclaimed at any time. To freeze the filesystem cluster-wide,
the node initiating the freeze would take the freeze glock in exclusive
mode, which would cause the freeze glock's freeze_go_sync() callback to
run on each node. There, gfs2 would freeze the filesystem and schedule
gfs2_freeze_func() to run. gfs2_freeze_func() would re-acquire the
freeze glock in shared mode, thaw the filesystem, and drop the freeze
glock again. The initiating node would keep the freeze glock held in
exclusive mode. To thaw the filesystem, the initiating node would drop
the freeze glock again, which would allow gfs2_freeze_func() to resume
on all nodes, leaving the filesystem in the thawed state.
It turns out that in freeze_go_sync(), we cannot reliably and safely
freeze the filesystem. This is primarily because the final unmount of a
filesystem takes a write lock on the s_umount rw semaphore before
calling into gfs2_put_super(), and freeze_go_sync() needs to call
freeze_super() which also takes a write lock on the same semaphore,
causing a deadlock. We could work around this by trying to take an
active reference on the super block first, which would prevent unmount
from running at the same time. But that can fail, and freeze_go_sync()
isn't actually allowed to fail.
To get around this, this patch changes the freeze glock locking scheme
as follows:
At mount time, each node takes the freeze glock in shared mode. To
freeze a filesystem, the initiating node first freezes the filesystem
locally and then drops and re-acquires the freeze glock in exclusive
mode. All other nodes notice that there is contention on the freeze
glock in their go_callback callbacks, and they schedule
gfs2_freeze_func() to run. There, they freeze the filesystem locally
and drop and re-acquire the freeze glock before re-thawing the
filesystem. This is happening outside of the glock state engine, so
there, we are allowed to fail.
From a cluster point of view, taking and immediately dropping a glock is
indistinguishable from taking the glock and only dropping it upon
contention, so this new scheme is compatible with the old one.
Thanks to Li Dong <lidong@vivo.com> for reporting a locking bug in
gfs2_freeze_func() in a previous version of this commit.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
2022-11-14 23:34:50 +01:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2009-07-23 18:52:34 -05:00
|
|
|
|
2021-12-14 10:40:12 -05:00
|
|
|
|
2020-11-23 10:53:35 -05:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2025-08-05 01:26:25 +02:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
2006-08-30 09:30:00 -04:00
|
|
|
|
2006-09-05 10:53:09 -04:00
|
|
|
|
2006-01-16 16:50:04 +00:00
|
|
|
|
|
|
|
|
|
GFS2: Add a "demote a glock" interface to sysfs
This adds a sysfs file called demote_rq to GFS2's
per filesystem directory. Its possible to use this
file to demote arbitrary glocks in exactly the same
way as if a request had come in from a remote node.
This is intended for testing issues relating to caching
of data under glocks. Despite that, the interface is
generic enough to send requests to any type of glock,
but be careful as its not always safe to send an
arbitrary message to an arbitrary glock. For that reason
and to prevent DoS, this interface is restricted to root
only.
The messages look like this:
<type>:<glocknumber> <mode>
Example:
echo -n "2:13324 EX" >/sys/fs/gfs2/unity:myfs/demote_rq
Which means "please demote inode glock (type 2) number 13324 so that
I can get an EX (exclusive) lock". The lock modes are those which
would normally be sent by a remote node in its callback so if you
want to unlock a glock, you use EX, to demote to shared, use SH or PR
(depending on whether you like GFS2 or DLM lock modes better!).
If the glock doesn't exist, you'll get -ENOENT returned. If the
arguments don't make sense, you'll get -EINVAL returned.
The plan is that this interface will be used in combination with
the blktrace patch which I recently posted for comments although
it is, of course, still useful in its own right.
Signed-off-by: Steven Whitehouse <swhiteho@redhat.com>
2009-02-12 13:31:58 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|