2006-10-11 01:20:50 -07:00
|
|
|
|
2008-04-29 18:13:32 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-04-29 18:13:32 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:05 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2009-01-05 21:49:55 -05:00
|
|
|
|
2009-01-22 18:13:05 +01:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
ext4: Fix handling of extended tv_sec
In ext4, the bottom two bits of {a,c,m}time_extra are used to extend
the {a,c,m}time fields, deferring the year 2038 problem to the year
2446.
When decoding these extended fields, for times whose bottom 32 bits
would represent a negative number, sign extension causes the 64-bit
extended timestamp to be negative as well, which is not what's
intended. This patch corrects that issue, so that the only negative
{a,c,m}times are those between 1901 and 1970 (as per 32-bit signed
timestamps).
Some older kernels might have written pre-1970 dates with 1,1 in the
extra bits. This patch treats those incorrectly-encoded dates as
pre-1970, instead of post-2311, until kernel 4.20 is released.
Hopefully by then e2fsck will have fixed up the bad data.
Also add a comment explaining the encoding of ext4's extra {a,c,m}time
bits.
Signed-off-by: David Turner <novalis@novalis.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Mark Harris <mh8928@yahoo.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=23732
Cc: stable@vger.kernel.org
2015-11-24 14:34:37 -05:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-10-17 21:11:01 -04:00
|
|
|
|
2012-04-29 18:27:10 -04:00
|
|
|
|
2014-02-22 06:18:17 -05:00
|
|
|
|
2010-05-17 06:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2008-07-13 21:03:29 -04:00
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2012-03-19 23:11:43 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2012-11-28 13:03:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-09 01:55:03 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-09-30 00:32:42 -04:00
|
|
|
|
2010-05-17 07:00:00 -04:00
|
|
|
|
2009-09-30 00:32:42 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2008-07-14 17:52:37 -04:00
|
|
|
|
2009-08-09 16:46:13 -04:00
|
|
|
|
2009-08-09 22:01:13 -04:00
|
|
|
|
|
|
|
|
|
2011-05-25 07:41:26 -04:00
|
|
|
|
|
|
|
|
|
2013-04-09 22:11:22 -04:00
|
|
|
|
|
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-05 00:14:04 -05:00
|
|
|
|
2009-07-13 10:24:17 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2008-11-05 00:14:04 -05:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
|
|
|
|
|
2010-05-16 19:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-25 11:41:49 -05:00
|
|
|
|
2010-05-16 19:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-10-27 21:30:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-03-05 10:40:22 -05:00
|
|
|
|
2014-04-20 23:45:47 -04:00
|
|
|
|
2013-06-04 13:21:11 -04:00
|
|
|
|
2012-03-05 10:40:22 -05:00
|
|
|
|
2009-09-28 15:49:08 -04:00
|
|
|
|
2010-07-27 11:56:06 -04:00
|
|
|
|
2013-06-04 13:21:11 -04:00
|
|
|
|
|
|
|
|
|
2009-09-28 15:49:08 -04:00
|
|
|
|
2013-06-04 14:23:41 -04:00
|
|
|
|
|
|
|
|
|
2009-09-28 15:48:29 -04:00
|
|
|
|
2010-02-04 23:58:38 -05:00
|
|
|
|
|
|
|
|
|
2013-06-04 11:58:58 -04:00
|
|
|
|
2009-09-28 15:49:08 -04:00
|
|
|
|
|
|
|
|
|
2010-10-27 21:30:10 -04:00
|
|
|
|
2015-07-21 23:50:24 -04:00
|
|
|
|
2010-10-27 21:30:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2011-05-24 19:00:39 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-07-18 08:38:01 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
2010-10-27 21:29:12 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2007-07-17 21:42:41 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2011-09-09 18:44:51 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-12-20 09:29:35 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 18:44:51 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
|
|
|
|
|
2009-01-05 22:20:24 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Ext4: Uninitialized Block Groups
In pass1 of e2fsck, every inode table in the fileystem is scanned and checked,
regardless of whether it is in use. This is this the most time consuming part
of the filesystem check. The unintialized block group feature can greatly
reduce e2fsck time by eliminating checking of uninitialized inodes.
With this feature, there is a a high water mark of used inodes for each block
group. Block and inode bitmaps can be uninitialized on disk via a flag in the
group descriptor to avoid reading or scanning them at e2fsck time. A checksum
of each group descriptor is used to ensure that corruption in the group
descriptor's bit flags does not cause incorrect operation.
The feature is enabled through a mkfs option
mke2fs /dev/ -O uninit_groups
A patch adding support for uninitialized block groups to e2fsprogs tools has
been posted to the linux-ext4 mailing list.
The patches have been stress tested with fsstress and fsx. In performance
tests testing e2fsck time, we have seen that e2fsck time on ext3 grows
linearly with the total number of inodes in the filesytem. In ext4 with the
uninitialized block groups feature, the e2fsck time is constant, based
solely on the number of used inodes rather than the total inode count.
Since typical ext4 filesystems only use 1-10% of their inodes, this feature can
greatly reduce e2fsck time for users. With performance improvement of 2-20
times, depending on how full the filesystem is.
The attached graph shows the major improvements in e2fsck times in filesystems
with a large total inode count, but few inodes in use.
In each group descriptor if we have
EXT4_BG_INODE_UNINIT set in bg_flags:
Inode table is not initialized/used in this group. So we can skip
the consistency check during fsck.
EXT4_BG_BLOCK_UNINIT set in bg_flags:
No block in the group is used. So we can skip the block bitmap
verification for this group.
We also add two new fields to group descriptor as a part of
uninitialized group patch.
__le16 bg_itable_unused; /* Unused inodes count */
__le16 bg_checksum; /* crc16(sb_uuid+group+desc) */
bg_itable_unused:
If we have EXT4_BG_INODE_UNINIT not set in bg_flags
then bg_itable_unused will give the offset within
the inode table till the inodes are used. This can be
used by fsck to skip list of inodes that are marked unused.
bg_checksum:
Now that we depend on bg_flags and bg_itable_unused to determine
the block and inode usage, we need to make sure group descriptor
is not corrupt. We add checksum to group descriptor to
detect corruption. If the descriptor is found to be corrupt, we
mark all the blocks and inodes in the group used.
Signed-off-by: Avantika Mathur <mathur@us.ibm.com>
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
2007-10-16 18:38:25 -04:00
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 22:20:24 -05:00
|
|
|
|
Ext4: Uninitialized Block Groups
In pass1 of e2fsck, every inode table in the fileystem is scanned and checked,
regardless of whether it is in use. This is this the most time consuming part
of the filesystem check. The unintialized block group feature can greatly
reduce e2fsck time by eliminating checking of uninitialized inodes.
With this feature, there is a a high water mark of used inodes for each block
group. Block and inode bitmaps can be uninitialized on disk via a flag in the
group descriptor to avoid reading or scanning them at e2fsck time. A checksum
of each group descriptor is used to ensure that corruption in the group
descriptor's bit flags does not cause incorrect operation.
The feature is enabled through a mkfs option
mke2fs /dev/ -O uninit_groups
A patch adding support for uninitialized block groups to e2fsprogs tools has
been posted to the linux-ext4 mailing list.
The patches have been stress tested with fsstress and fsx. In performance
tests testing e2fsck time, we have seen that e2fsck time on ext3 grows
linearly with the total number of inodes in the filesytem. In ext4 with the
uninitialized block groups feature, the e2fsck time is constant, based
solely on the number of used inodes rather than the total inode count.
Since typical ext4 filesystems only use 1-10% of their inodes, this feature can
greatly reduce e2fsck time for users. With performance improvement of 2-20
times, depending on how full the filesystem is.
The attached graph shows the major improvements in e2fsck times in filesystems
with a large total inode count, but few inodes in use.
In each group descriptor if we have
EXT4_BG_INODE_UNINIT set in bg_flags:
Inode table is not initialized/used in this group. So we can skip
the consistency check during fsck.
EXT4_BG_BLOCK_UNINIT set in bg_flags:
No block in the group is used. So we can skip the block bitmap
verification for this group.
We also add two new fields to group descriptor as a part of
uninitialized group patch.
__le16 bg_itable_unused; /* Unused inodes count */
__le16 bg_checksum; /* crc16(sb_uuid+group+desc) */
bg_itable_unused:
If we have EXT4_BG_INODE_UNINIT not set in bg_flags
then bg_itable_unused will give the offset within
the inode table till the inodes are used. This can be
used by fsck to skip list of inodes that are marked unused.
bg_checksum:
Now that we depend on bg_flags and bg_itable_unused to determine
the block and inode usage, we need to make sure group descriptor
is not corrupt. We add checksum to group descriptor to
detect corruption. If the descriptor is found to be corrupt, we
mark all the blocks and inodes in the group used.
Signed-off-by: Avantika Mathur <mathur@us.ibm.com>
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:15 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 22:20:24 -05:00
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2012-04-29 18:33:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-03-11 23:39:59 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
|
|
|
|
|
Ext4: Uninitialized Block Groups
In pass1 of e2fsck, every inode table in the fileystem is scanned and checked,
regardless of whether it is in use. This is this the most time consuming part
of the filesystem check. The unintialized block group feature can greatly
reduce e2fsck time by eliminating checking of uninitialized inodes.
With this feature, there is a a high water mark of used inodes for each block
group. Block and inode bitmaps can be uninitialized on disk via a flag in the
group descriptor to avoid reading or scanning them at e2fsck time. A checksum
of each group descriptor is used to ensure that corruption in the group
descriptor's bit flags does not cause incorrect operation.
The feature is enabled through a mkfs option
mke2fs /dev/ -O uninit_groups
A patch adding support for uninitialized block groups to e2fsprogs tools has
been posted to the linux-ext4 mailing list.
The patches have been stress tested with fsstress and fsx. In performance
tests testing e2fsck time, we have seen that e2fsck time on ext3 grows
linearly with the total number of inodes in the filesytem. In ext4 with the
uninitialized block groups feature, the e2fsck time is constant, based
solely on the number of used inodes rather than the total inode count.
Since typical ext4 filesystems only use 1-10% of their inodes, this feature can
greatly reduce e2fsck time for users. With performance improvement of 2-20
times, depending on how full the filesystem is.
The attached graph shows the major improvements in e2fsck times in filesystems
with a large total inode count, but few inodes in use.
In each group descriptor if we have
EXT4_BG_INODE_UNINIT set in bg_flags:
Inode table is not initialized/used in this group. So we can skip
the consistency check during fsck.
EXT4_BG_BLOCK_UNINIT set in bg_flags:
No block in the group is used. So we can skip the block bitmap
verification for this group.
We also add two new fields to group descriptor as a part of
uninitialized group patch.
__le16 bg_itable_unused; /* Unused inodes count */
__le16 bg_checksum; /* crc16(sb_uuid+group+desc) */
bg_itable_unused:
If we have EXT4_BG_INODE_UNINIT not set in bg_flags
then bg_itable_unused will give the offset within
the inode table till the inodes are used. This can be
used by fsck to skip list of inodes that are marked unused.
bg_checksum:
Now that we depend on bg_flags and bg_itable_unused to determine
the block and inode usage, we need to make sure group descriptor
is not corrupt. We add checksum to group descriptor to
detect corruption. If the descriptor is found to be corrupt, we
mark all the blocks and inodes in the group used.
Signed-off-by: Avantika Mathur <mathur@us.ibm.com>
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
2007-10-16 18:38:25 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:14 -07:00
|
|
|
|
2006-10-11 01:21:15 -07:00
|
|
|
|
2006-10-11 01:21:14 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:21:14 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-19 16:00:58 -05:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:21:03 -07:00
|
|
|
|
2010-01-25 03:31:32 -05:00
|
|
|
|
2010-02-24 09:52:53 -05:00
|
|
|
|
2012-12-10 14:04:46 -05:00
|
|
|
|
2015-10-17 16:15:18 -04:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2010-02-24 09:52:53 -05:00
|
|
|
|
2013-04-03 22:04:52 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2009-02-15 18:57:26 -05:00
|
|
|
|
|
|
|
|
|
2011-08-31 11:54:51 -04:00
|
|
|
|
2009-02-15 18:57:26 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-02-15 18:09:20 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-11 07:44:12 -04:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:04:46 -05:00
|
|
|
|
2015-10-17 16:15:18 -04:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: ensure Inode flags consistency are checked at build time
Flags being used by atomic operations in inode flags (e.g.
ext4_test_inode_flag(), should be consistent with that actually stored
in inodes, i.e.: EXT4_XXX_FL.
It ensures that this consistency is checked at build-time, not at
run-time.
Currently, the flags consistency are being checked at run-time, but,
there is no real reason to not do a build-time check instead of a
run-time check. The code is comparing macro defined values with enum
type variables, where both are constants, so, there is no problem in
comparing constants at build-time.
enum variables are treated as constants by the C compiler, according
to the C99 specs (see www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf
sec. 6.2.5, item 16), so, there is no real problem in comparing an
enumeration type at build time
Signed-off-by: Carlos Maiolino <cmaiolino@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2012-12-10 16:30:45 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: ensure Inode flags consistency are checked at build time
Flags being used by atomic operations in inode flags (e.g.
ext4_test_inode_flag(), should be consistent with that actually stored
in inodes, i.e.: EXT4_XXX_FL.
It ensures that this consistency is checked at build-time, not at
run-time.
Currently, the flags consistency are being checked at run-time, but,
there is no real reason to not do a build-time check instead of a
run-time check. The code is comparing macro defined values with enum
type variables, where both are constants, so, there is no problem in
comparing constants at build-time.
enum variables are treated as constants by the C compiler, according
to the C99 specs (see www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf
sec. 6.2.5, item 16), so, there is no real problem in comparing an
enumeration type at build time
Signed-off-by: Carlos Maiolino <cmaiolino@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2012-12-10 16:30:45 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-19 16:00:58 -05:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:04:46 -05:00
|
|
|
|
2015-10-17 16:15:18 -04:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2007-05-24 13:04:54 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-17 06:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-01-03 23:32:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-07-17 21:42:41 -04:00
|
|
|
|
2010-07-27 11:56:07 -04:00
|
|
|
|
2007-07-17 21:42:41 -04:00
|
|
|
|
2014-04-20 23:45:47 -04:00
|
|
|
|
2009-05-14 00:58:52 -04:00
|
|
|
|
2009-05-14 13:57:08 -04:00
|
|
|
|
2014-04-20 23:45:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-14 00:58:52 -04:00
|
|
|
|
2013-04-09 22:11:22 -04:00
|
|
|
|
|
|
|
|
|
2009-06-09 00:17:05 -04:00
|
|
|
|
2009-09-28 15:49:08 -04:00
|
|
|
|
2014-04-20 23:45:47 -04:00
|
|
|
|
2009-09-28 15:49:08 -04:00
|
|
|
|
2010-03-02 13:28:44 -05:00
|
|
|
|
2010-01-15 01:27:59 -05:00
|
|
|
|
2010-03-02 13:28:44 -05:00
|
|
|
|
2014-04-20 23:45:47 -04:00
|
|
|
|
2010-03-02 13:28:44 -05:00
|
|
|
|
|
|
|
|
|
2014-04-20 23:45:47 -04:00
|
|
|
|
2013-04-09 22:11:22 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-25 07:41:54 -04:00
|
|
|
|
|
|
|
|
|
2011-10-25 08:15:12 -04:00
|
|
|
|
|
|
|
|
|
2012-07-09 16:29:29 -04:00
|
|
|
|
|
|
|
|
|
2014-03-18 18:05:35 -04:00
|
|
|
|
2014-11-25 11:44:37 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2013-08-16 21:23:41 -04:00
|
|
|
|
2013-08-16 22:05:14 -04:00
|
|
|
|
2014-09-01 14:43:09 -04:00
|
|
|
|
2013-08-16 21:23:41 -04:00
|
|
|
|
2013-08-16 22:05:14 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-08-16 21:23:41 -04:00
|
|
|
|
2014-09-04 18:09:29 -04:00
|
|
|
|
|
|
|
|
|
2013-08-16 21:23:41 -04:00
|
|
|
|
2009-11-23 07:17:05 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: Add block validity check when truncating indirect block mapped inodes
Add checks to ext4_free_branches() to make sure a block number found
in an indirect block are valid before trying to free it. If a bad
block number is found, stop freeing the indirect block immediately,
since the file system is corrupt and we will need to run fsck anyway.
This also avoids spamming the logs, and specifically avoids
driver-level "attempt to access beyond end of device" errors obscure
what is really going on.
If you get *really*, *really*, *really* unlucky, without this patch, a
supposed indirect block containing garbage might contain a reference
to a primary block group descriptor, in which case
ext4_free_branches() could end up zero'ing out a block group
descriptor block, and if then one of the block bitmaps for a block
group described by that bg descriptor block is not in memory, and is
read in by ext4_read_block_bitmap(). This function calls
ext4_valid_block_bitmap(), which assumes that bg_inode_table() was
validated at mount time and hasn't been modified since. Since this
assumption is no longer valid, it's possible for the value
(ext4_inode_table(sb, desc) - group_first_block) to go negative, which
will cause ext4_find_next_zero_bit() to trigger a kernel GPF.
Addresses-Google-Bug: #2220436
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-01-22 17:40:42 -05:00
|
|
|
|
ext4: fix i_blocks/quota accounting when extent insertion fails
The current implementation of ext4_free_blocks() always calls
dquot_free_block This looks quite sensible in the most cases: blocks
to be freed are associated with inode and were accounted in quota and
i_blocks some time ago.
However, there is a case when blocks to free were not accounted by the
time calling ext4_free_blocks() yet:
1. delalloc is on, write_begin pre-allocated some space in quota
2. write-back happens, ext4 allocates some blocks in ext4_ext_map_blocks()
3. then ext4_ext_map_blocks() gets an error (e.g. ENOSPC) from
ext4_ext_insert_extent() and calls ext4_free_blocks().
In this scenario, ext4_free_blocks() calls dquot_free_block() who, in
turn, decrements i_blocks for blocks which were not accounted yet (due
to delalloc) After clean umount, e2fsck reports something like:
> Inode 21, i_blocks is 5080, should be 5128. Fix<y>?
because i_blocks was erroneously decremented as explained above.
The patch fixes the problem by passing the new flag
EXT4_FREE_BLOCKS_NO_QUOT_UPDATE to ext4_free_blocks(), to request
that the dquot_free_block() call be skipped.
Signed-off-by: Maxim Patlasov <maxim.patlasov@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: stable@kernel.org
2011-07-10 19:37:48 -04:00
|
|
|
|
2011-09-09 18:50:51 -04:00
|
|
|
|
|
|
|
|
|
2009-11-23 07:17:05 -05:00
|
|
|
|
2015-04-11 07:44:12 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-11 07:48:01 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-09-13 19:54:35 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-02-26 01:04:07 -05:00
|
|
|
|
2008-09-13 19:54:35 -04:00
|
|
|
|
2009-02-26 01:04:07 -05:00
|
|
|
|
2009-06-17 19:24:03 -04:00
|
|
|
|
2012-01-04 17:09:44 -05:00
|
|
|
|
2013-04-08 12:54:05 -04:00
|
|
|
|
2013-08-16 22:05:14 -04:00
|
|
|
|
2015-04-11 07:48:01 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2010-05-17 05:00:00 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-17 06:00:00 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2010-05-17 05:00:00 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
tree-wide: fix comment/printk typos
"gadget", "through", "command", "maintain", "maintain", "controller", "address",
"between", "initiali[zs]e", "instead", "function", "select", "already",
"equal", "access", "management", "hierarchy", "registration", "interest",
"relative", "memory", "offset", "already",
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2010-11-01 15:38:34 -04:00
|
|
|
|
ext4: limit block allocations for indirect-block files to < 2^32
Today, the ext4 allocator will happily allocate blocks past
2^32 for indirect-block files, which results in the block
numbers getting truncated, and corruption ensues.
This patch limits such allocations to < 2^32, and adds
BUG_ONs if we do get blocks larger than that.
This should address RH Bug 519471, ext4 bitmap allocator
must limit blocks to < 2^32
* ext4_find_goal() is modified to choose a goal < UINT_MAX,
so that our starting point is in an acceptable range.
* ext4_xattr_block_set() is modified such that the goal block
is < UINT_MAX, as above.
* ext4_mb_regular_allocator() is modified so that the group
search does not continue into groups which are too high
* ext4_mb_use_preallocated() has a check that we don't use
preallocated space which is too far out
* ext4_alloc_blocks() and ext4_xattr_block_set() add some BUG_ONs
No attempt has been made to limit inode locations to < 2^32,
so we may wind up with blocks far from their inodes. Doing
this much already will lead to some odd ENOSPC issues when the
"lower 32" gets full, and further restricting inodes could
make that even weirder.
For high inodes, choosing a goal of the original, % UINT_MAX,
may be a bit odd, but then we're in an odd situation anyway,
and I don't know of a better heuristic.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2009-09-16 14:45:10 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:26 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:26 -05:00
|
|
|
|
2006-10-11 01:21:09 -07:00
|
|
|
|
2007-05-24 13:04:54 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:09 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2015-10-17 16:15:18 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2009-06-17 19:24:03 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: Fix handling of extended tv_sec
In ext4, the bottom two bits of {a,c,m}time_extra are used to extend
the {a,c,m}time fields, deferring the year 2038 problem to the year
2446.
When decoding these extended fields, for times whose bottom 32 bits
would represent a negative number, sign extension causes the 64-bit
extended timestamp to be negative as well, which is not what's
intended. This patch corrects that issue, so that the only negative
{a,c,m}times are those between 1901 and 1970 (as per 32-bit signed
timestamps).
Some older kernels might have written pre-1970 dates with 1,1 in the
extra bits. This patch treats those incorrectly-encoded dates as
pre-1970, instead of post-2311, until kernel 4.20 is released.
Hopefully by then e2fsck will have fixed up the bad data.
Also add a comment explaining the encoding of ext4's extra {a,c,m}time
bits.
Signed-off-by: David Turner <novalis@novalis.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Mark Harris <mh8928@yahoo.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=23732
Cc: stable@vger.kernel.org
2015-11-24 14:34:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
ext4: Fix handling of extended tv_sec
In ext4, the bottom two bits of {a,c,m}time_extra are used to extend
the {a,c,m}time fields, deferring the year 2038 problem to the year
2446.
When decoding these extended fields, for times whose bottom 32 bits
would represent a negative number, sign extension causes the 64-bit
extended timestamp to be negative as well, which is not what's
intended. This patch corrects that issue, so that the only negative
{a,c,m}times are those between 1901 and 1970 (as per 32-bit signed
timestamps).
Some older kernels might have written pre-1970 dates with 1,1 in the
extra bits. This patch treats those incorrectly-encoded dates as
pre-1970, instead of post-2311, until kernel 4.20 is released.
Hopefully by then e2fsck will have fixed up the bad data.
Also add a comment explaining the encoding of ext4's extra {a,c,m}time
bits.
Signed-off-by: David Turner <novalis@novalis.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Mark Harris <mh8928@yahoo.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=23732
Cc: stable@vger.kernel.org
2015-11-24 14:34:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: Fix handling of extended tv_sec
In ext4, the bottom two bits of {a,c,m}time_extra are used to extend
the {a,c,m}time fields, deferring the year 2038 problem to the year
2446.
When decoding these extended fields, for times whose bottom 32 bits
would represent a negative number, sign extension causes the 64-bit
extended timestamp to be negative as well, which is not what's
intended. This patch corrects that issue, so that the only negative
{a,c,m}times are those between 1901 and 1970 (as per 32-bit signed
timestamps).
Some older kernels might have written pre-1970 dates with 1,1 in the
extra bits. This patch treats those incorrectly-encoded dates as
pre-1970, instead of post-2311, until kernel 4.20 is released.
Hopefully by then e2fsck will have fixed up the bad data.
Also add a comment explaining the encoding of ext4's extra {a,c,m}time
bits.
Signed-off-by: David Turner <novalis@novalis.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Mark Harris <mh8928@yahoo.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=23732
Cc: stable@vger.kernel.org
2015-11-24 14:34:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-12-19 22:10:31 -05:00
|
|
|
|
|
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-16 19:29:32 -05:00
|
|
|
|
|
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-12-19 22:10:31 -05:00
|
|
|
|
|
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:09 -07:00
|
|
|
|
2008-01-28 23:58:26 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:09 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-11-08 15:18:54 -05:00
|
|
|
|
|
|
|
|
|
2016-04-01 01:31:28 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:13:42 -05:00
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
2010-01-24 14:34:07 -05:00
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-12-07 14:28:03 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
2011-01-10 12:29:43 -05:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
2014-04-21 14:37:55 -04:00
|
|
|
|
|
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-11-08 15:18:54 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-25 11:45:37 -05:00
|
|
|
|
2014-09-01 22:26:49 -04:00
|
|
|
|
2014-11-25 11:45:37 -05:00
|
|
|
|
2014-11-25 11:51:23 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-11-08 15:18:54 -05:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 18:56:51 -04:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:13:03 -05:00
|
|
|
|
2010-01-01 02:41:30 -05:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:04:46 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-12-14 15:21:14 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-09-28 15:48:29 -04:00
|
|
|
|
2013-06-04 14:21:02 -04:00
|
|
|
|
2010-03-04 16:14:02 -05:00
|
|
|
|
2013-06-04 14:21:02 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:13:42 -05:00
|
|
|
|
2012-09-28 23:24:52 -04:00
|
|
|
|
2013-06-04 14:21:02 -04:00
|
|
|
|
2011-01-10 12:13:42 -05:00
|
|
|
|
|
|
|
|
|
2009-12-08 23:51:10 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:31:10 -04:00
|
|
|
|
2014-09-29 14:58:25 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:31:10 -04:00
|
|
|
|
|
|
|
|
|
2015-04-12 00:43:56 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-04-12 00:43:56 -04:00
|
|
|
|
2009-05-01 13:44:33 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-02-10 01:11:44 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2013-04-03 22:12:52 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-03-03 17:56:23 -05:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2015-02-16 15:59:38 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-16 23:12:23 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-03-04 16:14:02 -05:00
|
|
|
|
2009-11-02 10:15:27 -08:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
2008-10-10 22:12:43 -04:00
|
|
|
|
2009-05-17 15:38:01 -04:00
|
|
|
|
2009-11-19 14:25:42 -05:00
|
|
|
|
ext4: add support for lazy inode table initialization
When the lazy_itable_init extended option is passed to mke2fs, it
considerably speeds up filesystem creation because inode tables are
not zeroed out. The fact that parts of the inode table are
uninitialized is not a problem so long as the block group descriptors,
which contain information regarding how much of the inode table has
been initialized, has not been corrupted However, if the block group
checksums are not valid, e2fsck must scan the entire inode table, and
the the old, uninitialized data could potentially cause e2fsck to
report false problems.
Hence, it is important for the inode tables to be initialized as soon
as possble. This commit adds this feature so that mke2fs can safely
use the lazy inode table initialization feature to speed up formatting
file systems.
This is done via a new new kernel thread called ext4lazyinit, which is
created on demand and destroyed, when it is no longer needed. There
is only one thread for all ext4 filesystems in the system. When the
first filesystem with inititable mount option is mounted, ext4lazyinit
thread is created, then the filesystem can register its request in the
request list.
This thread then walks through the list of requests picking up
scheduled requests and invoking ext4_init_inode_table(). Next schedule
time for the request is computed by multiplying the time it took to
zero out last inode table with wait multiplier, which can be set with
the (init_itable=n) mount option (default is 10). We are doing
this so we do not take the whole I/O bandwidth. When the thread is no
longer necessary (request list is empty) it frees the appropriate
structures and exits (and can be created later later by another
filesystem).
We do not disturb regular inode allocations in any way, it just do not
care whether the inode table is, or is not zeroed. But when zeroing, we
have to skip used inodes, obviously. Also we should prevent new inode
allocations from the group, while zeroing is on the way. For that we
take write alloc_sem lock in ext4_init_inode_table() and read alloc_sem
in the ext4_claim_inode, so when we are unlucky and allocator hits the
group which is currently being zeroed, it just has to wait.
This can be suppresed using the mount option no_init_itable.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-10-27 21:30:05 -04:00
|
|
|
|
2008-10-10 22:12:43 -04:00
|
|
|
|
2013-04-03 22:12:52 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-03 18:22:38 -04:00
|
|
|
|
|
|
|
|
|
2013-04-03 22:12:52 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-03-24 14:09:06 -04:00
|
|
|
|
|
|
|
|
|
2011-09-03 18:22:38 -04:00
|
|
|
|
ext4: do not allow journal_opts for fs w/o journal
It is appeared that we can pass journal related mount options and such options
be shown in /proc/mounts
Example:
#mkfs.ext4 -F /dev/vdb
#tune2fs -O ^has_journal /dev/vdb
#mount /dev/vdb /mnt/ -ocommit=20,journal_async_commit
#cat /proc/mounts | grep /mnt
/dev/vdb /mnt ext4 rw,relatime,journal_checksum,journal_async_commit,commit=20,data=ordered 0 0
But options:"journal_checksum,journal_async_commit,commit=20,data=ordered" has
nothing with reality because there is no journal at all.
This patch disallow following options for journalless configurations:
- journal_checksum
- journal_async_commit
- commit=%ld
- data={writeback,ordered,journal}
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
2015-10-18 23:50:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-12-15 20:26:48 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2010-12-15 20:30:48 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: use proper little-endian bitops
ext4_{set,clear}_bit() is defined as __test_and_{set,clear}_bit_le() for
ext4. Only two ext4_{set,clear}_bit() calls check the return value. The
rest of calls ignore the return value and they can be replaced with
__{set,clear}_bit_le().
This changes ext4_{set,clear}_bit() from __test_and_{set,clear}_bit_le()
to __{set,clear}_bit_le() and introduces ext4_test_and_{set,clear}_bit()
for the two places where old bit needs to be returned.
This ext4_{set,clear}_bit() change is considered safe, because if someone
uses these macros without noticing the change, new ext4_{set,clear}_bit
don't have return value and causes compiler errors where the return value
is used.
This also removes unused ext4_find_first_zero_bit().
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-12-28 20:32:07 -05:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
ext4: use proper little-endian bitops
ext4_{set,clear}_bit() is defined as __test_and_{set,clear}_bit_le() for
ext4. Only two ext4_{set,clear}_bit() calls check the return value. The
rest of calls ignore the return value and they can be replaced with
__{set,clear}_bit_le().
This changes ext4_{set,clear}_bit() from __test_and_{set,clear}_bit_le()
to __{set,clear}_bit_le() and introduces ext4_test_and_{set,clear}_bit()
for the two places where old bit needs to be returned.
This ext4_{set,clear}_bit() change is considered safe, because if someone
uses these macros without noticing the change, new ext4_{set,clear}_bit
don't have return value and causes compiler errors where the return value
is used.
This also removes unused ext4_find_first_zero_bit().
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-12-28 20:32:07 -05:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2011-03-23 16:42:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2011-07-26 22:05:53 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:14 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-27 11:56:04 -04:00
|
|
|
|
2006-10-11 01:21:14 -07:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-24 18:31:25 -04:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
|
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
2015-04-11 07:44:12 -04:00
|
|
|
|
|
|
|
|
|
2009-02-28 19:39:58 -05:00
|
|
|
|
2010-06-29 11:00:23 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-27 11:56:03 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-08-01 23:14:20 -04:00
|
|
|
|
|
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-05-12 10:16:06 -04:00
|
|
|
|
2015-01-19 16:00:58 -05:00
|
|
|
|
2015-04-11 07:44:12 -04:00
|
|
|
|
|
|
|
|
|
2015-10-17 16:15:18 -04:00
|
|
|
|
2015-10-17 16:16:02 -04:00
|
|
|
|
|
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2010-07-27 11:56:03 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2009-06-13 10:09:48 -04:00
|
|
|
|
|
|
|
|
|
2009-06-13 10:09:36 -04:00
|
|
|
|
2009-06-13 10:09:48 -04:00
|
|
|
|
2015-04-16 01:56:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-06-13 10:09:48 -04:00
|
|
|
|
2014-09-11 11:15:15 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: limit block allocations for indirect-block files to < 2^32
Today, the ext4 allocator will happily allocate blocks past
2^32 for indirect-block files, which results in the block
numbers getting truncated, and corruption ensues.
This patch limits such allocations to < 2^32, and adds
BUG_ONs if we do get blocks larger than that.
This should address RH Bug 519471, ext4 bitmap allocator
must limit blocks to < 2^32
* ext4_find_goal() is modified to choose a goal < UINT_MAX,
so that our starting point is in an acceptable range.
* ext4_xattr_block_set() is modified such that the goal block
is < UINT_MAX, as above.
* ext4_mb_regular_allocator() is modified so that the group
search does not continue into groups which are too high
* ext4_mb_use_preallocated() has a check that we don't use
preallocated space which is too far out
* ext4_alloc_blocks() and ext4_xattr_block_set() add some BUG_ONs
No attempt has been made to limit inode locations to < 2^32,
so we may wind up with blocks far from their inodes. Doing
this much already will lead to some odd ENOSPC issues when the
"lower 32" gets full, and further restricting inodes could
make that even weirder.
For high inodes, choosing a goal of the original, % UINT_MAX,
may be a bit odd, but then we're in an odd situation anyway,
and I don't know of a better heuristic.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2009-09-16 14:45:10 -04:00
|
|
|
|
2012-07-09 16:27:05 -04:00
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
|
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-06-13 10:09:41 -04:00
|
|
|
|
2010-12-15 20:30:48 -05:00
|
|
|
|
2009-06-13 10:09:48 -04:00
|
|
|
|
2012-03-04 19:27:31 -05:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
2013-04-09 22:11:22 -04:00
|
|
|
|
2012-02-07 15:41:49 -08:00
|
|
|
|
|
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-06-13 11:45:35 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 18:56:51 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
2011-09-09 18:56:51 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-05-30 22:56:46 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-26 21:35:44 -04:00
|
|
|
|
|
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-11 11:15:15 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-17 15:38:01 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-09-05 01:31:50 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-08-17 09:48:17 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: Avoid group preallocation for closed files
Currently the group preallocation code tries to find a large (512)
free block from which to do per-cpu group allocation for small files.
The problem with this scheme is that it leaves the filesystem horribly
fragmented. In the worst case, if the filesystem is unmounted and
remounted (after a system shutdown, for example) we forget the fact
that wee were using a particular (now-partially filled) 512 block
extent. So the next time we try to allocate space for a small file,
we will find *another* completely free 512 block chunk to allocate
small files. Given that there are 32,768 blocks in a block group,
after 64 iterations of "mount, write one 4k file in a directory,
unmount", the block group will have 64 files, each separated by 511
blocks, and the block group will no longer have any free 512
completely free chunks of blocks for group preallocation space.
So if we try to allocate blocks for a file that has been closed, such
that we know the final size of the file, and the filesystem is not
busy, avoid using group preallocation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2009-09-18 13:34:02 -04:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
2010-02-02 14:39:01 +09:00
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-08-17 09:54:17 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
2012-09-05 01:29:50 -04:00
|
|
|
|
2009-09-28 15:48:41 -04:00
|
|
|
|
2013-06-04 14:21:02 -04:00
|
|
|
|
|
|
|
|
|
2010-07-27 11:56:04 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: add support for lazy inode table initialization
When the lazy_itable_init extended option is passed to mke2fs, it
considerably speeds up filesystem creation because inode tables are
not zeroed out. The fact that parts of the inode table are
uninitialized is not a problem so long as the block group descriptors,
which contain information regarding how much of the inode table has
been initialized, has not been corrupted However, if the block group
checksums are not valid, e2fsck must scan the entire inode table, and
the the old, uninitialized data could potentially cause e2fsck to
report false problems.
Hence, it is important for the inode tables to be initialized as soon
as possble. This commit adds this feature so that mke2fs can safely
use the lazy inode table initialization feature to speed up formatting
file systems.
This is done via a new new kernel thread called ext4lazyinit, which is
created on demand and destroyed, when it is no longer needed. There
is only one thread for all ext4 filesystems in the system. When the
first filesystem with inititable mount option is mounted, ext4lazyinit
thread is created, then the filesystem can register its request in the
request list.
This thread then walks through the list of requests picking up
scheduled requests and invoking ext4_init_inode_table(). Next schedule
time for the request is computed by multiplying the time it took to
zero out last inode table with wait multiplier, which can be set with
the (init_itable=n) mount option (default is 10). We are doing
this so we do not take the whole I/O bandwidth. When the thread is no
longer necessary (request list is empty) it frees the appropriate
structures and exits (and can be created later later by another
filesystem).
We do not disturb regular inode allocations in any way, it just do not
care whether the inode table is, or is not zeroed. But when zeroing, we
have to skip used inodes, obviously. Also we should prevent new inode
allocations from the group, while zeroing is on the way. For that we
take write alloc_sem lock in ext4_init_inode_table() and read alloc_sem
in the ext4_claim_inode, so when we are unlucky and allocator hits the
group which is currently being zeroed, it just has to wait.
This can be suppresed using the mount option no_init_itable.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-10-27 21:30:05 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-24 18:31:25 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-11 00:03:38 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:27:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:29:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-18 00:32:55 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-25 11:51:23 -05:00
|
|
|
|
2014-11-25 11:45:37 -05:00
|
|
|
|
2014-09-01 22:26:49 -04:00
|
|
|
|
2014-03-18 19:24:49 -04:00
|
|
|
|
2014-11-25 11:45:37 -05:00
|
|
|
|
2013-10-17 21:11:01 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-03 16:33:44 -04:00
|
|
|
|
|
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
ext4: make quota as first class supported feature
This patch adds support for quotas as a first class feature in ext4;
which is to say, the quota files are stored in hidden inodes as file
system metadata, instead of as separate files visible in the file system
directory hierarchy.
It is based on the proposal at:
https://ext4.wiki.kernel.org/index.php/Design_For_1st_Class_Quota_in_Ext4
This patch introduces a new feature - EXT4_FEATURE_RO_COMPAT_QUOTA
which, when turned on, enables quota accounting at mount time
iteself. Also, the quota inodes are stored in two additional superblock
fields. Some changes introduced by this patch that should be pointed
out are:
1) Two new ext4-superblock fields - s_usr_quota_inum and
s_grp_quota_inum for storing the quota inodes in use.
2) Default quota inodes are: inode#3 for tracking userquota and inode#4
for tracking group quota. The superblock fields can be set to use
other inodes as well.
3) If the QUOTA feature and corresponding quota inodes are set in
superblock, the quota usage tracking is turned on at mount time. On
'quotaon' ioctl, the quota limits enforcement is turned
on. 'quotaoff' ioctl turns off only the limits enforcement in this
case.
4) When QUOTA feature is in use, the quota mount options 'quota',
'usrquota', 'grpquota' are ignored by the kernel.
5) mke2fs or tune2fs can be used to set the QUOTA feature and initialize
quota inodes. The default reserved inodes will not be visible to user
as regular files.
6) The quota-tools will need to be modified to support hidden quota
files on ext4. E2fsprogs will also include support for creating and
fixing quota files.
7) Support is only for the new V2 quota file format.
Tested-by: Jan Kara <jack@suse.cz>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2012-07-22 20:21:31 -04:00
|
|
|
|
|
|
|
|
|
2013-04-08 12:54:05 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2010-01-24 14:34:07 -05:00
|
|
|
|
2011-10-31 17:30:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-09-28 23:24:52 -04:00
|
|
|
|
2011-10-31 17:30:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-09-28 23:21:09 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-01-24 14:34:07 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-17 08:00:00 -04:00
|
|
|
|
2012-09-29 00:41:21 -04:00
|
|
|
|
|
|
|
|
|
2012-12-10 14:04:46 -05:00
|
|
|
|
2013-04-03 12:39:17 -04:00
|
|
|
|
2013-08-16 22:05:14 -04:00
|
|
|
|
2010-01-24 14:34:07 -05:00
|
|
|
|
|
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
2010-01-24 14:34:07 -05:00
|
|
|
|
|
|
|
|
|
2013-11-11 22:40:40 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
2013-11-11 22:40:40 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:18:25 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2015-04-12 00:43:56 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2008-11-22 15:04:59 -05:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2008-11-22 15:04:59 -05:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2008-11-22 15:04:59 -05:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-05-12 10:16:06 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:26 -05:00
|
|
|
|
Ext4: Uninitialized Block Groups
In pass1 of e2fsck, every inode table in the fileystem is scanned and checked,
regardless of whether it is in use. This is this the most time consuming part
of the filesystem check. The unintialized block group feature can greatly
reduce e2fsck time by eliminating checking of uninitialized inodes.
With this feature, there is a a high water mark of used inodes for each block
group. Block and inode bitmaps can be uninitialized on disk via a flag in the
group descriptor to avoid reading or scanning them at e2fsck time. A checksum
of each group descriptor is used to ensure that corruption in the group
descriptor's bit flags does not cause incorrect operation.
The feature is enabled through a mkfs option
mke2fs /dev/ -O uninit_groups
A patch adding support for uninitialized block groups to e2fsprogs tools has
been posted to the linux-ext4 mailing list.
The patches have been stress tested with fsstress and fsx. In performance
tests testing e2fsck time, we have seen that e2fsck time on ext3 grows
linearly with the total number of inodes in the filesytem. In ext4 with the
uninitialized block groups feature, the e2fsck time is constant, based
solely on the number of used inodes rather than the total inode count.
Since typical ext4 filesystems only use 1-10% of their inodes, this feature can
greatly reduce e2fsck time for users. With performance improvement of 2-20
times, depending on how full the filesystem is.
The attached graph shows the major improvements in e2fsck times in filesystems
with a large total inode count, but few inodes in use.
In each group descriptor if we have
EXT4_BG_INODE_UNINIT set in bg_flags:
Inode table is not initialized/used in this group. So we can skip
the consistency check during fsck.
EXT4_BG_BLOCK_UNINIT set in bg_flags:
No block in the group is used. So we can skip the block bitmap
verification for this group.
We also add two new fields to group descriptor as a part of
uninitialized group patch.
__le16 bg_itable_unused; /* Unused inodes count */
__le16 bg_checksum; /* crc16(sb_uuid+group+desc) */
bg_itable_unused:
If we have EXT4_BG_INODE_UNINIT not set in bg_flags
then bg_itable_unused will give the offset within
the inode table till the inodes are used. This can be
used by fsck to skip list of inodes that are marked unused.
bg_checksum:
Now that we depend on bg_flags and bg_itable_unused to determine
the block and inode usage, we need to make sure group descriptor
is not corrupt. We add checksum to group descriptor to
detect corruption. If the descriptor is found to be corrupt, we
mark all the blocks and inodes in the group used.
Signed-off-by: Avantika Mathur <mathur@us.ibm.com>
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
2007-10-16 18:38:25 -04:00
|
|
|
|
2007-07-18 08:38:01 -04:00
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
2011-05-24 19:00:39 -04:00
|
|
|
|
2011-09-09 18:34:51 -04:00
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-01-04 22:01:53 -05:00
|
|
|
|
2015-02-12 22:31:21 -05:00
|
|
|
|
2015-10-17 16:15:18 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:03 -07:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
2008-01-28 23:58:26 -05:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2010-01-25 03:31:32 -05:00
|
|
|
|
|
|
|
|
|
2015-10-17 16:16:02 -04:00
|
|
|
|
2012-01-04 22:01:53 -05:00
|
|
|
|
2012-12-10 14:04:46 -05:00
|
|
|
|
2015-01-19 16:00:58 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-04-18 17:29:14 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:03 -07:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2011-05-24 18:31:25 -04:00
|
|
|
|
2015-04-16 01:56:00 -04:00
|
|
|
|
|
|
|
|
|
2015-10-17 16:16:02 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
Ext4: Uninitialized Block Groups
In pass1 of e2fsck, every inode table in the fileystem is scanned and checked,
regardless of whether it is in use. This is this the most time consuming part
of the filesystem check. The unintialized block group feature can greatly
reduce e2fsck time by eliminating checking of uninitialized inodes.
With this feature, there is a a high water mark of used inodes for each block
group. Block and inode bitmaps can be uninitialized on disk via a flag in the
group descriptor to avoid reading or scanning them at e2fsck time. A checksum
of each group descriptor is used to ensure that corruption in the group
descriptor's bit flags does not cause incorrect operation.
The feature is enabled through a mkfs option
mke2fs /dev/ -O uninit_groups
A patch adding support for uninitialized block groups to e2fsprogs tools has
been posted to the linux-ext4 mailing list.
The patches have been stress tested with fsstress and fsx. In performance
tests testing e2fsck time, we have seen that e2fsck time on ext3 grows
linearly with the total number of inodes in the filesytem. In ext4 with the
uninitialized block groups feature, the e2fsck time is constant, based
solely on the number of used inodes rather than the total inode count.
Since typical ext4 filesystems only use 1-10% of their inodes, this feature can
greatly reduce e2fsck time for users. With performance improvement of 2-20
times, depending on how full the filesystem is.
The attached graph shows the major improvements in e2fsck times in filesystems
with a large total inode count, but few inodes in use.
In each group descriptor if we have
EXT4_BG_INODE_UNINIT set in bg_flags:
Inode table is not initialized/used in this group. So we can skip
the consistency check during fsck.
EXT4_BG_BLOCK_UNINIT set in bg_flags:
No block in the group is used. So we can skip the block bitmap
verification for this group.
We also add two new fields to group descriptor as a part of
uninitialized group patch.
__le16 bg_itable_unused; /* Unused inodes count */
__le16 bg_checksum; /* crc16(sb_uuid+group+desc) */
bg_itable_unused:
If we have EXT4_BG_INODE_UNINIT not set in bg_flags
then bg_itable_unused will give the offset within
the inode table till the inodes are used. This can be
used by fsck to skip list of inodes that are marked unused.
bg_checksum:
Now that we depend on bg_flags and bg_itable_unused to determine
the block and inode usage, we need to make sure group descriptor
is not corrupt. We add checksum to group descriptor to
detect corruption. If the descriptor is found to be corrupt, we
mark all the blocks and inodes in the group used.
Signed-off-by: Avantika Mathur <mathur@us.ibm.com>
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
2007-10-16 18:38:25 -04:00
|
|
|
|
2007-07-18 08:38:01 -04:00
|
|
|
|
2007-07-18 09:15:20 -04:00
|
|
|
|
2008-01-28 23:58:26 -05:00
|
|
|
|
2011-09-09 19:06:51 -04:00
|
|
|
|
2012-05-27 08:12:42 -04:00
|
|
|
|
ext4: make quota as first class supported feature
This patch adds support for quotas as a first class feature in ext4;
which is to say, the quota files are stored in hidden inodes as file
system metadata, instead of as separate files visible in the file system
directory hierarchy.
It is based on the proposal at:
https://ext4.wiki.kernel.org/index.php/Design_For_1st_Class_Quota_in_Ext4
This patch introduces a new feature - EXT4_FEATURE_RO_COMPAT_QUOTA
which, when turned on, enables quota accounting at mount time
iteself. Also, the quota inodes are stored in two additional superblock
fields. Some changes introduced by this patch that should be pointed
out are:
1) Two new ext4-superblock fields - s_usr_quota_inum and
s_grp_quota_inum for storing the quota inodes in use.
2) Default quota inodes are: inode#3 for tracking userquota and inode#4
for tracking group quota. The superblock fields can be set to use
other inodes as well.
3) If the QUOTA feature and corresponding quota inodes are set in
superblock, the quota usage tracking is turned on at mount time. On
'quotaon' ioctl, the quota limits enforcement is turned
on. 'quotaoff' ioctl turns off only the limits enforcement in this
case.
4) When QUOTA feature is in use, the quota mount options 'quota',
'usrquota', 'grpquota' are ignored by the kernel.
5) mke2fs or tune2fs can be used to set the QUOTA feature and initialize
quota inodes. The default reserved inodes will not be visible to user
as regular files.
6) The quota-tools will need to be modified to support hidden quota
files on ext4. E2fsprogs will also include support for creating and
fixing quota files.
7) Support is only for the new V2 quota file format.
Tested-by: Jan Kara <jack@suse.cz>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2012-07-22 20:21:31 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-10-09 23:53:47 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-08-01 23:14:20 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2009-01-03 20:27:38 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-12 12:18:34 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:05:59 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
2010-08-05 01:46:37 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2010-05-16 22:00:00 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-10-28 13:21:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2012-04-29 18:27:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-22 20:25:31 -04:00
|
|
|
|
2012-04-29 18:27:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-22 20:25:31 -04:00
|
|
|
|
|
|
|
|
|
2012-04-29 18:27:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-03-18 22:44:40 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-08-23 17:47:19 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
ext4: add support for lazy inode table initialization
When the lazy_itable_init extended option is passed to mke2fs, it
considerably speeds up filesystem creation because inode tables are
not zeroed out. The fact that parts of the inode table are
uninitialized is not a problem so long as the block group descriptors,
which contain information regarding how much of the inode table has
been initialized, has not been corrupted However, if the block group
checksums are not valid, e2fsck must scan the entire inode table, and
the the old, uninitialized data could potentially cause e2fsck to
report false problems.
Hence, it is important for the inode tables to be initialized as soon
as possble. This commit adds this feature so that mke2fs can safely
use the lazy inode table initialization feature to speed up formatting
file systems.
This is done via a new new kernel thread called ext4lazyinit, which is
created on demand and destroyed, when it is no longer needed. There
is only one thread for all ext4 filesystems in the system. When the
first filesystem with inititable mount option is mounted, ext4lazyinit
thread is created, then the filesystem can register its request in the
request list.
This thread then walks through the list of requests picking up
scheduled requests and invoking ext4_init_inode_table(). Next schedule
time for the request is computed by multiplying the time it took to
zero out last inode table with wait multiplier, which can be set with
the (init_itable=n) mount option (default is 10). We are doing
this so we do not take the whole I/O bandwidth. When the thread is no
longer necessary (request list is empty) it frees the appropriate
structures and exits (and can be created later later by another
filesystem).
We do not disturb regular inode allocations in any way, it just do not
care whether the inode table is, or is not zeroed. But when zeroing, we
have to skip used inodes, obviously. Also we should prevent new inode
allocations from the group, while zeroing is on the way. For that we
take write alloc_sem lock in ext4_init_inode_table() and read alloc_sem
in the ext4_claim_inode, so when we are unlucky and allocator hits the
group which is currently being zeroed, it just has to wait.
This can be suppresed using the mount option no_init_itable.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-10-27 21:30:05 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-10-27 21:30:05 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-05-24 18:31:25 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:23:10 -04:00
|
|
|
|
|
|
|
|
|
2011-05-24 18:31:25 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2007-05-24 13:04:54 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-11-05 00:14:04 -05:00
|
|
|
|
2012-06-30 19:14:57 -04:00
|
|
|
|
2012-04-29 18:33:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:35:10 -04:00
|
|
|
|
|
|
|
|
|
2012-10-22 00:34:32 -04:00
|
|
|
|
2012-04-29 18:35:10 -04:00
|
|
|
|
|
|
|
|
|
2012-10-22 00:34:32 -04:00
|
|
|
|
2008-11-05 00:14:04 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2013-04-03 23:32:34 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:05 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
2011-05-25 07:41:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 19:14:51 -04:00
|
|
|
|
|
|
|
|
|
2011-09-09 19:10:51 -04:00
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2012-02-20 17:52:46 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 19:12:51 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-06-28 10:01:31 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2015-04-11 07:48:01 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-12 00:43:56 -04:00
|
|
|
|
2015-05-18 13:19:47 -04:00
|
|
|
|
2015-04-12 00:43:56 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 10:49:26 -04:00
|
|
|
|
2015-04-12 00:43:56 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2015-04-12 00:43:56 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-31 13:35:32 -04:00
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-05-01 16:56:45 -04:00
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
|
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-05-01 16:56:45 -04:00
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
|
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
|
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
|
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-12 00:56:17 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-12 00:55:06 -04:00
|
|
|
|
2015-05-31 13:34:22 -04:00
|
|
|
|
|
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
2015-04-12 00:55:06 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-31 13:35:02 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-12 00:55:06 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-18 13:17:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-31 13:35:02 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-12 00:55:06 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2010-07-27 11:54:40 -04:00
|
|
|
|
2011-01-10 12:10:55 -05:00
|
|
|
|
2010-07-27 11:54:40 -04:00
|
|
|
|
2012-12-10 14:05:58 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-10 12:10:55 -05:00
|
|
|
|
2012-12-10 14:05:58 -05:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2015-04-12 00:56:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2012-12-10 14:05:58 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
2012-12-10 14:05:58 -05:00
|
|
|
|
2015-04-12 00:56:28 -04:00
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:05:58 -05:00
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2012-12-10 14:05:58 -05:00
|
|
|
|
|
|
|
|
|
2012-12-10 14:05:59 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2012-12-10 14:05:59 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-07-28 13:06:26 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2011-07-16 20:44:56 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-09 16:27:09 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-10-27 21:30:15 -04:00
|
|
|
|
ext4: add support for lazy inode table initialization
When the lazy_itable_init extended option is passed to mke2fs, it
considerably speeds up filesystem creation because inode tables are
not zeroed out. The fact that parts of the inode table are
uninitialized is not a problem so long as the block group descriptors,
which contain information regarding how much of the inode table has
been initialized, has not been corrupted However, if the block group
checksums are not valid, e2fsck must scan the entire inode table, and
the the old, uninitialized data could potentially cause e2fsck to
report false problems.
Hence, it is important for the inode tables to be initialized as soon
as possble. This commit adds this feature so that mke2fs can safely
use the lazy inode table initialization feature to speed up formatting
file systems.
This is done via a new new kernel thread called ext4lazyinit, which is
created on demand and destroyed, when it is no longer needed. There
is only one thread for all ext4 filesystems in the system. When the
first filesystem with inititable mount option is mounted, ext4lazyinit
thread is created, then the filesystem can register its request in the
request list.
This thread then walks through the list of requests picking up
scheduled requests and invoking ext4_init_inode_table(). Next schedule
time for the request is computed by multiplying the time it took to
zero out last inode table with wait multiplier, which can be set with
the (init_itable=n) mount option (default is 10). We are doing
this so we do not take the whole I/O bandwidth. When the thread is no
longer necessary (request list is empty) it frees the appropriate
structures and exits (and can be created later later by another
filesystem).
We do not disturb regular inode allocations in any way, it just do not
care whether the inode table is, or is not zeroed. But when zeroing, we
have to skip used inodes, obviously. Also we should prevent new inode
allocations from the group, while zeroing is on the way. For that we
take write alloc_sem lock in ext4_init_inode_table() and read alloc_sem
in the ext4_claim_inode, so when we are unlucky and allocator hits the
group which is currently being zeroed, it just has to wait.
This can be suppresed using the mount option no_init_itable.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-10-27 21:30:05 -04:00
|
|
|
|
|
|
|
|
|
2012-02-20 17:52:46 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2015-09-23 12:46:17 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
|
|
|
|
|
2012-05-28 14:19:25 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-10-10 09:40:52 -04:00
|
|
|
|
2010-10-27 21:30:14 -04:00
|
|
|
|
|
|
|
|
|
2009-11-22 07:44:56 -05:00
|
|
|
|
2009-11-23 07:17:05 -05:00
|
|
|
|
|
|
|
|
|
2012-09-05 01:31:50 -04:00
|
|
|
|
|
|
|
|
|
2009-01-05 21:36:19 -05:00
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
2011-07-26 21:46:07 -04:00
|
|
|
|
2011-05-09 10:46:41 -04:00
|
|
|
|
2010-10-27 21:30:12 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2015-04-16 01:55:00 -04:00
|
|
|
|
2014-08-29 20:51:32 -04:00
|
|
|
|
2014-08-29 20:52:15 -04:00
|
|
|
|
2012-12-10 14:05:51 -05:00
|
|
|
|
|
|
|
|
|
2015-09-08 14:59:20 -07:00
|
|
|
|
|
|
|
|
|
2008-10-07 00:46:36 -04:00
|
|
|
|
|
|
|
|
|
2012-12-10 14:05:57 -05:00
|
|
|
|
|
|
|
|
|
2012-12-10 14:05:51 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:05:57 -05:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-02-07 00:15:37 -08:00
|
|
|
|
ext4: add ext4_iget_normal() which is to be used for dir tree lookups
If there is a corrupted file system which has directory entries that
point at reserved, metadata inodes, prohibit them from being used by
treating them the same way we treat Boot Loader inodes --- that is,
mark them to be bad inodes. This prohibits them from being opened,
deleted, or modified via chmod, chown, utimes, etc.
In particular, this prevents a corrupted file system which has a
directory entry which points at the journal inode from being deleted
and its blocks released, after which point Much Hilarity Ensues.
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: stable@vger.kernel.org
2014-10-05 22:56:00 -04:00
|
|
|
|
2010-03-05 09:21:37 +01:00
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
|
|
|
|
|
2010-06-07 13:16:22 -04:00
|
|
|
|
|
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2011-05-27 06:53:02 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
2013-08-16 21:19:41 -04:00
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2013-07-01 08:12:38 -04:00
|
|
|
|
2009-08-17 22:17:20 -04:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2007-07-18 09:24:20 -04:00
|
|
|
|
2009-02-26 01:04:07 -05:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:21:03 -07:00
|
|
|
|
2008-08-19 22:16:03 -04:00
|
|
|
|
2013-05-27 23:32:35 -04:00
|
|
|
|
|
|
|
|
|
2009-03-31 15:23:21 -07:00
|
|
|
|
2015-12-07 14:28:03 -05:00
|
|
|
|
2009-12-14 15:21:14 +03:00
|
|
|
|
2010-01-25 04:00:31 -05:00
|
|
|
|
|
|
|
|
|
2011-06-27 19:40:50 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-03-16 04:33:52 -07:00
|
|
|
|
|
|
|
|
|
2011-06-27 19:40:50 -04:00
|
|
|
|
2013-06-04 12:56:55 -04:00
|
|
|
|
2013-04-03 12:47:17 -04:00
|
|
|
|
2014-07-15 06:03:38 -04:00
|
|
|
|
|
|
|
|
|
2011-06-27 16:36:31 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-04-29 22:03:54 -04:00
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-01-28 23:58:26 -05:00
|
|
|
|
2008-09-13 12:52:26 -04:00
|
|
|
|
2013-04-10 23:32:52 -04:00
|
|
|
|
2009-02-14 23:01:36 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2012-04-29 18:41:10 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:06:00 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-11 07:46:49 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-01-04 17:09:44 -05:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2015-09-23 12:46:17 -04:00
|
|
|
|
2012-07-09 16:27:05 -04:00
|
|
|
|
2012-10-10 01:06:58 -04:00
|
|
|
|
2011-08-01 08:45:02 -04:00
|
|
|
|
|
|
|
|
|
2012-09-05 01:29:50 -04:00
|
|
|
|
|
|
|
|
|
2013-02-08 13:00:31 -05:00
|
|
|
|
|
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
2011-10-31 17:11:33 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
2011-10-31 17:11:33 -07:00
|
|
|
|
|
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
2011-10-31 17:11:33 -07:00
|
|
|
|
2010-07-27 11:56:40 -04:00
|
|
|
|
|
|
|
|
|
2011-10-31 17:11:33 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-15 14:50:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-10-31 17:11:33 -07:00
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
2011-05-24 18:31:25 -04:00
|
|
|
|
|
|
|
|
|
2011-10-31 17:11:33 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
2015-06-15 14:50:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-15 14:50:26 -04:00
|
|
|
|
|
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-15 14:50:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-01 08:12:37 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:15 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 19:08:51 -04:00
|
|
|
|
|
|
|
|
|
2009-01-05 22:20:24 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:15 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-09-09 19:08:51 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 22:20:24 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-29 18:45:10 -04:00
|
|
|
|
2009-05-01 19:44:44 -04:00
|
|
|
|
2012-04-29 18:45:10 -04:00
|
|
|
|
|
|
|
|
|
2013-01-13 08:41:45 -05:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
2012-04-29 18:33:10 -04:00
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
|
|
|
|
|
2012-04-29 18:33:10 -04:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
2014-10-13 03:36:16 -04:00
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2014-10-13 03:36:16 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2007-10-16 18:38:25 -04:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
2009-01-17 18:41:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
2006-10-11 01:21:10 -07:00
|
|
|
|
2008-01-28 23:58:27 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
|
|
|
|
|
2008-09-08 22:25:24 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-06-06 11:16:43 -04:00
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-01 08:50:38 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-01-29 00:19:52 -05:00
|
|
|
|
2008-07-11 19:27:31 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2010-07-27 11:56:40 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2008-10-09 10:56:23 -04:00
|
|
|
|
2011-09-09 19:16:51 -04:00
|
|
|
|
|
|
|
|
|
2009-01-06 14:41:04 -08:00
|
|
|
|
2008-10-09 10:56:23 -04:00
|
|
|
|
2011-09-09 19:16:51 -04:00
|
|
|
|
2008-10-09 10:56:23 -04:00
|
|
|
|
2011-09-09 19:16:51 -04:00
|
|
|
|
2008-10-09 10:56:23 -04:00
|
|
|
|
|
|
|
|
|
2013-08-17 10:09:31 -04:00
|
|
|
|
2008-09-13 13:06:18 -04:00
|
|
|
|
|
|
|
|
|
2013-08-17 10:09:31 -04:00
|
|
|
|
|
|
|
|
|
2008-09-13 13:06:18 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-08-17 10:09:31 -04:00
|
|
|
|
|
|
|
|
|
2014-08-23 17:48:28 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-12-07 14:34:49 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-08-25 22:36:45 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-05-16 15:00:00 -04:00
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-08-25 22:36:45 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
|
|
|
|
|
2011-07-11 00:03:38 -04:00
|
|
|
|
|
|
|
|
|
ext4: mark block group as corrupt on block bitmap error
When we notice a block-bitmap corruption (because of device failure or
something else), we should mark this group as corrupt and prevent
further block allocations/deallocations from it. Currently, we end up
generating one error message for every block in the bitmap. This
potentially could make the system unstable as noticed in some
bugs. With this patch, the error will be printed only the first time
and mark the entire block group as corrupted. This prevents future
access allocations/deallocations from it.
Also tested by corrupting the block
bitmap and forcefully introducing the mb_free_blocks error:
(1) create a largefile (2Gb)
$ dd if=/dev/zero of=largefile oflag=direct bs=10485760 count=200
(2) umount filesystem. use dumpe2fs to see which block-bitmaps
are in use by largefile and note their block numbers
(3) use dd to zero-out the used block bitmaps
$ dd if=/dev/zero of=/dev/hdc4 bs=4096 seek=14 count=8 oflag=direct
(4) mount the FS and delete the largefile.
(5) recreate the largefile. verify that the new largefile does not
get any blocks from the groups marked as bad.
Without the patch, we will see mb_free_blocks error for each bit in
each zero'ed out bitmap at (4). With the patch, we only see the error
once per blockgroup:
[ 309.706803] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 15: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.720824] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 14: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.732858] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.748321] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 13: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.760331] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.769695] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 12: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.781721] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.798166] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 11: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.810184] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.819532] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 10: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
Google-Bug-Id: 7258357
[darrick.wong@oracle.com]
Further modifications (by Darrick) to make more obvious that this corruption
bit applies to blocks only. Set the corruption flag if the block group bitmap
verification fails.
Original-author: Aditya Kali <adityakali@google.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2013-08-28 17:35:51 -04:00
|
|
|
|
2013-08-28 18:32:58 -04:00
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: mark block group as corrupt on block bitmap error
When we notice a block-bitmap corruption (because of device failure or
something else), we should mark this group as corrupt and prevent
further block allocations/deallocations from it. Currently, we end up
generating one error message for every block in the bitmap. This
potentially could make the system unstable as noticed in some
bugs. With this patch, the error will be printed only the first time
and mark the entire block group as corrupted. This prevents future
access allocations/deallocations from it.
Also tested by corrupting the block
bitmap and forcefully introducing the mb_free_blocks error:
(1) create a largefile (2Gb)
$ dd if=/dev/zero of=largefile oflag=direct bs=10485760 count=200
(2) umount filesystem. use dumpe2fs to see which block-bitmaps
are in use by largefile and note their block numbers
(3) use dd to zero-out the used block bitmaps
$ dd if=/dev/zero of=/dev/hdc4 bs=4096 seek=14 count=8 oflag=direct
(4) mount the FS and delete the largefile.
(5) recreate the largefile. verify that the new largefile does not
get any blocks from the groups marked as bad.
Without the patch, we will see mb_free_blocks error for each bit in
each zero'ed out bitmap at (4). With the patch, we only see the error
once per blockgroup:
[ 309.706803] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 15: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.720824] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 14: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.732858] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.748321] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 13: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.760331] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.769695] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 12: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.781721] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.798166] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 11: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
[ 309.810184] EXT4-fs error (device sdb4) in ext4_free_blocks:4802: IO failure
[ 309.819532] EXT4-fs error (device sdb4): ext4_mb_generate_buddy:735: group 10: 32768 clusters in bitmap, 0 in gd. blk grp corrupted.
Google-Bug-Id: 7258357
[darrick.wong@oracle.com]
Further modifications (by Darrick) to make more obvious that this corruption
bit applies to blocks only. Set the corruption flag if the block group bitmap
verification fails.
Original-author: Aditya Kali <adityakali@google.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2013-08-28 17:35:51 -04:00
|
|
|
|
|
|
|
|
|
2013-08-28 18:32:58 -04:00
|
|
|
|
|
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
2011-07-11 00:03:38 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: Avoid group preallocation for closed files
Currently the group preallocation code tries to find a large (512)
free block from which to do per-cpu group allocation for small files.
The problem with this scheme is that it leaves the filesystem horribly
fragmented. In the worst case, if the filesystem is unmounted and
remounted (after a system shutdown, for example) we forget the fact
that wee were using a particular (now-partially filled) 512 block
extent. So the next time we try to allocate space for a small file,
we will find *another* completely free 512 block chunk to allocate
small files. Given that there are 32,768 blocks in a block group,
after 64 iterations of "mount, write one 4k file in a directory,
unmount", the block group will have 64 files, each separated by 511
blocks, and the block group will no longer have any free 512
completely free chunks of blocks for group preallocation space.
So if we try to allocate blocks for a file that has been closed, such
that we know the final size of the file, and the filesystem is not
busy, avoid using group preallocation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2009-09-18 13:34:02 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-02 20:35:09 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ext4: Avoid group preallocation for closed files
Currently the group preallocation code tries to find a large (512)
free block from which to do per-cpu group allocation for small files.
The problem with this scheme is that it leaves the filesystem horribly
fragmented. In the worst case, if the filesystem is unmounted and
remounted (after a system shutdown, for example) we forget the fact
that wee were using a particular (now-partially filled) 512 block
extent. So the next time we try to allocate space for a small file,
we will find *another* completely free 512 block chunk to allocate
small files. Given that there are 32,768 blocks in a block group,
after 64 iterations of "mount, write one 4k file in a directory,
unmount", the block group will have 64 files, each separated by 511
blocks, and the block group will no longer have any free 512
completely free chunks of blocks for group preallocation space.
So if we try to allocate blocks for a file that has been closed, such
that we know the final size of the file, and the filesystem is not
busy, avoid using group preallocation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2009-09-18 13:34:02 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
|
|
|
|
|
ext4: Avoid group preallocation for closed files
Currently the group preallocation code tries to find a large (512)
free block from which to do per-cpu group allocation for small files.
The problem with this scheme is that it leaves the filesystem horribly
fragmented. In the worst case, if the filesystem is unmounted and
remounted (after a system shutdown, for example) we forget the fact
that wee were using a particular (now-partially filled) 512 block
extent. So the next time we try to allocate space for a small file,
we will find *another* completely free 512 block chunk to allocate
small files. Given that there are 32,768 blocks in a block group,
after 64 iterations of "mount, write one 4k file in a directory,
unmount", the block group will have 64 files, each separated by 511
blocks, and the block group will no longer have any free 512
completely free chunks of blocks for group preallocation space.
So if we try to allocate blocks for a file that has been closed, such
that we know the final size of the file, and the filesystem is not
busy, avoid using group preallocation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2009-09-18 13:34:02 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-02 20:35:09 -04:00
|
|
|
|
2009-01-05 22:19:52 -05:00
|
|
|
|
|
|
|
|
|
2011-06-27 19:16:02 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2007-02-12 00:55:40 -08:00
|
|
|
|
2006-10-11 01:20:53 -07:00
|
|
|
|
2010-10-27 21:30:06 -04:00
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2013-02-09 15:23:03 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-09 15:23:03 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-17 16:08:53 -04:00
|
|
|
|
2013-02-09 15:23:03 -05:00
|
|
|
|
2013-04-19 17:53:09 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-09 15:23:03 -05:00
|
|
|
|
2015-05-18 13:14:47 -04:00
|
|
|
|
2013-02-09 15:23:03 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-12-02 16:11:20 -05:00
|
|
|
|
2013-02-09 15:23:03 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-07-15 10:10:04 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2007-02-12 00:55:40 -08:00
|
|
|
|
|
|
|
|
|
2009-05-01 13:49:15 -04:00
|
|
|
|
2012-12-10 14:05:57 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-10 14:05:59 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-19 17:53:09 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-17 16:18:43 -04:00
|
|
|
|
2013-04-19 17:53:09 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-08 00:00:32 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2015-04-27 17:51:30 -04:00
|
|
|
|
2007-02-12 00:55:40 -08:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
2015-09-23 12:44:17 -04:00
|
|
|
|
|
|
|
|
|
2015-09-23 12:46:17 -04:00
|
|
|
|
2015-09-23 12:44:17 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-05-17 15:38:01 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-10-27 21:30:14 -04:00
|
|
|
|
|
|
|
|
|
2009-05-17 15:38:01 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-06-27 19:16:02 -04:00
|
|
|
|
|
|
|
|
|
2009-05-17 15:38:01 -04:00
|
|
|
|
2006-10-11 01:21:03 -07:00
|
|
|
|
2012-11-28 13:03:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-08-28 14:47:06 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:03 -07:00
|
|
|
|
|
|
|
|
|
2013-06-04 13:01:11 -04:00
|
|
|
|
2010-05-16 19:00:00 -04:00
|
|
|
|
|
|
|
|
|
2013-04-03 12:47:17 -04:00
|
|
|
|
2013-04-03 12:45:17 -04:00
|
|
|
|
|
|
|
|
|
2006-10-11 01:21:03 -07:00
|
|
|
|
|
|
|
|
|
2011-01-14 13:07:43 +01:00
|
|
|
|
2007-07-17 21:42:41 -04:00
|
|
|
|
2013-06-04 13:21:11 -04:00
|
|
|
|
|
|
|
|
|
2010-05-16 19:00:00 -04:00
|
|
|
|
|
|
|
|
|
2012-11-28 13:03:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-01 14:37:09 -04:00
|
|
|
|
2012-11-28 13:03:30 -05:00
|
|
|
|
2014-09-01 14:43:09 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-11-28 13:03:30 -05:00
|
|
|
|
|
|
|
|
|
2013-02-18 00:28:47 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-11-28 13:03:30 -05:00
|
|
|
|
2014-08-30 23:52:19 -04:00
|
|
|
|
2008-11-22 15:04:59 -05:00
|
|
|
|
|
|
|
|
|
2013-08-16 22:05:14 -04:00
|
|
|
|
2014-02-23 15:18:59 -05:00
|
|
|
|
2015-06-09 01:55:03 -04:00
|
|
|
|
2014-08-30 23:52:19 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-11-28 13:03:30 -05:00
|
|
|
|
2009-06-17 19:24:03 -04:00
|
|
|
|
2013-04-08 12:54:05 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-06-17 19:24:03 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-10-27 21:30:10 -04:00
|
|
|
|
2010-10-27 21:30:14 -04:00
|
|
|
|
|
|
|
|
|
2010-10-27 21:30:10 -04:00
|
|
|
|
2013-06-04 11:58:58 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-06-04 14:21:02 -04:00
|
|
|
|
2010-10-27 21:30:10 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-05-12 08:12:25 -04:00
|
|
|
|
|
|
|
|
|
2008-11-22 15:04:59 -05:00
|
|
|
|
2011-05-24 18:31:25 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-05 21:49:55 -05:00
|
|
|
|
2012-09-19 21:48:00 -04:00
|
|
|
|
2009-01-05 21:49:55 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-09-29 00:41:21 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-03-03 23:55:01 -05:00
|
|
|
|
|
|
|
|
|
ext4: serialize unaligned asynchronous DIO
ext4 has a data corruption case when doing non-block-aligned
asynchronous direct IO into a sparse file, as demonstrated
by xfstest 240.
The root cause is that while ext4 preallocates space in the
hole, mappings of that space still look "new" and
dio_zero_block() will zero out the unwritten portions. When
more than one AIO thread is going, they both find this "new"
block and race to zero out their portion; this is uncoordinated
and causes data corruption.
Dave Chinner fixed this for xfs by simply serializing all
unaligned asynchronous direct IO. I've done the same here.
The difference is that we only wait on conversions, not all IO.
This is a very big hammer, and I'm not very pleased with
stuffing this into ext4_file_write(). But since ext4 is
DIO_LOCKING, we need to serialize it at this high level.
I tried to move this into ext4_ext_direct_IO, but by then
we have the i_mutex already, and we will wait on the
work queue to do conversions - which must also take the
i_mutex. So that won't work.
This was originally exposed by qemu-kvm installing to
a raw disk image with a normal sector-63 alignment. I've
tested a backport of this patch with qemu, and it does
avoid the corruption. It is also quite a lot slower
(14 min for package installs, vs. 8 min for well-aligned)
but I'll take slow correctness over fast corruption any day.
Mingming suggested that we can track outstanding
conversions, and wait on those so that non-sparse
files won't be affected, and I've implemented that here;
unaligned AIO to nonsparse files won't take a perf hit.
[tytso@mit.edu: Keep the mutex as a hashed array instead
of bloating the ext4 inode]
[tytso@mit.edu: Fix up namespace issues so that global
variables are protected with an "ext4_" prefix.]
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-02-12 08:17:34 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-26 21:35:44 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2006-10-11 01:20:50 -07:00
|
|
|
|
|
|
|
|
|
2015-10-17 16:16:04 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-04-29 18:13:32 -04:00
|
|
|
|
