linux/fs/hfsplus/dir.c

643 lines
15 KiB

License cleanup: add SPDX GPL-2.0 license identifier to files with no license Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 15:07:57 +01:00
hfsplus: fix worst-case unicode to char conversion of file names and attributes This is a series of 3 patches which corrects issues in HFS+ concerning the use of non-english file names and attributes. Names and attributes are stored internally as UTF-16 units up to a fixed maximum size, and convert to and from user-representation by NLS. The code incorrectly assume that NLS string lengths are equal to unicode lengths, which is only true for English ascii usage. This patch (of 3): The HFS Plus Volume Format specification (TN1150) states that file names are stored internally as a maximum of 255 unicode characters, as defined by The Unicode Standard, Version 2.0 [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by the NLS system on Linux before presented to the user. 255 CJK characters converts to UTF-8 with 1 unicode character to up to 3 bytes, and to GB18030 with 1 unicode character to up to 4 bytes. Thus, trying in a UTF-8 locale to list files with names of more than 85 CJK characters results in: $ ls /mnt ls: reading directory /mnt: File name too long The receiving buffer to hfsplus_uni2asc() needs to be 255 x NLS_MAX_CHARSET_SIZE bytes, not 255 bytes as the code has always been. Similar consideration applies to attributes, which are stored internally as a maximum of 127 UTF-16BE units. See XNU source for an up-to-date reference on attributes. Strictly speaking, the maximum value of NLS_MAX_CHARSET_SIZE = 6 is not attainable in the case of conversion to UTF-8, as going beyond 3 bytes requires the use of surrogate pairs, i.e. consuming two input units. Thanks Anton Altaparmakov for reviewing an earlier version of this change. This patch fixes all callers of hfsplus_uni2asc(), and also enables the use of long non-English file names in HFS+. The getting and setting, and general usage of long non-English attributes requires further forthcoming work, in the following patches of this series. [akpm@linux-foundation.org: fix build] Signed-off-by: Hin-Tak Leung <htl10@users.sourceforge.net> Reviewed-by: Anton Altaparmakov <anton@tuxera.com> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christoph Hellwig <hch@infradead.org> Cc: Sougata Santra <sougata@tuxera.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-06-06 14:36:21 -07:00
hfsplus: fix longname handling Longname is not correctly handled by hfsplus driver. If an attempt to create a longname(>255) file/directory is made, it succeeds by creating a file/directory with HFSPLUS_MAX_STRLEN and incorrect catalog key. Thus leaving the volume in an inconsistent state. This patch fixes this issue. Although lookup is always called first to create a negative entry, so just doing a check in lookup would probably fix this issue. I choose to propagate error to other iops as well. Please NOTE: I have factored out hfsplus_cat_build_key_with_cnid from hfsplus_cat_build_key, to avoid unncessary branching. Thanks a lot. TEST: ------ dir="TEST_DIR" cdir=`pwd` name255="_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_1234" name256="${name255}5" mkdir $dir cd $dir touch $name255 rm -f $name255 touch $name256 ls -la cd $cdir rm -rf $dir RESULT: ------- [sougata@ultrabook tmp]$ cdir=`pwd` [sougata@ultrabook tmp]$ name255="_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_1234" [sougata@ultrabook tmp]$ name256="${name255}5" [sougata@ultrabook tmp]$ [sougata@ultrabook tmp]$ mkdir $dir [sougata@ultrabook tmp]$ cd $dir [sougata@ultrabook TEST_DIR]$ touch $name255 [sougata@ultrabook TEST_DIR]$ rm -f $name255 [sougata@ultrabook TEST_DIR]$ touch $name256 [sougata@ultrabook TEST_DIR]$ ls -la ls: cannot access _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234: No such file or directory total 0 drwxrwxr-x 1 sougata sougata 3 Feb 20 19:56 . drwxrwxrwx 1 root root 6 Feb 20 19:56 .. -????????? ? ? ? ? ? _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234 [sougata@ultrabook TEST_DIR]$ cd $cdir [sougata@ultrabook tmp]$ rm -rf $dir rm: cannot remove `TEST_DIR': Directory not empty -ENAMETOOLONG returned from hfsplus_asc2uni was not propaged to iops. This allowed hfsplus to create files/directories with HFSPLUS_MAX_STRLEN and incorrect keys, leaving the FS in an inconsistent state. This patch fixes this issue. Signed-off-by: Sougata Santra <sougata@tuxera.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-12-18 16:17:12 -08:00
hfsplus: fix uninit-value by validating catalog record size Syzbot reported a KMSAN uninit-value issue in hfsplus_strcasecmp(). The root cause is that hfs_brec_read() doesn't validate that the on-disk record size matches the expected size for the record type being read. When mounting a corrupted filesystem, hfs_brec_read() may read less data than expected. For example, when reading a catalog thread record, the debug output showed: HFSPLUS_BREC_READ: rec_len=520, fd->entrylength=26 HFSPLUS_BREC_READ: WARNING - entrylength (26) < rec_len (520) - PARTIAL READ! hfs_brec_read() only validates that entrylength is not greater than the buffer size, but doesn't check if it's less than expected. It successfully reads 26 bytes into a 520-byte structure and returns success, leaving 494 bytes uninitialized. This uninitialized data in tmp.thread.nodeName then gets copied by hfsplus_cat_build_key_uni() and used by hfsplus_strcasecmp(), triggering the KMSAN warning when the uninitialized bytes are used as array indices in case_fold(). Fix by introducing hfsplus_brec_read_cat() wrapper that: 1. Calls hfs_brec_read() to read the data 2. Validates the record size based on the type field: - Fixed size for folder and file records - Variable size for thread records (depends on string length) 3. Returns -EIO if size doesn't match expected For thread records, check against HFSPLUS_MIN_THREAD_SZ before reading nodeName.length to avoid reading uninitialized data at call sites that don't zero-initialize the entry structure. Also initialize the tmp variable in hfsplus_find_cat() as defensive programming to ensure no uninitialized data even if validation is bypassed. Reported-by: syzbot+d80abb5b890d39261e72@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=d80abb5b890d39261e72 Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Tested-by: syzbot+d80abb5b890d39261e72@syzkaller.appspotmail.com Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com> Tested-by: Viacheslav Dubeyko <slava@dubeyko.com> Suggested-by: Charalampos Mitrodimas <charmitro@posteo.net> Link: https://lore.kernel.org/all/20260120051114.1281285-1-kartikey406@gmail.com/ [v1] Link: https://lore.kernel.org/all/20260121063109.1830263-1-kartikey406@gmail.com/ [v2] Link: https://lore.kernel.org/all/20260212014233.2422046-1-kartikey406@gmail.com/ [v3] Link: https://lore.kernel.org/all/20260214002100.436125-1-kartikey406@gmail.com/T/ [v4] Link: https://lore.kernel.org/all/20260221061626.15853-1-kartikey406@gmail.com/T/ [v5] Signed-off-by: Deepanshu Kartikey <kartikey406@gmail.com> Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> Link: https://lore.kernel.org/r/20260307010302.41547-1-kartikey406@gmail.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2026-03-07 06:33:02 +05:30
hfsplus: fix longname handling Longname is not correctly handled by hfsplus driver. If an attempt to create a longname(>255) file/directory is made, it succeeds by creating a file/directory with HFSPLUS_MAX_STRLEN and incorrect catalog key. Thus leaving the volume in an inconsistent state. This patch fixes this issue. Although lookup is always called first to create a negative entry, so just doing a check in lookup would probably fix this issue. I choose to propagate error to other iops as well. Please NOTE: I have factored out hfsplus_cat_build_key_with_cnid from hfsplus_cat_build_key, to avoid unncessary branching. Thanks a lot. TEST: ------ dir="TEST_DIR" cdir=`pwd` name255="_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_1234" name256="${name255}5" mkdir $dir cd $dir touch $name255 rm -f $name255 touch $name256 ls -la cd $cdir rm -rf $dir RESULT: ------- [sougata@ultrabook tmp]$ cdir=`pwd` [sougata@ultrabook tmp]$ name255="_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_1234" [sougata@ultrabook tmp]$ name256="${name255}5" [sougata@ultrabook tmp]$ [sougata@ultrabook tmp]$ mkdir $dir [sougata@ultrabook tmp]$ cd $dir [sougata@ultrabook TEST_DIR]$ touch $name255 [sougata@ultrabook TEST_DIR]$ rm -f $name255 [sougata@ultrabook TEST_DIR]$ touch $name256 [sougata@ultrabook TEST_DIR]$ ls -la ls: cannot access _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234: No such file or directory total 0 drwxrwxr-x 1 sougata sougata 3 Feb 20 19:56 . drwxrwxrwx 1 root root 6 Feb 20 19:56 .. -????????? ? ? ? ? ? _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234 [sougata@ultrabook TEST_DIR]$ cd $cdir [sougata@ultrabook tmp]$ rm -rf $dir rm: cannot remove `TEST_DIR': Directory not empty -ENAMETOOLONG returned from hfsplus_asc2uni was not propaged to iops. This allowed hfsplus to create files/directories with HFSPLUS_MAX_STRLEN and incorrect keys, leaving the FS in an inconsistent state. This patch fixes this issue. Signed-off-by: Sougata Santra <sougata@tuxera.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-12-18 16:17:12 -08:00
hfsplus: fix longname handling Longname is not correctly handled by hfsplus driver. If an attempt to create a longname(>255) file/directory is made, it succeeds by creating a file/directory with HFSPLUS_MAX_STRLEN and incorrect catalog key. Thus leaving the volume in an inconsistent state. This patch fixes this issue. Although lookup is always called first to create a negative entry, so just doing a check in lookup would probably fix this issue. I choose to propagate error to other iops as well. Please NOTE: I have factored out hfsplus_cat_build_key_with_cnid from hfsplus_cat_build_key, to avoid unncessary branching. Thanks a lot. TEST: ------ dir="TEST_DIR" cdir=`pwd` name255="_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_1234" name256="${name255}5" mkdir $dir cd $dir touch $name255 rm -f $name255 touch $name256 ls -la cd $cdir rm -rf $dir RESULT: ------- [sougata@ultrabook tmp]$ cdir=`pwd` [sougata@ultrabook tmp]$ name255="_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_1234" [sougata@ultrabook tmp]$ name256="${name255}5" [sougata@ultrabook tmp]$ [sougata@ultrabook tmp]$ mkdir $dir [sougata@ultrabook tmp]$ cd $dir [sougata@ultrabook TEST_DIR]$ touch $name255 [sougata@ultrabook TEST_DIR]$ rm -f $name255 [sougata@ultrabook TEST_DIR]$ touch $name256 [sougata@ultrabook TEST_DIR]$ ls -la ls: cannot access _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234: No such file or directory total 0 drwxrwxr-x 1 sougata sougata 3 Feb 20 19:56 . drwxrwxrwx 1 root root 6 Feb 20 19:56 .. -????????? ? ? ? ? ? _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234 [sougata@ultrabook TEST_DIR]$ cd $cdir [sougata@ultrabook tmp]$ rm -rf $dir rm: cannot remove `TEST_DIR': Directory not empty -ENAMETOOLONG returned from hfsplus_asc2uni was not propaged to iops. This allowed hfsplus to create files/directories with HFSPLUS_MAX_STRLEN and incorrect keys, leaving the FS in an inconsistent state. This patch fixes this issue. Signed-off-by: Sougata Santra <sougata@tuxera.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-12-18 16:17:12 -08:00
hfsplus: fix worst-case unicode to char conversion of file names and attributes This is a series of 3 patches which corrects issues in HFS+ concerning the use of non-english file names and attributes. Names and attributes are stored internally as UTF-16 units up to a fixed maximum size, and convert to and from user-representation by NLS. The code incorrectly assume that NLS string lengths are equal to unicode lengths, which is only true for English ascii usage. This patch (of 3): The HFS Plus Volume Format specification (TN1150) states that file names are stored internally as a maximum of 255 unicode characters, as defined by The Unicode Standard, Version 2.0 [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by the NLS system on Linux before presented to the user. 255 CJK characters converts to UTF-8 with 1 unicode character to up to 3 bytes, and to GB18030 with 1 unicode character to up to 4 bytes. Thus, trying in a UTF-8 locale to list files with names of more than 85 CJK characters results in: $ ls /mnt ls: reading directory /mnt: File name too long The receiving buffer to hfsplus_uni2asc() needs to be 255 x NLS_MAX_CHARSET_SIZE bytes, not 255 bytes as the code has always been. Similar consideration applies to attributes, which are stored internally as a maximum of 127 UTF-16BE units. See XNU source for an up-to-date reference on attributes. Strictly speaking, the maximum value of NLS_MAX_CHARSET_SIZE = 6 is not attainable in the case of conversion to UTF-8, as going beyond 3 bytes requires the use of surrogate pairs, i.e. consuming two input units. Thanks Anton Altaparmakov for reviewing an earlier version of this change. This patch fixes all callers of hfsplus_uni2asc(), and also enables the use of long non-English file names in HFS+. The getting and setting, and general usage of long non-English attributes requires further forthcoming work, in the following patches of this series. [akpm@linux-foundation.org: fix build] Signed-off-by: Hin-Tak Leung <htl10@users.sourceforge.net> Reviewed-by: Anton Altaparmakov <anton@tuxera.com> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christoph Hellwig <hch@infradead.org> Cc: Sougata Santra <sougata@tuxera.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-06-06 14:36:21 -07:00
hfsplus: fix worst-case unicode to char conversion of file names and attributes This is a series of 3 patches which corrects issues in HFS+ concerning the use of non-english file names and attributes. Names and attributes are stored internally as UTF-16 units up to a fixed maximum size, and convert to and from user-representation by NLS. The code incorrectly assume that NLS string lengths are equal to unicode lengths, which is only true for English ascii usage. This patch (of 3): The HFS Plus Volume Format specification (TN1150) states that file names are stored internally as a maximum of 255 unicode characters, as defined by The Unicode Standard, Version 2.0 [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by the NLS system on Linux before presented to the user. 255 CJK characters converts to UTF-8 with 1 unicode character to up to 3 bytes, and to GB18030 with 1 unicode character to up to 4 bytes. Thus, trying in a UTF-8 locale to list files with names of more than 85 CJK characters results in: $ ls /mnt ls: reading directory /mnt: File name too long The receiving buffer to hfsplus_uni2asc() needs to be 255 x NLS_MAX_CHARSET_SIZE bytes, not 255 bytes as the code has always been. Similar consideration applies to attributes, which are stored internally as a maximum of 127 UTF-16BE units. See XNU source for an up-to-date reference on attributes. Strictly speaking, the maximum value of NLS_MAX_CHARSET_SIZE = 6 is not attainable in the case of conversion to UTF-8, as going beyond 3 bytes requires the use of surrogate pairs, i.e. consuming two input units. Thanks Anton Altaparmakov for reviewing an earlier version of this change. This patch fixes all callers of hfsplus_uni2asc(), and also enables the use of long non-English file names in HFS+. The getting and setting, and general usage of long non-English attributes requires further forthcoming work, in the following patches of this series. [akpm@linux-foundation.org: fix build] Signed-off-by: Hin-Tak Leung <htl10@users.sourceforge.net> Reviewed-by: Anton Altaparmakov <anton@tuxera.com> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christoph Hellwig <hch@infradead.org> Cc: Sougata Santra <sougata@tuxera.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-06-06 14:36:21 -07:00
hfsplus: fix longname handling Longname is not correctly handled by hfsplus driver. If an attempt to create a longname(>255) file/directory is made, it succeeds by creating a file/directory with HFSPLUS_MAX_STRLEN and incorrect catalog key. Thus leaving the volume in an inconsistent state. This patch fixes this issue. Although lookup is always called first to create a negative entry, so just doing a check in lookup would probably fix this issue. I choose to propagate error to other iops as well. Please NOTE: I have factored out hfsplus_cat_build_key_with_cnid from hfsplus_cat_build_key, to avoid unncessary branching. Thanks a lot. TEST: ------ dir="TEST_DIR" cdir=`pwd` name255="_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ _123456789_123456789_123456789_123456789_123456789_1234" name256="${name255}5" mkdir $dir cd $dir touch $name255 rm -f $name255 touch $name256 ls -la cd $cdir rm -rf $dir RESULT: ------- [sougata@ultrabook tmp]$ cdir=`pwd` [sougata@ultrabook tmp]$ name255="_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_123456789_123456789\ > _123456789_123456789_123456789_123456789_123456789_1234" [sougata@ultrabook tmp]$ name256="${name255}5" [sougata@ultrabook tmp]$ [sougata@ultrabook tmp]$ mkdir $dir [sougata@ultrabook tmp]$ cd $dir [sougata@ultrabook TEST_DIR]$ touch $name255 [sougata@ultrabook TEST_DIR]$ rm -f $name255 [sougata@ultrabook TEST_DIR]$ touch $name256 [sougata@ultrabook TEST_DIR]$ ls -la ls: cannot access _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234: No such file or directory total 0 drwxrwxr-x 1 sougata sougata 3 Feb 20 19:56 . drwxrwxrwx 1 root root 6 Feb 20 19:56 .. -????????? ? ? ? ? ? _123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_1234 [sougata@ultrabook TEST_DIR]$ cd $cdir [sougata@ultrabook tmp]$ rm -rf $dir rm: cannot remove `TEST_DIR': Directory not empty -ENAMETOOLONG returned from hfsplus_asc2uni was not propaged to iops. This allowed hfsplus to create files/directories with HFSPLUS_MAX_STRLEN and incorrect keys, leaving the FS in an inconsistent state. This patch fixes this issue. Signed-off-by: Sougata Santra <sougata@tuxera.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-12-18 16:17:12 -08:00
hfsplus: rework hfsplus_readdir() logic The xfstests' test-case generic/637 fails with error: FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP PREEMPT_DYNAMIC Thu May 1 16:43:22 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch QA output created by 637 entries 7 and 8 have duplicate d_off 8 Found unlinked files in open dir (see xfstests-dev/results//generic/637.full for details) Debugging of the hfsplus_readdir() logic showed this: hfsplus: hfsplus_readdir(): 163 ctx->pos 0 hfsplus: hfsplus_readdir(): 189 ctx->pos 1 hfsplus: hfsplus_readdir(): 264 ctx->pos 2, ino 18 hfsplus: hfsplus_readdir(): 264 ctx->pos 3, ino 19 hfsplus: hfsplus_readdir(): 264 ctx->pos 4, ino 28 hfsplus: hfsplus_readdir(): 264 ctx->pos 5, ino 118 hfsplus: hfsplus_readdir(): 264 ctx->pos 6, ino 29 hfsplus: hfsplus_readdir(): 264 ctx->pos 7, ino 30 hfsplus: hfsplus_readdir(): 264 ctx->pos 8, ino 31 hfsplus: hfsplus_readdir(): 304 ctx->pos 8 hfsplus: hfsplus_unlink():420 dir->i_ino 17, inode->i_ino 28 hfsplus: hfsplus_readdir(): 141 ctx->pos 7 hfsplus: hfsplus_readdir(): 264 ctx->pos 7, ino 31 hfsplus: hfsplus_readdir(): 264 ctx->pos 8, ino 32 hfsplus: hfsplus_readdir(): 264 ctx->pos 9, ino 33 It means that hfsplus_readdir() stopped the processing of folder's items on ctx->pos 8, then, item with ino 28 has been deleted and hfsplus_readdir() re-started the logic from ctx->pos 7. As a result, previous and new sets of folder's items have overlapping values for the case of d_off 8. Currently, HFS+ has very complicated and fragile logic of rd->file->f_pos correction in hfsplus_delete_cat(). This patch removes this logic and it stores the current pos into hfsplus_readdir_data. Finally, if rd->pos == ctx->pos then hfsplus_readdir() tries to find the position in b-tree's node by means of hfsplus_cat_key. This position is used to re-start the folder's content traversal. sudo ./check generic/637 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 7.1.0-rc1+ #44 SMP PREEMPT_DYNAMIC Mon May 4 15:58:45 PDT 2026 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/637 22s ... 22s Ran: generic/637 Passed all 1 tests Closes: https://github.com/hfs-linux-kernel/hfs-linux-kernel/issues/198 cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> Link: https://lore.kernel.org/r/20260505220051.2854696-2-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2026-05-05 15:00:52 -07:00
hfsplus: fix worst-case unicode to char conversion of file names and attributes This is a series of 3 patches which corrects issues in HFS+ concerning the use of non-english file names and attributes. Names and attributes are stored internally as UTF-16 units up to a fixed maximum size, and convert to and from user-representation by NLS. The code incorrectly assume that NLS string lengths are equal to unicode lengths, which is only true for English ascii usage. This patch (of 3): The HFS Plus Volume Format specification (TN1150) states that file names are stored internally as a maximum of 255 unicode characters, as defined by The Unicode Standard, Version 2.0 [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by the NLS system on Linux before presented to the user. 255 CJK characters converts to UTF-8 with 1 unicode character to up to 3 bytes, and to GB18030 with 1 unicode character to up to 4 bytes. Thus, trying in a UTF-8 locale to list files with names of more than 85 CJK characters results in: $ ls /mnt ls: reading directory /mnt: File name too long The receiving buffer to hfsplus_uni2asc() needs to be 255 x NLS_MAX_CHARSET_SIZE bytes, not 255 bytes as the code has always been. Similar consideration applies to attributes, which are stored internally as a maximum of 127 UTF-16BE units. See XNU source for an up-to-date reference on attributes. Strictly speaking, the maximum value of NLS_MAX_CHARSET_SIZE = 6 is not attainable in the case of conversion to UTF-8, as going beyond 3 bytes requires the use of surrogate pairs, i.e. consuming two input units. Thanks Anton Altaparmakov for reviewing an earlier version of this change. This patch fixes all callers of hfsplus_uni2asc(), and also enables the use of long non-English file names in HFS+. The getting and setting, and general usage of long non-English attributes requires further forthcoming work, in the following patches of this series. [akpm@linux-foundation.org: fix build] Signed-off-by: Hin-Tak Leung <htl10@users.sourceforge.net> Reviewed-by: Anton Altaparmakov <anton@tuxera.com> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christoph Hellwig <hch@infradead.org> Cc: Sougata Santra <sougata@tuxera.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-06-06 14:36:21 -07:00
hfsplus: fix slab-out-of-bounds read in hfsplus_uni2asc() BUG: KASAN: slab-out-of-bounds in hfsplus_uni2asc+0xa71/0xb90 fs/hfsplus/unicode.c:186 Read of size 2 at addr ffff8880289ef218 by task syz.6.248/14290 CPU: 0 UID: 0 PID: 14290 Comm: syz.6.248 Not tainted 6.16.4 #1 PREEMPT(full) Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014 Call Trace: <TASK> __dump_stack lib/dump_stack.c:94 [inline] dump_stack_lvl+0x116/0x1b0 lib/dump_stack.c:120 print_address_description mm/kasan/report.c:378 [inline] print_report+0xca/0x5f0 mm/kasan/report.c:482 kasan_report+0xca/0x100 mm/kasan/report.c:595 hfsplus_uni2asc+0xa71/0xb90 fs/hfsplus/unicode.c:186 hfsplus_listxattr+0x5b6/0xbd0 fs/hfsplus/xattr.c:738 vfs_listxattr+0xbe/0x140 fs/xattr.c:493 listxattr+0xee/0x190 fs/xattr.c:924 filename_listxattr fs/xattr.c:958 [inline] path_listxattrat+0x143/0x360 fs/xattr.c:988 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0xcb/0x4c0 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7fe0e9fae16d Code: 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007fe0eae67f98 EFLAGS: 00000246 ORIG_RAX: 00000000000000c3 RAX: ffffffffffffffda RBX: 00007fe0ea205fa0 RCX: 00007fe0e9fae16d RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000200000000000 RBP: 00007fe0ea0480f0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fe0ea206038 R14: 00007fe0ea205fa0 R15: 00007fe0eae48000 </TASK> Allocated by task 14290: kasan_save_stack+0x24/0x50 mm/kasan/common.c:47 kasan_save_track+0x14/0x30 mm/kasan/common.c:68 poison_kmalloc_redzone mm/kasan/common.c:377 [inline] __kasan_kmalloc+0xaa/0xb0 mm/kasan/common.c:394 kasan_kmalloc include/linux/kasan.h:260 [inline] __do_kmalloc_node mm/slub.c:4333 [inline] __kmalloc_noprof+0x219/0x540 mm/slub.c:4345 kmalloc_noprof include/linux/slab.h:909 [inline] hfsplus_find_init+0x95/0x1f0 fs/hfsplus/bfind.c:21 hfsplus_listxattr+0x331/0xbd0 fs/hfsplus/xattr.c:697 vfs_listxattr+0xbe/0x140 fs/xattr.c:493 listxattr+0xee/0x190 fs/xattr.c:924 filename_listxattr fs/xattr.c:958 [inline] path_listxattrat+0x143/0x360 fs/xattr.c:988 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0xcb/0x4c0 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f When hfsplus_uni2asc is called from hfsplus_listxattr, it actually passes in a struct hfsplus_attr_unistr*. The size of the corresponding structure is different from that of hfsplus_unistr, so the previous fix (94458781aee6) is insufficient. The pointer on the unicode buffer is still going beyond the allocated memory. This patch introduces two warpper functions hfsplus_uni2asc_xattr_str and hfsplus_uni2asc_str to process two unicode buffers, struct hfsplus_attr_unistr* and struct hfsplus_unistr* respectively. When ustrlen value is bigger than the allocated memory size, the ustrlen value is limited to an safe size. Fixes: 94458781aee6 ("hfsplus: fix slab-out-of-bounds read in hfsplus_uni2asc()") Signed-off-by: Kang Chen <k.chen@smail.nju.edu.cn> Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com> Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> Link: https://lore.kernel.org/r/20250909031316.1647094-1-k.chen@smail.nju.edu.cn Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-09-09 11:13:16 +08:00
hfsplus: rework hfsplus_readdir() logic The xfstests' test-case generic/637 fails with error: FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP PREEMPT_DYNAMIC Thu May 1 16:43:22 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch QA output created by 637 entries 7 and 8 have duplicate d_off 8 Found unlinked files in open dir (see xfstests-dev/results//generic/637.full for details) Debugging of the hfsplus_readdir() logic showed this: hfsplus: hfsplus_readdir(): 163 ctx->pos 0 hfsplus: hfsplus_readdir(): 189 ctx->pos 1 hfsplus: hfsplus_readdir(): 264 ctx->pos 2, ino 18 hfsplus: hfsplus_readdir(): 264 ctx->pos 3, ino 19 hfsplus: hfsplus_readdir(): 264 ctx->pos 4, ino 28 hfsplus: hfsplus_readdir(): 264 ctx->pos 5, ino 118 hfsplus: hfsplus_readdir(): 264 ctx->pos 6, ino 29 hfsplus: hfsplus_readdir(): 264 ctx->pos 7, ino 30 hfsplus: hfsplus_readdir(): 264 ctx->pos 8, ino 31 hfsplus: hfsplus_readdir(): 304 ctx->pos 8 hfsplus: hfsplus_unlink():420 dir->i_ino 17, inode->i_ino 28 hfsplus: hfsplus_readdir(): 141 ctx->pos 7 hfsplus: hfsplus_readdir(): 264 ctx->pos 7, ino 31 hfsplus: hfsplus_readdir(): 264 ctx->pos 8, ino 32 hfsplus: hfsplus_readdir(): 264 ctx->pos 9, ino 33 It means that hfsplus_readdir() stopped the processing of folder's items on ctx->pos 8, then, item with ino 28 has been deleted and hfsplus_readdir() re-started the logic from ctx->pos 7. As a result, previous and new sets of folder's items have overlapping values for the case of d_off 8. Currently, HFS+ has very complicated and fragile logic of rd->file->f_pos correction in hfsplus_delete_cat(). This patch removes this logic and it stores the current pos into hfsplus_readdir_data. Finally, if rd->pos == ctx->pos then hfsplus_readdir() tries to find the position in b-tree's node by means of hfsplus_cat_key. This position is used to re-start the folder's content traversal. sudo ./check generic/637 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 7.1.0-rc1+ #44 SMP PREEMPT_DYNAMIC Mon May 4 15:58:45 PDT 2026 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/637 22s ... 22s Ran: generic/637 Passed all 1 tests Closes: https://github.com/hfs-linux-kernel/hfs-linux-kernel/issues/198 cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> Link: https://lore.kernel.org/r/20260505220051.2854696-2-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2026-05-05 15:00:52 -07:00
hfsplus: fix worst-case unicode to char conversion of file names and attributes This is a series of 3 patches which corrects issues in HFS+ concerning the use of non-english file names and attributes. Names and attributes are stored internally as UTF-16 units up to a fixed maximum size, and convert to and from user-representation by NLS. The code incorrectly assume that NLS string lengths are equal to unicode lengths, which is only true for English ascii usage. This patch (of 3): The HFS Plus Volume Format specification (TN1150) states that file names are stored internally as a maximum of 255 unicode characters, as defined by The Unicode Standard, Version 2.0 [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by the NLS system on Linux before presented to the user. 255 CJK characters converts to UTF-8 with 1 unicode character to up to 3 bytes, and to GB18030 with 1 unicode character to up to 4 bytes. Thus, trying in a UTF-8 locale to list files with names of more than 85 CJK characters results in: $ ls /mnt ls: reading directory /mnt: File name too long The receiving buffer to hfsplus_uni2asc() needs to be 255 x NLS_MAX_CHARSET_SIZE bytes, not 255 bytes as the code has always been. Similar consideration applies to attributes, which are stored internally as a maximum of 127 UTF-16BE units. See XNU source for an up-to-date reference on attributes. Strictly speaking, the maximum value of NLS_MAX_CHARSET_SIZE = 6 is not attainable in the case of conversion to UTF-8, as going beyond 3 bytes requires the use of surrogate pairs, i.e. consuming two input units. Thanks Anton Altaparmakov for reviewing an earlier version of this change. This patch fixes all callers of hfsplus_uni2asc(), and also enables the use of long non-English file names in HFS+. The getting and setting, and general usage of long non-English attributes requires further forthcoming work, in the following patches of this series. [akpm@linux-foundation.org: fix build] Signed-off-by: Hin-Tak Leung <htl10@users.sourceforge.net> Reviewed-by: Anton Altaparmakov <anton@tuxera.com> Cc: Vyacheslav Dubeyko <slava@dubeyko.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christoph Hellwig <hch@infradead.org> Cc: Sougata Santra <sougata@tuxera.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-06-06 14:36:21 -07:00
hfsplus: rework hfsplus_readdir() logic The xfstests' test-case generic/637 fails with error: FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP PREEMPT_DYNAMIC Thu May 1 16:43:22 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch QA output created by 637 entries 7 and 8 have duplicate d_off 8 Found unlinked files in open dir (see xfstests-dev/results//generic/637.full for details) Debugging of the hfsplus_readdir() logic showed this: hfsplus: hfsplus_readdir(): 163 ctx->pos 0 hfsplus: hfsplus_readdir(): 189 ctx->pos 1 hfsplus: hfsplus_readdir(): 264 ctx->pos 2, ino 18 hfsplus: hfsplus_readdir(): 264 ctx->pos 3, ino 19 hfsplus: hfsplus_readdir(): 264 ctx->pos 4, ino 28 hfsplus: hfsplus_readdir(): 264 ctx->pos 5, ino 118 hfsplus: hfsplus_readdir(): 264 ctx->pos 6, ino 29 hfsplus: hfsplus_readdir(): 264 ctx->pos 7, ino 30 hfsplus: hfsplus_readdir(): 264 ctx->pos 8, ino 31 hfsplus: hfsplus_readdir(): 304 ctx->pos 8 hfsplus: hfsplus_unlink():420 dir->i_ino 17, inode->i_ino 28 hfsplus: hfsplus_readdir(): 141 ctx->pos 7 hfsplus: hfsplus_readdir(): 264 ctx->pos 7, ino 31 hfsplus: hfsplus_readdir(): 264 ctx->pos 8, ino 32 hfsplus: hfsplus_readdir(): 264 ctx->pos 9, ino 33 It means that hfsplus_readdir() stopped the processing of folder's items on ctx->pos 8, then, item with ino 28 has been deleted and hfsplus_readdir() re-started the logic from ctx->pos 7. As a result, previous and new sets of folder's items have overlapping values for the case of d_off 8. Currently, HFS+ has very complicated and fragile logic of rd->file->f_pos correction in hfsplus_delete_cat(). This patch removes this logic and it stores the current pos into hfsplus_readdir_data. Finally, if rd->pos == ctx->pos then hfsplus_readdir() tries to find the position in b-tree's node by means of hfsplus_cat_key. This position is used to re-start the folder's content traversal. sudo ./check generic/637 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 7.1.0-rc1+ #44 SMP PREEMPT_DYNAMIC Mon May 4 15:58:45 PDT 2026 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/637 22s ... 22s Ran: generic/637 Passed all 1 tests Closes: https://github.com/hfs-linux-kernel/hfs-linux-kernel/issues/198 cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> Link: https://lore.kernel.org/r/20260505220051.2854696-2-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2026-05-05 15:00:52 -07:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
hfs/hfsplus updates for v7.1 - hfsplus: fix generic/642 failure - hfsplus: rework logic of map nodes creation in xattr b-tree - hfsplus: fix logic of alloc/free b-tree node - hfsplus: fix error processing issue in hfs_bmap_free() - hfsplus: fix potential race conditions in b-tree functionality - hfsplus: extract hidden directory search into a helper function - hfsplus: fix held lock freed on hfsplus_fill_super() - hfsplus: fix generic/523 test-case failure - hfsplus: validate b-tree node 0 bitmap at mount time - hfsplus: refactor b-tree map page access and add node-type validation - hfsplus: fix to update ctime after rename - hfsplus: fix generic/533 test-case failure - hfsplus: set ctime after setxattr and removexattr - hfsplus: fix uninit-value by validating catalog record size - hfsplus: fix potential Allocation File corruption after fsync -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQT4wVoLCG92poNnMFAhI4xTh21NnQUCadlj7wAKCRAhI4xTh21N nRzsAQDhjGNvUe9QJUDk7D/GjMxWUs+8jrGw9Ruu8GhvRIePzwEAtkZoJRwNQzwa 9XEyenrKqxLTaqaPtRChicbmdKw7eQQ= =TUrR -----END PGP SIGNATURE----- Merge tag 'hfs-v7.1-tag1' of git://git.kernel.org/pub/scm/linux/kernel/git/vdubeyko/hfs Pull hfsplus updates from Viacheslav Dubeyko: "This contains several fixes of syzbot reported issues and HFS+ fixes of xfstests failures. - Fix a syzbot reported issue of a KMSAN uninit-value in hfsplus_strcasecmp(). The root cause was that hfs_brec_read() doesn't validate that the on-disk record size matches the expected size for the record type being read. The fix introduced hfsplus_brec_read_cat() wrapper that validates the record size based on the type field and returns -EIO if size doesn't match (Deepanshu Kartikey) - Fix a syzbot reported issue of processing corrupted HFS+ images where the b-tree allocation bitmap indicates that the header node (Node 0) is free. Node 0 must always be allocated. Violating this invariant leads to allocator corruption, which cascades into kernel panics or undefined behavior. Prevent trusting a corrupted allocator state by adding a validation check during hfs_btree_open(). If corruption is detected, print a warning identifying the specific corrupted tree and force the filesystem to mount read-only (SB_RDONLY). This prevents kernel panics from corrupted images while enabling data recovery (Shardul Bankar) - Fix a potential deadlock in hfsplus_fill_super(). hfsplus_fill_super() calls hfs_find_init() to initialize a search structure, which acquires tree->tree_lock. If the subsequent call to hfsplus_cat_build_key() fails, the function jumps to the out_put_root error label without releasing the lock. Fix this by adding the missing hfs_find_exit(&fd) call before jumping to the out_put_root error label. This ensures that tree->tree_lock is properly released on the error path (Zilin Guan) - Update a files ctime after rename in hfsplus_rename() (Yangtao Li) The rest of the patches introduce the HFS+ fixes for the case of generic/348, generic/728, generic/533, generic/523, and generic/642 test-cases of xfstests suite" * tag 'hfs-v7.1-tag1' of git://git.kernel.org/pub/scm/linux/kernel/git/vdubeyko/hfs: hfsplus: fix generic/642 failure hfsplus: rework logic of map nodes creation in xattr b-tree hfsplus: fix logic of alloc/free b-tree node hfsplus: fix error processing issue in hfs_bmap_free() hfsplus: fix potential race conditions in b-tree functionality hfsplus: extract hidden directory search into a helper function hfsplus: fix held lock freed on hfsplus_fill_super() hfsplus: fix generic/523 test-case failure hfsplus: validate b-tree node 0 bitmap at mount time hfsplus: refactor b-tree map page access and add node-type validation hfsplus: fix to update ctime after rename hfsplus: fix generic/533 test-case failure hfsplus: set ctime after setxattr and removexattr hfsplus: fix uninit-value by validating catalog record size hfsplus: fix potential Allocation File corruption after fsync
2026-04-13 16:50:38 -07:00
hfsplus: fix potential Allocation File corruption after fsync The generic/348 test-case has revealed the issue of HFS+ volume corruption after simulated power failure: FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP PREEMPT_DYNAMIC Thu May 1 16:43:22 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/348 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see xfstests-dev/results//generic/348.full for details) The fsck tool complains about Allocation File (block bitmap) corruption as a result of such event. The generic/348 creates a symlink, fsync its parent directory, power fail and mount again the filesystem. Currently, HFS+ logic has several flags HFSPLUS_I_CAT_DIRTY, HFSPLUS_I_EXT_DIRTY, HFSPLUS_I_ATTR_DIRTY, HFSPLUS_I_ALLOC_DIRTY. If inode operation modified the Catalog File, Extents Overflow File, Attributes File, or Allocation File, then inode is marked as dirty and one of the mentioned flags has been set. When hfsplus_file_fsync() has been called, then this set of flags is checked and dirty b-tree or/and block bitmap is flushed. However, block bitmap can be modified during file's content allocation. It means that if we call hfsplus_file_fsync() for directory, then we never flush the modified Allocation File in such case because such inode cannot receive HFSPLUS_I_ALLOC_DIRTY flag. Moreover, this inode-centric model is not good at all because Catalog File, Extents Overflow File, Attributes File, and Allocation File represent the whole state of file system metadata. This inode-centric policy is the main reason of the issue. This patch saves the whole approach of using HFSPLUS_I_CAT_DIRTY, HFSPLUS_I_EXT_DIRTY, HFSPLUS_I_ATTR_DIRTY, and HFSPLUS_I_ALLOC_DIRTY flags. But Catalog File, Extents Overflow File, Attributes File, and Allocation File have associated inodes. And namely these inodes become the mechanism of checking the dirty state of metadata. The hfsplus_file_fsync() method checks the dirtiness of file system metadata by testing HFSPLUS_I_CAT_DIRTY, HFSPLUS_I_EXT_DIRTY, HFSPLUS_I_ATTR_DIRTY, and HFSPLUS_I_ALLOC_DIRTY flags of Catalog File's, Extents Overflow File's, Attributes File's, or Allocation File's inodes. As a result, even if we call hfsplus_file_fsync() for parent folder, then dirty Allocation File will be flushed anyway. Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20260220220152.152721-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2026-02-20 14:01:53 -08:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
hfs/hfsplus updates for v7.1 - hfsplus: fix generic/642 failure - hfsplus: rework logic of map nodes creation in xattr b-tree - hfsplus: fix logic of alloc/free b-tree node - hfsplus: fix error processing issue in hfs_bmap_free() - hfsplus: fix potential race conditions in b-tree functionality - hfsplus: extract hidden directory search into a helper function - hfsplus: fix held lock freed on hfsplus_fill_super() - hfsplus: fix generic/523 test-case failure - hfsplus: validate b-tree node 0 bitmap at mount time - hfsplus: refactor b-tree map page access and add node-type validation - hfsplus: fix to update ctime after rename - hfsplus: fix generic/533 test-case failure - hfsplus: set ctime after setxattr and removexattr - hfsplus: fix uninit-value by validating catalog record size - hfsplus: fix potential Allocation File corruption after fsync -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQT4wVoLCG92poNnMFAhI4xTh21NnQUCadlj7wAKCRAhI4xTh21N nRzsAQDhjGNvUe9QJUDk7D/GjMxWUs+8jrGw9Ruu8GhvRIePzwEAtkZoJRwNQzwa 9XEyenrKqxLTaqaPtRChicbmdKw7eQQ= =TUrR -----END PGP SIGNATURE----- Merge tag 'hfs-v7.1-tag1' of git://git.kernel.org/pub/scm/linux/kernel/git/vdubeyko/hfs Pull hfsplus updates from Viacheslav Dubeyko: "This contains several fixes of syzbot reported issues and HFS+ fixes of xfstests failures. - Fix a syzbot reported issue of a KMSAN uninit-value in hfsplus_strcasecmp(). The root cause was that hfs_brec_read() doesn't validate that the on-disk record size matches the expected size for the record type being read. The fix introduced hfsplus_brec_read_cat() wrapper that validates the record size based on the type field and returns -EIO if size doesn't match (Deepanshu Kartikey) - Fix a syzbot reported issue of processing corrupted HFS+ images where the b-tree allocation bitmap indicates that the header node (Node 0) is free. Node 0 must always be allocated. Violating this invariant leads to allocator corruption, which cascades into kernel panics or undefined behavior. Prevent trusting a corrupted allocator state by adding a validation check during hfs_btree_open(). If corruption is detected, print a warning identifying the specific corrupted tree and force the filesystem to mount read-only (SB_RDONLY). This prevents kernel panics from corrupted images while enabling data recovery (Shardul Bankar) - Fix a potential deadlock in hfsplus_fill_super(). hfsplus_fill_super() calls hfs_find_init() to initialize a search structure, which acquires tree->tree_lock. If the subsequent call to hfsplus_cat_build_key() fails, the function jumps to the out_put_root error label without releasing the lock. Fix this by adding the missing hfs_find_exit(&fd) call before jumping to the out_put_root error label. This ensures that tree->tree_lock is properly released on the error path (Zilin Guan) - Update a files ctime after rename in hfsplus_rename() (Yangtao Li) The rest of the patches introduce the HFS+ fixes for the case of generic/348, generic/728, generic/533, generic/523, and generic/642 test-cases of xfstests suite" * tag 'hfs-v7.1-tag1' of git://git.kernel.org/pub/scm/linux/kernel/git/vdubeyko/hfs: hfsplus: fix generic/642 failure hfsplus: rework logic of map nodes creation in xattr b-tree hfsplus: fix logic of alloc/free b-tree node hfsplus: fix error processing issue in hfs_bmap_free() hfsplus: fix potential race conditions in b-tree functionality hfsplus: extract hidden directory search into a helper function hfsplus: fix held lock freed on hfsplus_fill_super() hfsplus: fix generic/523 test-case failure hfsplus: validate b-tree node 0 bitmap at mount time hfsplus: refactor b-tree map page access and add node-type validation hfsplus: fix to update ctime after rename hfsplus: fix generic/533 test-case failure hfsplus: set ctime after setxattr and removexattr hfsplus: fix uninit-value by validating catalog record size hfsplus: fix potential Allocation File corruption after fsync
2026-04-13 16:50:38 -07:00
hfsplus: fix potential Allocation File corruption after fsync The generic/348 test-case has revealed the issue of HFS+ volume corruption after simulated power failure: FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP PREEMPT_DYNAMIC Thu May 1 16:43:22 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/348 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see xfstests-dev/results//generic/348.full for details) The fsck tool complains about Allocation File (block bitmap) corruption as a result of such event. The generic/348 creates a symlink, fsync its parent directory, power fail and mount again the filesystem. Currently, HFS+ logic has several flags HFSPLUS_I_CAT_DIRTY, HFSPLUS_I_EXT_DIRTY, HFSPLUS_I_ATTR_DIRTY, HFSPLUS_I_ALLOC_DIRTY. If inode operation modified the Catalog File, Extents Overflow File, Attributes File, or Allocation File, then inode is marked as dirty and one of the mentioned flags has been set. When hfsplus_file_fsync() has been called, then this set of flags is checked and dirty b-tree or/and block bitmap is flushed. However, block bitmap can be modified during file's content allocation. It means that if we call hfsplus_file_fsync() for directory, then we never flush the modified Allocation File in such case because such inode cannot receive HFSPLUS_I_ALLOC_DIRTY flag. Moreover, this inode-centric model is not good at all because Catalog File, Extents Overflow File, Attributes File, and Allocation File represent the whole state of file system metadata. This inode-centric policy is the main reason of the issue. This patch saves the whole approach of using HFSPLUS_I_CAT_DIRTY, HFSPLUS_I_EXT_DIRTY, HFSPLUS_I_ATTR_DIRTY, and HFSPLUS_I_ALLOC_DIRTY flags. But Catalog File, Extents Overflow File, Attributes File, and Allocation File have associated inodes. And namely these inodes become the mechanism of checking the dirty state of metadata. The hfsplus_file_fsync() method checks the dirtiness of file system metadata by testing HFSPLUS_I_CAT_DIRTY, HFSPLUS_I_EXT_DIRTY, HFSPLUS_I_ATTR_DIRTY, and HFSPLUS_I_ALLOC_DIRTY flags of Catalog File's, Extents Overflow File's, Attributes File's, or Allocation File's inodes. As a result, even if we call hfsplus_file_fsync() for parent folder, then dirty Allocation File will be flushed anyway. Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20260220220152.152721-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2026-02-20 14:01:53 -08:00
hfsplus: fix volume corruption issue for generic/480 The xfstests' test-case generic/480 leaves HFS+ volume in corrupted state: sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/480.full for details) Ran: generic/480 Failures: generic/480 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. CheckHardLinks: found 1 pre-Leopard file inodes. Incorrect number of file hard links ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. invalid VHB nextCatalogID Volume header needs minor repair (2, 0) Verify Status: VIStat = 0x8000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00000002 ** Repairing volume. Incorrect flags for file hard link (id = 19) (It should be 0x22 instead of 0x2) Incorrect flags for file inode (id = 18) (It should be 0x22 instead of 0x2) first link ID=0 is < 16 for fileinode=18 Error getting first link ID for inode = 18 (result=2) Invalid first link in hard link chain (id = 18) (It should be 19 instead of 0) Indirect node 18 needs link count adjustment (It should be 1 instead of 2) ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The generic/480 test executes such steps on final phase: "Now remove of the links of our file and create a new file with the same name and in the same parent directory, and finally fsync this new file." unlink $SCRATCH_MNT/testdir/bar touch $SCRATCH_MNT/testdir/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir/bar "Simulate a power failure and mount the filesystem to check that replaying the fsync log/journal succeeds, that is the mount operation does not fail." _flakey_drop_and_remount The key issue in HFS+ logic is that hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods don't call hfsplus_cat_write_inode() for the case of modified inode objects. As a result, even if hfsplus_file_fsync() is trying to flush the dirty Catalog File, but because of not calling hfsplus_cat_write_inode() not all modified inodes save the new state into Catalog File's records. Finally, simulation of power failure results in inconsistent state of Catalog File and FSCK tool reports about volume corruption. This patch adds calling of hfsplus_cat_write_inode() method for modified inodes in hfsplus_link(), hfsplus_unlink(), hfsplus_rmdir(), hfsplus_symlink(), and hfsplus_mknod() methods. Also, it adds debug output in several methods. sudo ./check generic/480 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc1+ #18 SMP PREEMPT_DYNAMIC Thu Dec 4 12:24:45 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/480 16s ... 16s Ran: generic/480 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251205000054.3670326-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-12-04 16:00:55 -08:00
Change inode_operations.mkdir to return struct dentry * Some filesystems, such as NFS, cifs, ceph, and fuse, do not have complete control of sequencing on the actual filesystem (e.g. on a different server) and may find that the inode created for a mkdir request already exists in the icache and dcache by the time the mkdir request returns. For example, if the filesystem is mounted twice the directory could be visible on the other mount before it is on the original mount, and a pair of name_to_handle_at(), open_by_handle_at() calls could instantiate the directory inode with an IS_ROOT() dentry before the first mkdir returns. This means that the dentry passed to ->mkdir() may not be the one that is associated with the inode after the ->mkdir() completes. Some callers need to interact with the inode after the ->mkdir completes and they currently need to perform a lookup in the (rare) case that the dentry is no longer hashed. This lookup-after-mkdir requires that the directory remains locked to avoid races. Planned future patches to lock the dentry rather than the directory will mean that this lookup cannot be performed atomically with the mkdir. To remove this barrier, this patch changes ->mkdir to return the resulting dentry if it is different from the one passed in. Possible returns are: NULL - the directory was created and no other dentry was used ERR_PTR() - an error occurred non-NULL - this other dentry was spliced in This patch only changes file-systems to return "ERR_PTR(err)" instead of "err" or equivalent transformations. Subsequent patches will make further changes to some file-systems to return a correct dentry. Not all filesystems reliably result in a positive hashed dentry: - NFS, cifs, hostfs will sometimes need to perform a lookup of the name to get inode information. Races could result in this returning something different. Note that this lookup is non-atomic which is what we are trying to avoid. Placing the lookup in filesystem code means it only happens when the filesystem has no other option. - kernfs and tracefs leave the dentry negative and the ->revalidate operation ensures that lookup will be called to correctly populate the dentry. This could be fixed but I don't think it is important to any of the users of vfs_mkdir() which look at the dentry. The recommendation to use d_drop();d_splice_alias() is ugly but fits with current practice. A planned future patch will change this. Reviewed-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Jan Kara <jack@suse.cz> Signed-off-by: NeilBrown <neilb@suse.de> Link: https://lore.kernel.org/r/20250227013949.536172-2-neilb@suse.de Signed-off-by: Christian Brauner <brauner@kernel.org>
2025-02-27 12:32:53 +11:00
VFS: (Scripted) Convert S_ISLNK/DIR/REG(dentry->d_inode) to d_is_*(dentry) Convert the following where appropriate: (1) S_ISLNK(dentry->d_inode) to d_is_symlink(dentry). (2) S_ISREG(dentry->d_inode) to d_is_reg(dentry). (3) S_ISDIR(dentry->d_inode) to d_is_dir(dentry). This is actually more complicated than it appears as some calls should be converted to d_can_lookup() instead. The difference is whether the directory in question is a real dir with a ->lookup op or whether it's a fake dir with a ->d_automount op. In some circumstances, we can subsume checks for dentry->d_inode not being NULL into this, provided we the code isn't in a filesystem that expects d_inode to be NULL if the dirent really *is* negative (ie. if we're going to use d_inode() rather than d_backing_inode() to get the inode pointer). Note that the dentry type field may be set to something other than DCACHE_MISS_TYPE when d_inode is NULL in the case of unionmount, where the VFS manages the fall-through from a negative dentry to a lower layer. In such a case, the dentry type of the negative union dentry is set to the same as the type of the lower dentry. However, if you know d_inode is not NULL at the call site, then you can use the d_is_xxx() functions even in a filesystem. There is one further complication: a 0,0 chardev dentry may be labelled DCACHE_WHITEOUT_TYPE rather than DCACHE_SPECIAL_TYPE. Strictly, this was intended for special directory entry types that don't have attached inodes. The following perl+coccinelle script was used: use strict; my @callers; open($fd, 'git grep -l \'S_IS[A-Z].*->d_inode\' |') || die "Can't grep for S_ISDIR and co. callers"; @callers = <$fd>; close($fd); unless (@callers) { print "No matches\n"; exit(0); } my @cocci = ( '@@', 'expression E;', '@@', '', '- S_ISLNK(E->d_inode->i_mode)', '+ d_is_symlink(E)', '', '@@', 'expression E;', '@@', '', '- S_ISDIR(E->d_inode->i_mode)', '+ d_is_dir(E)', '', '@@', 'expression E;', '@@', '', '- S_ISREG(E->d_inode->i_mode)', '+ d_is_reg(E)' ); my $coccifile = "tmp.sp.cocci"; open($fd, ">$coccifile") || die $coccifile; print($fd "$_\n") || die $coccifile foreach (@cocci); close($fd); foreach my $file (@callers) { chomp $file; print "Processing ", $file, "\n"; system("spatch", "--sp-file", $coccifile, $file, "--in-place", "--no-show-diff") == 0 || die "spatch failed"; } [AV: overlayfs parts skipped] Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-01-29 12:02:35 +00:00
hfsplus: fix volume corruption issue for generic/073 The xfstests' test-case generic/073 leaves HFS+ volume in corrupted state: sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/073.full for details) Ran: generic/073 Failures: generic/073 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. Invalid directory item count (It should be 1 instead of 0) ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. Verify Status: VIStat = 0x0000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00004000 ** Repairing volume. ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The test is doing these steps on final phase: mv $SCRATCH_MNT/testdir_1/bar $SCRATCH_MNT/testdir_2/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir_1 $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo So, we move file bar from testdir_1 into testdir_2 folder. It means that HFS+ logic decrements the number of entries in testdir_1 and increments number of entries in testdir_2. Finally, we do fsync only for testdir_1 and foo but not for testdir_2. As a result, this is the reason why fsck.hfsplus detects the volume corruption afterwards. This patch fixes the issue by means of adding the hfsplus_cat_write_inode() call for old_dir and new_dir in hfsplus_rename() after the successful ending of hfsplus_rename_cat(). This method makes modification of in-core inode objects for old_dir and new_dir but it doesn't save these modifications in Catalog File's entries. It was expected that hfsplus_write_inode() will save these modifications afterwards. However, because generic/073 does fsync only for testdir_1 and foo then testdir_2 modification hasn't beed saved into Catalog File's entry and it was flushed without this modification. And it was detected by fsck.hfsplus. Now, hfsplus_rename() stores in Catalog File all modified entries and correct state of Catalog File will be flushed during hfsplus_file_fsync() call. Finally, it makes fsck.hfsplus happy. sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc3+ #93 SMP PREEMPT_DYNAMIC Wed Nov 12 14:37:49 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 32s ... 32s Ran: generic/073 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251112232522.814038-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-11-12 15:25:23 -08:00
hfsplus: fix volume corruption issue for generic/073 The xfstests' test-case generic/073 leaves HFS+ volume in corrupted state: sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/073.full for details) Ran: generic/073 Failures: generic/073 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. Invalid directory item count (It should be 1 instead of 0) ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. Verify Status: VIStat = 0x0000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00004000 ** Repairing volume. ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The test is doing these steps on final phase: mv $SCRATCH_MNT/testdir_1/bar $SCRATCH_MNT/testdir_2/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir_1 $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo So, we move file bar from testdir_1 into testdir_2 folder. It means that HFS+ logic decrements the number of entries in testdir_1 and increments number of entries in testdir_2. Finally, we do fsync only for testdir_1 and foo but not for testdir_2. As a result, this is the reason why fsck.hfsplus detects the volume corruption afterwards. This patch fixes the issue by means of adding the hfsplus_cat_write_inode() call for old_dir and new_dir in hfsplus_rename() after the successful ending of hfsplus_rename_cat(). This method makes modification of in-core inode objects for old_dir and new_dir but it doesn't save these modifications in Catalog File's entries. It was expected that hfsplus_write_inode() will save these modifications afterwards. However, because generic/073 does fsync only for testdir_1 and foo then testdir_2 modification hasn't beed saved into Catalog File's entry and it was flushed without this modification. And it was detected by fsck.hfsplus. Now, hfsplus_rename() stores in Catalog File all modified entries and correct state of Catalog File will be flushed during hfsplus_file_fsync() call. Finally, it makes fsck.hfsplus happy. sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc3+ #93 SMP PREEMPT_DYNAMIC Wed Nov 12 14:37:49 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 32s ... 32s Ran: generic/073 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251112232522.814038-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-11-12 15:25:23 -08:00
hfsplus: fix volume corruption issue for generic/073 The xfstests' test-case generic/073 leaves HFS+ volume in corrupted state: sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/073.full for details) Ran: generic/073 Failures: generic/073 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. Invalid directory item count (It should be 1 instead of 0) ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. Verify Status: VIStat = 0x0000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00004000 ** Repairing volume. ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The test is doing these steps on final phase: mv $SCRATCH_MNT/testdir_1/bar $SCRATCH_MNT/testdir_2/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir_1 $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo So, we move file bar from testdir_1 into testdir_2 folder. It means that HFS+ logic decrements the number of entries in testdir_1 and increments number of entries in testdir_2. Finally, we do fsync only for testdir_1 and foo but not for testdir_2. As a result, this is the reason why fsck.hfsplus detects the volume corruption afterwards. This patch fixes the issue by means of adding the hfsplus_cat_write_inode() call for old_dir and new_dir in hfsplus_rename() after the successful ending of hfsplus_rename_cat(). This method makes modification of in-core inode objects for old_dir and new_dir but it doesn't save these modifications in Catalog File's entries. It was expected that hfsplus_write_inode() will save these modifications afterwards. However, because generic/073 does fsync only for testdir_1 and foo then testdir_2 modification hasn't beed saved into Catalog File's entry and it was flushed without this modification. And it was detected by fsck.hfsplus. Now, hfsplus_rename() stores in Catalog File all modified entries and correct state of Catalog File will be flushed during hfsplus_file_fsync() call. Finally, it makes fsck.hfsplus happy. sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc3+ #93 SMP PREEMPT_DYNAMIC Wed Nov 12 14:37:49 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 32s ... 32s Ran: generic/073 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251112232522.814038-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-11-12 15:25:23 -08:00
hfsplus: fix volume corruption issue for generic/073 The xfstests' test-case generic/073 leaves HFS+ volume in corrupted state: sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.17.0-rc1+ #4 SMP PREEMPT_DYNAMIC Wed Oct 1 15:02:44 PDT 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 _check_generic_filesystem: filesystem on /dev/loop51 is inconsistent (see XFSTESTS-2/xfstests-dev/results//generic/073.full for details) Ran: generic/073 Failures: generic/073 Failed 1 of 1 tests sudo fsck.hfsplus -d /dev/loop51 ** /dev/loop51 Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K. Executing fsck_hfs (version 540.1-Linux). ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. Invalid directory item count (It should be 1 instead of 0) ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. Verify Status: VIStat = 0x0000, ABTStat = 0x0000 EBTStat = 0x0000 CBTStat = 0x0000 CatStat = 0x00004000 ** Repairing volume. ** Rechecking volume. ** Checking non-journaled HFS Plus Volume. The volume name is untitled ** Checking extents overflow file. ** Checking catalog file. ** Checking multi-linked files. ** Checking catalog hierarchy. ** Checking extended attributes file. ** Checking volume bitmap. ** Checking volume information. ** The volume untitled was repaired successfully. The test is doing these steps on final phase: mv $SCRATCH_MNT/testdir_1/bar $SCRATCH_MNT/testdir_2/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/testdir_1 $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo So, we move file bar from testdir_1 into testdir_2 folder. It means that HFS+ logic decrements the number of entries in testdir_1 and increments number of entries in testdir_2. Finally, we do fsync only for testdir_1 and foo but not for testdir_2. As a result, this is the reason why fsck.hfsplus detects the volume corruption afterwards. This patch fixes the issue by means of adding the hfsplus_cat_write_inode() call for old_dir and new_dir in hfsplus_rename() after the successful ending of hfsplus_rename_cat(). This method makes modification of in-core inode objects for old_dir and new_dir but it doesn't save these modifications in Catalog File's entries. It was expected that hfsplus_write_inode() will save these modifications afterwards. However, because generic/073 does fsync only for testdir_1 and foo then testdir_2 modification hasn't beed saved into Catalog File's entry and it was flushed without this modification. And it was detected by fsck.hfsplus. Now, hfsplus_rename() stores in Catalog File all modified entries and correct state of Catalog File will be flushed during hfsplus_file_fsync() call. Finally, it makes fsck.hfsplus happy. sudo ./check generic/073 FSTYP -- hfsplus PLATFORM -- Linux/x86_64 hfsplus-testing-0001 6.18.0-rc3+ #93 SMP PREEMPT_DYNAMIC Wed Nov 12 14:37:49 PST 2025 MKFS_OPTIONS -- /dev/loop51 MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch generic/073 32s ... 32s Ran: generic/073 Passed all 1 tests Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com> cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> cc: Yangtao Li <frank.li@vivo.com> cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20251112232522.814038-1-slava@dubeyko.com Signed-off-by: Viacheslav Dubeyko <slava@dubeyko.com>
2025-11-12 15:25:23 -08:00