Commit graph

1,308,922 commits

Author SHA1 Message Date
Chandan Babu R
22d5a8e52d xfs: atomic file content exchanges [v30.3 03/16]
This series creates a new XFS_IOC_EXCHANGE_RANGE ioctl to exchange
 ranges of bytes between two files atomically.
 
 This new functionality enables data storage programs to stage and commit
 file updates such that reader programs will see either the old contents
 or the new contents in their entirety, with no chance of torn writes.  A
 successful call completion guarantees that the new contents will be seen
 even if the system fails.
 
 The ability to exchange file fork mappings between files in this manner
 is critical to supporting online filesystem repair, which is built upon
 the strategy of constructing a clean copy of a damaged structure and
 committing the new structure into the metadata file atomically.  The
 ioctls exist to facilitate testing of the new functionality and to
 enable future application program designs.
 
 User programs will be able to update files atomically by opening an
 O_TMPFILE, reflinking the source file to it, making whatever updates
 they want to make, and exchange the relevant ranges of the temp file
 with the original file.  If the updates are aligned with the file block
 size, a new (since v2) flag provides for exchanging only the written
 areas.  Note that application software must quiesce writes to the file
 while it stages an atomic update.  This will be addressed by a
 subsequent series.
 
 This mechanism solves the clunkiness of two existing atomic file update
 mechanisms: for O_TRUNC + rewrite, this eliminates the brief period
 where other programs can see an empty file.  For create tempfile +
 rename, the need to copy file attributes and extended attributes for
 each file update is eliminated.
 
 However, this method introduces its own awkwardness -- any program
 initiating an exchange now needs to have a way to signal to other
 programs that the file contents have changed.  For file access mediated
 via read and write, fanotify or inotify are probably sufficient.  For
 mmaped files, that may not be fast enough.
 
 Here is the proposed manual page:
 
 IOCTL-XFS-EXCHANGE-RANGE(2System Calls ManuIOCTL-XFS-EXCHANGE-RANGE(2)
 
 NAME
        ioctl_xfs_exchange_range  -  exchange  the contents of parts of
        two files
 
 SYNOPSIS
        #include <sys/ioctl.h>
        #include <xfs/xfs_fs.h>
 
        int ioctl(int file2_fd, XFS_IOC_EXCHANGE_RANGE, struct  xfs_ex‐
        change_range *arg);
 
 DESCRIPTION
        Given  a  range  of bytes in a first file file1_fd and a second
        range of bytes in a second file  file2_fd,  this  ioctl(2)  ex‐
        changes the contents of the two ranges.
 
        Exchanges  are  atomic  with  regards to concurrent file opera‐
        tions.  Implementations must guarantee that readers see  either
        the old contents or the new contents in their entirety, even if
        the system fails.
 
        The system call parameters are conveyed in  structures  of  the
        following form:
 
            struct xfs_exchange_range {
                __s32    file1_fd;
                __u32    pad;
                __u64    file1_offset;
                __u64    file2_offset;
                __u64    length;
                __u64    flags;
            };
 
        The field pad must be zero.
 
        The  fields file1_fd, file1_offset, and length define the first
        range of bytes to be exchanged.
 
        The fields file2_fd, file2_offset, and length define the second
        range of bytes to be exchanged.
 
        Both  files must be from the same filesystem mount.  If the two
        file descriptors represent the same file, the byte ranges  must
        not  overlap.   Most  disk-based  filesystems  require that the
        starts of both ranges must be aligned to the file  block  size.
        If  this  is  the  case, the ends of the ranges must also be so
        aligned unless the XFS_EXCHANGE_RANGE_TO_EOF flag is set.
 
        The field flags control the behavior of the exchange operation.
 
            XFS_EXCHANGE_RANGE_TO_EOF
                   Ignore the length parameter.  All bytes in  file1_fd
                   from  file1_offset to EOF are moved to file2_fd, and
                   file2's size is set to  (file2_offset+(file1_length-
                   file1_offset)).   Meanwhile, all bytes in file2 from
                   file2_offset to EOF are moved to file1  and  file1's
                   size    is   set   to   (file1_offset+(file2_length-
                   file2_offset)).
 
            XFS_EXCHANGE_RANGE_DSYNC
                   Ensure that all modified in-core data in  both  file
                   ranges  and  all  metadata updates pertaining to the
                   exchange operation are flushed to persistent storage
                   before  the  call  returns.  Opening either file de‐
                   scriptor with O_SYNC or O_DSYNC will have  the  same
                   effect.
 
            XFS_EXCHANGE_RANGE_FILE1_WRITTEN
                   Only  exchange sub-ranges of file1_fd that are known
                   to contain data  written  by  application  software.
                   Each  sub-range  may  be  expanded (both upwards and
                   downwards) to align with the file  allocation  unit.
                   For files on the data device, this is one filesystem
                   block.  For files on the realtime  device,  this  is
                   the realtime extent size.  This facility can be used
                   to implement fast atomic  scatter-gather  writes  of
                   any  complexity for software-defined storage targets
                   if all writes are aligned  to  the  file  allocation
                   unit.
 
            XFS_EXCHANGE_RANGE_DRY_RUN
                   Check  the parameters and the feasibility of the op‐
                   eration, but do not change anything.
 
 RETURN VALUE
        On error, -1 is returned, and errno is set to indicate the  er‐
        ror.
 
 ERRORS
        Error  codes can be one of, but are not limited to, the follow‐
        ing:
 
        EBADF  file1_fd is not open for reading and writing or is  open
               for  append-only  writes;  or  file2_fd  is not open for
               reading and writing or is open for append-only writes.
 
        EINVAL The parameters are not correct for  these  files.   This
               error  can  also appear if either file descriptor repre‐
               sents a device, FIFO, or socket.  Disk filesystems  gen‐
               erally  require  the  offset  and length arguments to be
               aligned to the fundamental block sizes of both files.
 
        EIO    An I/O error occurred.
 
        EISDIR One of the files is a directory.
 
        ENOMEM The kernel was unable to allocate sufficient  memory  to
               perform the operation.
 
        ENOSPC There  is  not  enough  free space in the filesystem ex‐
               change the contents safely.
 
        EOPNOTSUPP
               The filesystem does not support exchanging bytes between
               the two files.
 
        EPERM  file1_fd or file2_fd are immutable.
 
        ETXTBSY
               One of the files is a swap file.
 
        EUCLEAN
               The filesystem is corrupt.
 
        EXDEV  file1_fd  and  file2_fd  are  not  on  the  same mounted
               filesystem.
 
 CONFORMING TO
        This API is XFS-specific.
 
 USE CASES
        Several use cases are imagined for this system  call.   In  all
        cases, application software must coordinate updates to the file
        because the exchange is performed unconditionally.
 
        The first is a data storage program that wants to  commit  non-
        contiguous  updates  to a file atomically and coordinates write
        access to that file.  This can be done by creating a  temporary
        file, calling FICLONE(2) to share the contents, and staging the
        updates into the temporary file.  The FULL_FILES flag is recom‐
        mended  for this purpose.  The temporary file can be deleted or
        punched out afterwards.
 
        An example program might look like this:
 
            int fd = open("/some/file", O_RDWR);
            int temp_fd = open("/some", O_TMPFILE | O_RDWR);
 
            ioctl(temp_fd, FICLONE, fd);
 
            /* append 1MB of records */
            lseek(temp_fd, 0, SEEK_END);
            write(temp_fd, data1, 1000000);
 
            /* update record index */
            pwrite(temp_fd, data1, 600, 98765);
            pwrite(temp_fd, data2, 320, 54321);
            pwrite(temp_fd, data2, 15, 0);
 
            /* commit the entire update */
            struct xfs_exchange_range args = {
                .file1_fd = temp_fd,
                .flags = XFS_EXCHANGE_RANGE_TO_EOF,
            };
 
            ioctl(fd, XFS_IOC_EXCHANGE_RANGE, &args);
 
        The second is a software-defined  storage  host  (e.g.  a  disk
        jukebox)  which  implements an atomic scatter-gather write com‐
        mand.  Provided the exported disk's logical block size  matches
        the file's allocation unit size, this can be done by creating a
        temporary file and writing the data at the appropriate offsets.
        It  is  recommended that the temporary file be truncated to the
        size of the regular file before any writes are  staged  to  the
        temporary  file  to avoid issues with zeroing during EOF exten‐
        sion.  Use this call with the FILE1_WRITTEN  flag  to  exchange
        only  the  file  allocation  units involved in the emulated de‐
        vice's write command.  The temporary file should  be  truncated
        or  punched out completely before being reused to stage another
        write.
 
        An example program might look like this:
 
            int fd = open("/some/file", O_RDWR);
            int temp_fd = open("/some", O_TMPFILE | O_RDWR);
            struct stat sb;
            int blksz;
 
            fstat(fd, &sb);
            blksz = sb.st_blksize;
 
            /* land scatter gather writes between 100fsb and 500fsb */
            pwrite(temp_fd, data1, blksz * 2, blksz * 100);
            pwrite(temp_fd, data2, blksz * 20, blksz * 480);
            pwrite(temp_fd, data3, blksz * 7, blksz * 257);
 
            /* commit the entire update */
            struct xfs_exchange_range args = {
                .file1_fd = temp_fd,
                .file1_offset = blksz * 100,
                .file2_offset = blksz * 100,
                .length       = blksz * 400,
                .flags        = XFS_EXCHANGE_RANGE_FILE1_WRITTEN |
                                XFS_EXCHANGE_RANGE_FILE1_DSYNC,
            };
 
            ioctl(fd, XFS_IOC_EXCHANGE_RANGE, &args);
 
 NOTES
        Some filesystems may limit the amount of data or the number  of
        extents that can be exchanged in a single call.
 
 SEE ALSO
        ioctl(2)
 
 XFS                           2024-02-10   IOCTL-XFS-EXCHANGE-RANGE(2)
 
 The reference implementation in XFS creates a new log incompat feature
 and log intent items to track high level progress of swapping ranges of
 two files and finish interrupted work if the system goes down.  Sample
 code can be found in the corresponding changes to xfs_io to exercise the
 use case mentioned above.
 
 Note that this function is /not/ the O_DIRECT atomic untorn file writes
 concept that has also been floating around for years.  It is also not
 the RWF_ATOMIC patchset that has been shared.  This RFC is constructed
 entirely in software, which means that there are no limitations other
 than the general filesystem limits.
 
 As a side note, the original motivation behind the kernel functionality
 is online repair of file-based metadata.  The atomic file content
 exchange is implemented as an atomic exchange of file fork mappings,
 which means that we can implement online reconstruction of extended
 attributes and directories by building a new one in another inode and
 exchanging the contents.
 
 Subsequent patchsets adapt the online filesystem repair code to use
 atomic file exchanges.  This enables repair functions to construct a
 clean copy of a directory, xattr information, symbolic links, realtime
 bitmaps, and realtime summary information in a temporary inode.  If this
 completes successfully, the new contents can be committed atomically
 into the inode being repaired.  This is essential to avoid making
 corruption problems worse if the system goes down in the middle of
 running repair.
 
 For userspace, this series also includes the userspace pieces needed to
 test the new functionality, and a sample implementation of atomic file
 updates.
 
 This has been running on the djcloud for months with no problems.  Enjoy!
 
 Signed-off-by: Darrick J. Wong <djwong@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQQ2qTKExjcn+O1o2YRKO3ySh0YRpgUCZh23UgAKCRBKO3ySh0YR
 pmYQAQCGwoAev/oRzIJrZmbpzNaU9w7XEPF+tW3vJSX6tlxG+wD8DIi4kTAplu/9
 i860EFqZp5MuwHyGVDCac0owigtt6wk=
 =Lsls
 -----END PGP SIGNATURE-----

Merge tag 'atomic-file-updates-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: atomic file content exchanges

This series creates a new XFS_IOC_EXCHANGE_RANGE ioctl to exchange
ranges of bytes between two files atomically.

This new functionality enables data storage programs to stage and commit
file updates such that reader programs will see either the old contents
or the new contents in their entirety, with no chance of torn writes.  A
successful call completion guarantees that the new contents will be seen
even if the system fails.

The ability to exchange file fork mappings between files in this manner
is critical to supporting online filesystem repair, which is built upon
the strategy of constructing a clean copy of a damaged structure and
committing the new structure into the metadata file atomically.  The
ioctls exist to facilitate testing of the new functionality and to
enable future application program designs.

User programs will be able to update files atomically by opening an
O_TMPFILE, reflinking the source file to it, making whatever updates
they want to make, and exchange the relevant ranges of the temp file
with the original file.  If the updates are aligned with the file block
size, a new (since v2) flag provides for exchanging only the written
areas.  Note that application software must quiesce writes to the file
while it stages an atomic update.  This will be addressed by a
subsequent series.

This mechanism solves the clunkiness of two existing atomic file update
mechanisms: for O_TRUNC + rewrite, this eliminates the brief period
where other programs can see an empty file.  For create tempfile +
rename, the need to copy file attributes and extended attributes for
each file update is eliminated.

However, this method introduces its own awkwardness -- any program
initiating an exchange now needs to have a way to signal to other
programs that the file contents have changed.  For file access mediated
via read and write, fanotify or inotify are probably sufficient.  For
mmaped files, that may not be fast enough.

Here is the proposed manual page:

IOCTL-XFS-EXCHANGE-RANGE(2System Calls ManuIOCTL-XFS-EXCHANGE-RANGE(2)

NAME
       ioctl_xfs_exchange_range  -  exchange  the contents of parts of
       two files

SYNOPSIS
       #include <sys/ioctl.h>
       #include <xfs/xfs_fs.h>

       int ioctl(int file2_fd, XFS_IOC_EXCHANGE_RANGE, struct  xfs_ex‐
       change_range *arg);

DESCRIPTION
       Given  a  range  of bytes in a first file file1_fd and a second
       range of bytes in a second file  file2_fd,  this  ioctl(2)  ex‐
       changes the contents of the two ranges.

       Exchanges  are  atomic  with  regards to concurrent file opera‐
       tions.  Implementations must guarantee that readers see  either
       the old contents or the new contents in their entirety, even if
       the system fails.

       The system call parameters are conveyed in  structures  of  the
       following form:

           struct xfs_exchange_range {
               __s32    file1_fd;
               __u32    pad;
               __u64    file1_offset;
               __u64    file2_offset;
               __u64    length;
               __u64    flags;
           };

       The field pad must be zero.

       The  fields file1_fd, file1_offset, and length define the first
       range of bytes to be exchanged.

       The fields file2_fd, file2_offset, and length define the second
       range of bytes to be exchanged.

       Both  files must be from the same filesystem mount.  If the two
       file descriptors represent the same file, the byte ranges  must
       not  overlap.   Most  disk-based  filesystems  require that the
       starts of both ranges must be aligned to the file  block  size.
       If  this  is  the  case, the ends of the ranges must also be so
       aligned unless the XFS_EXCHANGE_RANGE_TO_EOF flag is set.

       The field flags control the behavior of the exchange operation.

           XFS_EXCHANGE_RANGE_TO_EOF
                  Ignore the length parameter.  All bytes in  file1_fd
                  from  file1_offset to EOF are moved to file2_fd, and
                  file2's size is set to  (file2_offset+(file1_length-
                  file1_offset)).   Meanwhile, all bytes in file2 from
                  file2_offset to EOF are moved to file1  and  file1's
                  size    is   set   to   (file1_offset+(file2_length-
                  file2_offset)).

           XFS_EXCHANGE_RANGE_DSYNC
                  Ensure that all modified in-core data in  both  file
                  ranges  and  all  metadata updates pertaining to the
                  exchange operation are flushed to persistent storage
                  before  the  call  returns.  Opening either file de‐
                  scriptor with O_SYNC or O_DSYNC will have  the  same
                  effect.

           XFS_EXCHANGE_RANGE_FILE1_WRITTEN
                  Only  exchange sub-ranges of file1_fd that are known
                  to contain data  written  by  application  software.
                  Each  sub-range  may  be  expanded (both upwards and
                  downwards) to align with the file  allocation  unit.
                  For files on the data device, this is one filesystem
                  block.  For files on the realtime  device,  this  is
                  the realtime extent size.  This facility can be used
                  to implement fast atomic  scatter-gather  writes  of
                  any  complexity for software-defined storage targets
                  if all writes are aligned  to  the  file  allocation
                  unit.

           XFS_EXCHANGE_RANGE_DRY_RUN
                  Check  the parameters and the feasibility of the op‐
                  eration, but do not change anything.

RETURN VALUE
       On error, -1 is returned, and errno is set to indicate the  er‐
       ror.

ERRORS
       Error  codes can be one of, but are not limited to, the follow‐
       ing:

       EBADF  file1_fd is not open for reading and writing or is  open
              for  append-only  writes;  or  file2_fd  is not open for
              reading and writing or is open for append-only writes.

       EINVAL The parameters are not correct for  these  files.   This
              error  can  also appear if either file descriptor repre‐
              sents a device, FIFO, or socket.  Disk filesystems  gen‐
              erally  require  the  offset  and length arguments to be
              aligned to the fundamental block sizes of both files.

       EIO    An I/O error occurred.

       EISDIR One of the files is a directory.

       ENOMEM The kernel was unable to allocate sufficient  memory  to
              perform the operation.

       ENOSPC There  is  not  enough  free space in the filesystem ex‐
              change the contents safely.

       EOPNOTSUPP
              The filesystem does not support exchanging bytes between
              the two files.

       EPERM  file1_fd or file2_fd are immutable.

       ETXTBSY
              One of the files is a swap file.

       EUCLEAN
              The filesystem is corrupt.

       EXDEV  file1_fd  and  file2_fd  are  not  on  the  same mounted
              filesystem.

CONFORMING TO
       This API is XFS-specific.

USE CASES
       Several use cases are imagined for this system  call.   In  all
       cases, application software must coordinate updates to the file
       because the exchange is performed unconditionally.

       The first is a data storage program that wants to  commit  non-
       contiguous  updates  to a file atomically and coordinates write
       access to that file.  This can be done by creating a  temporary
       file, calling FICLONE(2) to share the contents, and staging the
       updates into the temporary file.  The FULL_FILES flag is recom‐
       mended  for this purpose.  The temporary file can be deleted or
       punched out afterwards.

       An example program might look like this:

           int fd = open("/some/file", O_RDWR);
           int temp_fd = open("/some", O_TMPFILE | O_RDWR);

           ioctl(temp_fd, FICLONE, fd);

           /* append 1MB of records */
           lseek(temp_fd, 0, SEEK_END);
           write(temp_fd, data1, 1000000);

           /* update record index */
           pwrite(temp_fd, data1, 600, 98765);
           pwrite(temp_fd, data2, 320, 54321);
           pwrite(temp_fd, data2, 15, 0);

           /* commit the entire update */
           struct xfs_exchange_range args = {
               .file1_fd = temp_fd,
               .flags = XFS_EXCHANGE_RANGE_TO_EOF,
           };

           ioctl(fd, XFS_IOC_EXCHANGE_RANGE, &args);

       The second is a software-defined  storage  host  (e.g.  a  disk
       jukebox)  which  implements an atomic scatter-gather write com‐
       mand.  Provided the exported disk's logical block size  matches
       the file's allocation unit size, this can be done by creating a
       temporary file and writing the data at the appropriate offsets.
       It  is  recommended that the temporary file be truncated to the
       size of the regular file before any writes are  staged  to  the
       temporary  file  to avoid issues with zeroing during EOF exten‐
       sion.  Use this call with the FILE1_WRITTEN  flag  to  exchange
       only  the  file  allocation  units involved in the emulated de‐
       vice's write command.  The temporary file should  be  truncated
       or  punched out completely before being reused to stage another
       write.

       An example program might look like this:

           int fd = open("/some/file", O_RDWR);
           int temp_fd = open("/some", O_TMPFILE | O_RDWR);
           struct stat sb;
           int blksz;

           fstat(fd, &sb);
           blksz = sb.st_blksize;

           /* land scatter gather writes between 100fsb and 500fsb */
           pwrite(temp_fd, data1, blksz * 2, blksz * 100);
           pwrite(temp_fd, data2, blksz * 20, blksz * 480);
           pwrite(temp_fd, data3, blksz * 7, blksz * 257);

           /* commit the entire update */
           struct xfs_exchange_range args = {
               .file1_fd = temp_fd,
               .file1_offset = blksz * 100,
               .file2_offset = blksz * 100,
               .length       = blksz * 400,
               .flags        = XFS_EXCHANGE_RANGE_FILE1_WRITTEN |
                               XFS_EXCHANGE_RANGE_FILE1_DSYNC,
           };

           ioctl(fd, XFS_IOC_EXCHANGE_RANGE, &args);

NOTES
       Some filesystems may limit the amount of data or the number  of
       extents that can be exchanged in a single call.

SEE ALSO
       ioctl(2)

XFS                           2024-02-10   IOCTL-XFS-EXCHANGE-RANGE(2)

The reference implementation in XFS creates a new log incompat feature
and log intent items to track high level progress of swapping ranges of
two files and finish interrupted work if the system goes down.  Sample
code can be found in the corresponding changes to xfs_io to exercise the
use case mentioned above.

Note that this function is /not/ the O_DIRECT atomic untorn file writes
concept that has also been floating around for years.  It is also not
the RWF_ATOMIC patchset that has been shared.  This RFC is constructed
entirely in software, which means that there are no limitations other
than the general filesystem limits.

As a side note, the original motivation behind the kernel functionality
is online repair of file-based metadata.  The atomic file content
exchange is implemented as an atomic exchange of file fork mappings,
which means that we can implement online reconstruction of extended
attributes and directories by building a new one in another inode and
exchanging the contents.

Subsequent patchsets adapt the online filesystem repair code to use
atomic file exchanges.  This enables repair functions to construct a
clean copy of a directory, xattr information, symbolic links, realtime
bitmaps, and realtime summary information in a temporary inode.  If this
completes successfully, the new contents can be committed atomically
into the inode being repaired.  This is essential to avoid making
corruption problems worse if the system goes down in the middle of
running repair.

For userspace, this series also includes the userspace pieces needed to
test the new functionality, and a sample implementation of atomic file
updates.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>

* tag 'atomic-file-updates-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: enable logged file mapping exchange feature
  docs: update swapext -> exchmaps language
  xfs: capture inode generation numbers in the ondisk exchmaps log item
  xfs: support non-power-of-two rtextsize with exchange-range
  xfs: make file range exchange support realtime files
  xfs: condense symbolic links after a mapping exchange operation
  xfs: condense directories after a mapping exchange operation
  xfs: condense extended attributes after a mapping exchange operation
  xfs: add error injection to test file mapping exchange recovery
  xfs: bind together the front and back ends of the file range exchange code
  xfs: create deferred log items for file mapping exchanges
  xfs: introduce a file mapping exchange log intent item
  xfs: create a incompat flag for atomic file mapping exchanges
  xfs: introduce new file range exchange ioctl
  vfs: export remap and write check helpers
2024-04-16 11:25:09 +05:30
Chandan Babu R
4ec2e3c167 xfs: refactorings for atomic file content exchanges [v30.3 02/16]
This series applies various cleanups and refactorings to file IO
 handling code ahead of the main series to implement atomic file content
 exchanges.
 
 This has been running on the djcloud for months with no problems.  Enjoy!
 
 Signed-off-by: Darrick J. Wong <djwong@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQQ2qTKExjcn+O1o2YRKO3ySh0YRpgUCZh23UgAKCRBKO3ySh0YR
 pginAQCcZq5bFYJGYj4UInUOfDDjuq8R9Rl8DGDhnTFb8FxAUAD+Jsol2/wQduII
 Bly/+Hegen2BoayNuf3iZG7RYkJWygo=
 =tHoT
 -----END PGP SIGNATURE-----

Merge tag 'file-exchange-refactorings-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: refactorings for atomic file content exchanges

This series applies various cleanups and refactorings to file IO
handling code ahead of the main series to implement atomic file content
exchanges.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>

* tag 'file-exchange-refactorings-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: constify xfs_bmap_is_written_extent
  xfs: refactor non-power-of-two alignment checks
  xfs: hoist multi-fsb allocation unit detection to a helper
  xfs: create a new helper to return a file's allocation unit
  xfs: declare xfs_file.c symbols in xfs_file.h
  xfs: move xfs_iops.c declarations out of xfs_inode.h
  xfs: move inode lease breaking functions to xfs_inode.c
2024-04-16 11:17:28 +05:30
Chandan Babu R
ebe0f798e1 xfs: improve log incompat feature handling [v30.3 01/16]
This patchset improves the performance of log incompat feature bit
 handling by making a few changes to how the filesystem handles them.
 First, we now only clear the bits during a clean unmount to reduce calls
 to the (expensive) upgrade function to once per bit per mount.  Second,
 we now only allow incompat feature upgrades for sysadmins or if the
 sysadmin explicitly allows it via mount option.  Currently the only log
 incompat user is logged xattrs, which requires CONFIG_XFS_DEBUG=y, so
 there should be no user visible impact to this change.
 
 This has been running on the djcloud for months with no problems.  Enjoy!
 
 Signed-off-by: Darrick J. Wong <djwong@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQQ2qTKExjcn+O1o2YRKO3ySh0YRpgUCZh23UgAKCRBKO3ySh0YR
 pharAQDYPe2dmXxQ28fUC1GMy38DLCt0f/XXLkhUI0yeC+mbcQD/aLWU5MCy5TGN
 OF3HQwrgL9+SZWwhqRbu8hPG7ZLBWQQ=
 =yieh
 -----END PGP SIGNATURE-----

Merge tag 'log-incompat-permissions-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: improve log incompat feature handling

This patchset improves the performance of log incompat feature bit
handling by making a few changes to how the filesystem handles them.
First, we now only clear the bits during a clean unmount to reduce calls
to the (expensive) upgrade function to once per bit per mount.  Second,
we now only allow incompat feature upgrades for sysadmins or if the
sysadmin explicitly allows it via mount option.  Currently the only log
incompat user is logged xattrs, which requires CONFIG_XFS_DEBUG=y, so
there should be no user visible impact to this change.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>

* tag 'log-incompat-permissions-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: only clear log incompat flags at clean unmount
  xfs: fix error bailout in xrep_abt_build_new_trees
  xfs: fix potential AGI <-> ILOCK ABBA deadlock in xrep_dinode_findmode_walk_directory
  xfs: fix an AGI lock acquisition ordering problem in xrep_dinode_findmode
  xfs: pass xfs_buf lookup flags to xfs_*read_agi
2024-04-16 11:09:58 +05:30
Konrad Dybcio
39c596304e arm64: dts: qcom: Add SM8550 Xperia 1 V
Add support for Sony Xperia 1 V, a.k.a PDX234. This device is a part
of the SoMC SM8550 Yodo platform.

This commit brings support for:
* Remoteprocs (sans modem for now)
* Flash LED (the notification LED is gone :((((()
* SD Card
* USB (*including SuperSpeed*) + PMIC_GLINK (it's funky, requires a replug
  with an cable flip sometimes..)
* Most regulators
* Part of I2C-connected peripherals (notably no touch due to a
driver bug)
* PCIe0 (PCIe1 is unused)

Do note display via simplefb is not supported, as the display is blanked
upon exiting XBL.

To create a working boot image, you need to run:
cat arch/arm64/boot/Image.gz arch/arm64/boot/dts/qcom/sm8550-sony-xperia-\
yodo-pdx234.dtb > .Image.gz-dtb

mkbootimg \
--kernel .Image.gz-dtb \
--ramdisk some_initrd.img \
--pagesize 4096 \
--base 0x0 \
--kernel_offset 0x8000 \
--ramdisk_offset 0x1000000 \
--tags_offset 0x100 \
--cmdline "SOME_CMDLINE" \
--dtb_offset 0x1f00000 \
--header_version 2 \
-o boot.img-sony-xperia-pdx234

Then, you need to flash it on the device and get rid of all the
vendor_boot/dtbo mess:

// You have to either pull vbmeta{"","_system"} from
// /dev/block/bootdevice/by-name/ or build one as a part of AOSP build process
fastboot --disable-verity --disable-verification flash vbmeta vbmeta.img
fastboot --disable-verity --disable-verification flash vbmeta_system \
vbmeta_system.img

fastboot flash boot boot.img-sony-xperia-pdx234
fastboot erase vendor_boot
fastboot erase recovery
fastboot flash dtbo emptydtbo.img
fastboot erase init_boot // ? I don't remember if it's necessary, sorry
fastboot continue

Where emptydtbo.img is a tiny file that consists of 2 bytes (all zeroes), doing
a "fastboot erase" won't cut it, the bootloader will go crazy and things will
fall apart when it tries to overlay random bytes from an empty partition onto a
perfectly good appended DTB.

Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Reviewed-by: Neil Armstrong <neil.armstrong@linaro.org>
Link: https://lore.kernel.org/r/20240210-topic-1v-v1-7-fda0db38e29b@linaro.org
Signed-off-by: Bjorn Andersson <andersson@kernel.org>
2024-04-15 20:54:03 -05:00
Konrad Dybcio
6e4f7e5399 arm64: dts: qcom: sm8550: Mark DWC3 as dma-coherent
In a fairly new development, Qualcomm somehow made the DWC3 block
cache-coherent. Annotate that.

Fixes: 7f7e5c1b03 ("arm64: dts: qcom: sm8550: Add USB PHYs and controller nodes")
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Reviewed-by: Neil Armstrong <neil.armstrong@linaro.org>
Link: https://lore.kernel.org/r/20240210-topic-1v-v1-6-fda0db38e29b@linaro.org
Signed-off-by: Bjorn Andersson <andersson@kernel.org>
2024-04-15 20:54:03 -05:00
Konrad Dybcio
d18b5477dc arm64: dts: qcom: sm8550: Add missing DWC3 quirks
As expected, Qualcomm DWC3 implementation come with a sizable number
of quirks. Make sure to account for all of them.

Fixes: 7f7e5c1b03 ("arm64: dts: qcom: sm8550: Add USB PHYs and controller nodes")
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Reviewed-by: Neil Armstrong <neil.armstrong@linaro.org>
Link: https://lore.kernel.org/r/20240210-topic-1v-v1-5-fda0db38e29b@linaro.org
Signed-off-by: Bjorn Andersson <andersson@kernel.org>
2024-04-15 20:54:03 -05:00
Konrad Dybcio
93395f9a8d arm64: dts: qcom: sm8550: Mark APPS SMMU as dma-coherent
Like on earlier flagship Qualcomm SoCs, the SMMU is dma-coherent.
Mark it as such.

Fixes: ffc50b2d38 ("arm64: dts: qcom: Add base SM8550 dtsi")
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Reviewed-by: Neil Armstrong <neil.armstrong@linaro.org>
Link: https://lore.kernel.org/r/20240210-topic-1v-v1-4-fda0db38e29b@linaro.org
Signed-off-by: Bjorn Andersson <andersson@kernel.org>
2024-04-15 20:54:03 -05:00
Konrad Dybcio
91fc74458d arm64: dts: qcom: sm8550: Mark QUPs and GPI dma-coherent
These peripherals are DMA-coherent on 8550. Mark them as such.

Interestingly enough, the I2C master hubs are not.

Fixes: ffc50b2d38 ("arm64: dts: qcom: Add base SM8550 dtsi")
Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Reviewed-by: Neil Armstrong <neil.armstrong@linaro.org>
Link: https://lore.kernel.org/r/20240210-topic-1v-v1-3-fda0db38e29b@linaro.org
Signed-off-by: Bjorn Andersson <andersson@kernel.org>
2024-04-15 20:54:03 -05:00
Konrad Dybcio
212729551c dt-bindings: arm: qcom: Add Xperia 1 V
Document the SM8550 SONY Xperia 1 V (PDX234) board.

Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Link: https://lore.kernel.org/r/20240210-topic-1v-v1-2-fda0db38e29b@linaro.org
Signed-off-by: Bjorn Andersson <andersson@kernel.org>
2024-04-15 20:54:03 -05:00
Kees Cook
f4626c12e4 ubsan: Add awareness of signed integer overflow traps
On arm64, UBSAN traps can be decoded from the trap instruction. Add the
add, sub, and mul overflow trap codes now that CONFIG_UBSAN_SIGNED_WRAP
exists. Seen under clang 19:

  Internal error: UBSAN: unrecognized failure code: 00000000f2005515 [#1] PREEMPT SMP

Reported-by: Nathan Chancellor <nathan@kernel.org>
Closes: https://lore.kernel.org/lkml/20240411-fix-ubsan-in-hardening-config-v1-0-e0177c80ffaa@kernel.org
Fixes: 557f8c582a ("ubsan: Reintroduce signed overflow sanitizer")
Tested-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/r/20240415182832.work.932-kees@kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>
2024-04-15 17:42:43 -07:00
Andy Shevchenko
bb40996267
spi: Consistently use BIT for cs_index_mask (part 2)
For some reason the commit 1209c5566f ("spi: Consistently use BIT
for cs_index_mask") missed one place to change, do it here to finish
the job.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://lore.kernel.org/r/20240415184757.1198149-1-andriy.shevchenko@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
2024-04-16 08:45:18 +09:00
Andy Shevchenko
d707530b1e
spi: Introduce spi_for_each_valid_cs() in order of deduplication
In order of deduplication and better maintenance introduce a new
spi_for_each_valid_cs() helper macro.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://lore.kernel.org/r/20240415193340.1279360-3-andriy.shevchenko@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
2024-04-16 08:45:13 +09:00
Andy Shevchenko
e81582c080
spi: Extract spi_toggle_csgpiod() helper for better maintanance
The multi-CS support splits the comment and the code in the spi_set_cs().
To avoid this in the future extract spi_toggle_csgpiod() helper.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://lore.kernel.org/r/20240415193340.1279360-2-andriy.shevchenko@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
2024-04-16 08:45:12 +09:00
Shengjiu Wang
62c48dd33b
ASoC: dt-bindings: fsl-asoc-card: Add compatbile string for wm8904 codec
The wm8904 codec is used on an i.MX95 Toradex board.

Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
Link: https://lore.kernel.org/r/1713165456-3494-3-git-send-email-shengjiu.wang@nxp.com
Signed-off-by: Mark Brown <broonie@kernel.org>
2024-04-16 08:45:06 +09:00
Shengjiu Wang
51f67862ea
ASoC: fsl-asoc-card: add wm8904 codec support
wm8904 codec is used on i.MX95 Toradex board

Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
Link: https://lore.kernel.org/r/1713165456-3494-2-git-send-email-shengjiu.wang@nxp.com
Signed-off-by: Mark Brown <broonie@kernel.org>
2024-04-16 08:45:05 +09:00
Jack Yu
cebfbc89ae
ASoC: rt715: add vendor clear control register
Add vendor clear control register in readable register's
callback function. This prevents an access failure reported
in Intel CI tests.

Signed-off-by: Jack Yu <jack.yu@realtek.com>
Closes: https://github.com/thesofproject/linux/issues/4860
Tested-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Link: https://lore.kernel.org/r/6a103ce9134d49d8b3941172c87a7bd4@realtek.com
Signed-off-by: Mark Brown <broonie@kernel.org>
2024-04-16 08:44:59 +09:00
Fenglin Wu
48c0687a32 Input: pm8xxx-vibrator - correct VIB_MAX_LEVELS calculation
The output voltage is inclusive hence the max level calculation is
off-by-one-step. Correct it.

iWhile we are at it also add a define for the step size instead of
using the magic value.

Fixes: 11205bb63e ("Input: add support for pm8xxx based vibrator driver")
Signed-off-by: Fenglin Wu <quic_fenglinw@quicinc.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Link: https://lore.kernel.org/r/20240412-pm8xxx-vibrator-new-design-v10-1-0ec0ad133866@quicinc.com
Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
2024-04-15 16:21:45 -07:00
Ricardo Ribalda
ba2ec9c4f0 Input: sur40 - convert le16 to cpu before use
Smatch found this issue:
drivers/input/touchscreen/sur40.c:424:55: warning: incorrect type in argument 2 (different base types)
drivers/input/touchscreen/sur40.c:424:55:    expected int key
drivers/input/touchscreen/sur40.c:424:55:    got restricted __le16 [usertype] blob_id

Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Link: https://lore.kernel.org/r/20240410-smatch-v1-6-785d009a852b@chromium.org
Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
2024-04-15 16:21:44 -07:00
Ricardo B. Marliere
e6faf2b750 peci: constify the struct device_type usage
Since commit aed65af1cc ("drivers: make device_type const"), the driver
core can properly handle constant struct device_type. Move the
peci_controller_type and peci_device_type variables to be constant
structures as well, placing it into read-only memory which can not be
modified at runtime.

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: "Ricardo B. Marliere" <ricardo@marliere.net>
Reviewed-by: Iwona Winiarska <iwona.winiarska@intel.com>
Link: https://lore.kernel.org/r/20240219-device_cleanup-peci-v1-1-0727662616f7@marliere.net
Signed-off-by: Iwona Winiarska <iwona.winiarska@intel.com>
2024-04-16 00:14:43 +02:00
Ricardo B. Marliere
ed8c2dad25 peci: Make peci_bus_type const
Now that the driver core can properly handle constant struct bus_type,
move the peci_bus_type variable to be a constant structure as well,
placing it into read-only memory which can not be modified at runtime.

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: "Ricardo B. Marliere" <ricardo@marliere.net>
Link: https://lore.kernel.org/r/20240210-bus_cleanup-peci-v1-1-1e64bef6efc0@marliere.net
Signed-off-by: Iwona Winiarska <iwona.winiarska@intel.com>
2024-04-16 00:14:36 +02:00
Ricardo Ribalda
836e254852 media: usb: siano: Fix allocation of urbs
USB urbs must be allocated with usb_alloc_urb. Quoting the manual

Only use this function (usb_init_urb) if you _really_ understand what you
are doing.

Fix the following smatch error:

drivers/media/usb/siano/smsusb.c:53:38: warning: array of flexible structures

Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
c3224f4147 media: ttpci: coding style fixes: logging
This patch fixes the following checkpatch warnings:

WARNING:PRINTK_WITHOUT_KERN_LEVEL: printk() should include KERN_<LEVEL> facility level
WARNING:PREFER_PR_LEVEL: Prefer [subsystem eg: netdev]_dbg([subsystem]dev, ... then dev_dbg(dev, ... then pr_debug(...  to printk(KERN_DEBUG ...
WARNING:EMBEDDED_FUNCTION_NAME: Prefer using '"%s...", __func__' to using 'ciintf_slot_ts_enable', this function's name, in a string
CHECK:MACRO_ARG_PRECEDENCE: Macro argument 'level' may be better as '(level)' to avoid precedence issues

There was no consistent style of logging functions used so far.
Convert all logging to the appropriate pr_* macros.
Add a global pr_fmt macro for the whole driver.

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
58557349a8 media: ttpci: coding style fixes: miscellaneous
This patch fixes the following checkpatch warnings:

WARNING:RETURN_VOID: void function return statements are not generally useful
WARNING:TRAILING_SEMICOLON: macros should not use a trailing semicolon

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
22a8f8c216 media: ttpci: coding style fixes: constant_comparsation
This patch fixes the following checkpatch warnings:

WARNING:CONSTANT_COMPARISON: Comparisons should place the constant on the right side of the test

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
2b8db3ebaf media: ttpci: coding style fixes: trailing_statements
This patch fixes the following checkpatch errors:

ERROR:TRAILING_STATEMENTS: trailing statements should be on next line

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
23aebcf131 media: ttpci: coding style fixes: assign_in_if
This patch fixes the following checkpatch errors:

ERROR:ASSIGN_IN_IF: do not use assignment in if condition

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
f126e8a44a media: ttpci: coding style fixes: export_symbol
This patch fixes the following checkpatch warnings:

WARNING:EXPORT_SYMBOL: EXPORT_SYMBOL(foo); should immediately follow its function/variable

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
e4fc63e2a9 media: ttpci: coding style fixes: braces
This patch fixes the following checkpatch warnings and errors:

WARNING:BRACES: braces {} are not necessary for any arm of this statement
WARNING:BRACES: braces {} are not necessary for single statement blocks
ERROR:OPEN_BRACE: that open brace { should be on the previous line
CHECK:BRACES: braces {} should be used on all arms of this statement
CHECK:BRACES: Unbalanced braces around else statement

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
004e24f9f2 media: ttpci: coding style fixes: comments
This patch fixes the following checkpatch warnings:

WARNING:BLOCK_COMMENT_STYLE: Block comments use * on subsequent lines
WARNING:BLOCK_COMMENT_STYLE: Block comments use a trailing */ on a separate line
WARNING: It's generally not useful to have the filename in the file

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:53 +02:00
Stefan Herdler
77f5be68de media: ttpci: coding style fixes: whitespace
It fixes the following checkpatch errors:

ERROR:SPACING: space prohibited before that ',' (ctx:WxW)
ERROR:SPACING: space required after that ',' (ctx:VxV)
ERROR:SWITCH_CASE_INDENT_LEVEL: switch and case should be at the same indent

This patch contains the large multi line blocks.

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:52 +02:00
Stefan Herdler
a5bc649b1e media: ttpci: coding style fixes: whitespace and newlines
This patch fixes the following checkpatch warnings and errors:

ERROR:POINTER_LOCATION: "foo * bar" should be "foo *bar"
ERROR:POINTER_LOCATION: "foo* bar" should be "foo *bar"
ERROR:SPACING: space prohibited after that open parenthesis '('
ERROR:SPACING: space prohibited before that close parenthesis ')'
ERROR:SPACING: space prohibited before that ',' (ctx:WxW)
ERROR:SPACING: space required after that ',' (ctx:VxV)
ERROR:SPACING: space required before that '-' (ctx:OxV)
ERROR:SPACING: space required before the open parenthesis '('
ERROR:SPACING: spaces required around that '!=' (ctx:VxO)
ERROR:SPACING: spaces required around that '<' (ctx:VxV)
ERROR:SPACING: spaces required around that '=' (ctx:VxV)
ERROR:SPACING: spaces required around that '>=' (ctx:VxV)
WARNING:SPACING: space prohibited between function name and open parenthesis '('
WARNING:TABSTOP: Statements should start on a tabstop
WARNING:LEADING_SPACE: please, no spaces at the start of a line
WARNING:SUSPECT_CODE_INDENT: suspect code indent for conditional statements
WARNING:LINE_SPACING: Missing a blank line after declarations

Signed-off-by: Stefan Herdler <herdler@nurfuerspam.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2024-04-16 00:02:52 +02:00
Darrick J. Wong
df76047147 xfs: unlock new repair tempfiles after creation
After creation, drop the ILOCK on temporary files that have been created
to stage a repair.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:03 -07:00
Darrick J. Wong
34ef5e17d5 xfs: don't pick up IOLOCK during rmapbt repair scan
Now that we've fixed the directory operations to hold the ILOCK until
they're finished with rmapbt updates for directory shape changes, we no
longer need to take this lock when scanning directories for rmapbt
records.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:02 -07:00
Allison Henderson
69291726ca xfs: Hold inode locks in xfs_rename
Modify xfs_rename to hold all inode locks across a rename operation
We will need this later when we add parent pointers

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:02 -07:00
Allison Henderson
bd5562111d xfs: Hold inode locks in xfs_trans_alloc_dir
Modify xfs_trans_alloc_dir to hold locks after return.  Caller will be
responsible for manual unlock.  We will need this later to hold locks
across parent pointer operations

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:02 -07:00
Allison Henderson
267979b4ce xfs: Hold inode locks in xfs_ialloc
Modify xfs_ialloc to hold locks after return.  Caller will be
responsible for manual unlock.  We will need this later to hold locks
across parent pointer operations

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
[djwong: hold the parent ilocked across transaction rolls too]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:02 -07:00
Darrick J. Wong
67bdcd4999 docs: describe xfs directory tree online fsck
I've added a scrubber that checks the directory tree structure and fixes
them; describe this in the design documentation.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:01 -07:00
Allison Henderson
f103df7635 xfs: Increase XFS_QM_TRANS_MAXDQS to 5
With parent pointers enabled, a rename operation can update up to 5
inodes: src_dp, target_dp, src_ip, target_ip and wip.  This causes
their dquots to a be attached to the transaction chain, so we need
to increase XFS_QM_TRANS_MAXDQS.  This patch also add a helper
function xfs_dqlockn to lock an arbitrary number of dquots.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:01 -07:00
Darrick J. Wong
c91fe20e5a docs: update offline parent pointer repair strategy
Now update how xfs_repair checks and repairs parent pointer info.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:01 -07:00
Allison Henderson
7560c937b4 xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
Renames that generate parent pointer updates can join up to 5
inodes locked in sorted order.  So we need to increase the
number of defer ops inodes and relock them in the same way.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
[djwong: have one sorting function]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:01 -07:00
Darrick J. Wong
b0ffe661fa xfs: fix performance problems when fstrimming a subset of a fragmented AG
On a 10TB filesystem where the free space in each AG is heavily
fragmented, I noticed some very high runtimes on a FITRIM call for the
entire filesystem.  xfs_scrub likes to report progress information on
each phase of the scrub, which means that a strace for the entire
filesystem:

ioctl(3, FITRIM, {start=0x0, len=10995116277760, minlen=0}) = 0 <686.209839>

shows that scrub is uncommunicative for the entire duration.  Reducing
the size of the FITRIM requests to a single AG at a time produces lower
times for each individual call, but even this isn't quite acceptable,
because the time between progress reports are still very high:

Strace for the first 4x 1TB AGs looks like (2):
ioctl(3, FITRIM, {start=0x0, len=1099511627776, minlen=0}) = 0 <68.352033>
ioctl(3, FITRIM, {start=0x10000000000, len=1099511627776, minlen=0}) = 0 <68.760323>
ioctl(3, FITRIM, {start=0x20000000000, len=1099511627776, minlen=0}) = 0 <67.235226>
ioctl(3, FITRIM, {start=0x30000000000, len=1099511627776, minlen=0}) = 0 <69.465744>

I then had the idea to limit the length parameter of each call to a
smallish amount (~11GB) so that we could report progress relatively
quickly, but much to my surprise, each FITRIM call still took ~68
seconds!

Unfortunately, the by-length fstrim implementation handles this poorly
because it walks the entire free space by length index (cntbt), which is
a very inefficient way to walk a subset of the blocks of an AG.

Therefore, create a second implementation that will walk the bnobt and
perform the trims in block number order.  This implementation avoids the
worst problems of the original code, though it lacks the desirable
attribute of freeing the biggest chunks first.

On the other hand, this second implementation will be much easier to
constrain the system call latency, and makes it much easier to report
fstrim progress to anyone who's running xfs_scrub.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com
2024-04-15 14:59:00 -07:00
Darrick J. Wong
1a5f6e08d4 xfs: create subordinate scrub contexts for xchk_metadata_inode_subtype
When a file-based metadata structure is being scrubbed in
xchk_metadata_inode_subtype, we should create an entirely new scrub
context so that each scrubber doesn't trip over another's buffers.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:00 -07:00
Darrick J. Wong
5220727ce8 docs: update online directory and parent pointer repair sections
Update the case studies of online directory and parent pointer
reconstruction to reflect what they actually do in the final version.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:00 -07:00
Darrick J. Wong
d85fe250f2 docs: update the parent pointers documentation to the final version
Now that we've decided on the ondisk format of parent pointers, update
the documentation to reflect that.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:59:00 -07:00
Darrick J. Wong
5f204051d9 xfs: pin inodes that would otherwise overflow link count
The VFS inc_nlink function does not explicitly check for integer
overflows in the i_nlink field.  Instead, it checks the link count
against s_max_links in the vfs_{link,create,rename} functions.  XFS
sets the maximum link count to 2.1 billion, so integer overflows should
not be a problem.

However.  It's possible that online repair could find that a file has
more than four billion links, particularly if the link count got
corrupted while creating hardlinks to the file.  The di_nlinkv2 field is
not large enough to store a value larger than 2^32, so we ought to
define a magic pin value of ~0U which means that the inode never gets
deleted.  This will prevent a UAF error if the repair finds this
situation and users begin deleting links to the file.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:58:59 -07:00
Darrick J. Wong
2935213a68 xfs: try to avoid allocating from sick inode clusters
I noticed that xfs/413 and xfs/375 occasionally failed while fuzzing
core.mode of an inode.  The root cause of these problems is that the
field we fuzzed (core.mode or core.magic, typically) causes the entire
inode cluster buffer verification to fail, which affects several inodes
at once.  The repair process tries to create either a /lost+found or a
temporary repair file, but regrettably it picks the same inode cluster
that we just corrupted, with the result that repair triggers the demise
of the filesystem.

Try avoid this by making the inode allocation path detect when the perag
health status indicates that someone has found bad inode cluster
buffers, and try to read the inode cluster buffer.  If the cluster
buffer fails the verifiers, try another AG.  This isn't foolproof and
can result in premature ENOSPC, but that might be better than shutting
down.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:58:59 -07:00
Darrick J. Wong
40cb8613d6 xfs: check unused nlink fields in the ondisk inode
v2/v3 inodes use di_nlink and not di_onlink; and v1 inodes use di_onlink
and not di_nlink.  Whichever field is not in use, make sure its contents
are zero, and teach xfs_scrub to fix that if it is.

This clears a bunch of missing scrub failure errors in xfs/385 for
core.onlink.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:58:59 -07:00
Darrick J. Wong
ab97f4b1c0 xfs: repair AGI unlinked inode bucket lists
Teach the AGI repair code to rebuild the unlinked buckets and lists.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:58:58 -07:00
Darrick J. Wong
2651923d8d xfs: online repair of symbolic links
If a symbolic link target looks bad, try to sift through the rubble to
find as much of the target buffer that we can, and stage a new target
(short or remote format as needed) in a temporary file and use the
atomic extent swapping mechanism to commit the results.  In the worst
case, we replace the target with an overly long filename that cannot
possibly resolve.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:58:58 -07:00
Darrick J. Wong
5b57257025 xfs: hoist AGI repair context to a heap object
Save ~460 bytes of stack space by moving all the repair context to a
heap object.  We're going to add even more context data in the next
patch, which is why we really need to do this now.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2024-04-15 14:58:58 -07:00