No description
  • C 96.9%
  • Assembly 0.9%
  • Rust 0.6%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Chandan Babu R 8394a97c4b xfs: support in-memory btrees [v29.3 12/18]
Online repair of the reverse-mapping btrees presens some unique
 challenges.  To construct a new reverse mapping btree, we must scan the
 entire filesystem, but we cannot afford to quiesce the entire filesystem
 for the potentially lengthy scan.
 
 For rmap btrees, therefore, we relax our requirements of totally atomic
 repairs.  Instead, repairs will scan all inodes, construct a new reverse
 mapping dataset, format a new btree, and commit it before anyone trips
 over the corruption.  This is exactly the same strategy as was used in
 the quotacheck and nlink scanners.
 
 Unfortunately, the xfarray cannot perform key-based lookups and is
 therefore unsuitable for supporting live updates.  Luckily, we already a
 data structure that maintains an indexed rmap recordset -- the existing
 rmap btree code!  Hence we port the existing btree and buffer target
 code to be able to create a btree using the xfile we developed earlier.
 Live hooks keep the in-memory btree up to date for any resources that
 have already been scanned.
 
 This approach is not maximally memory efficient, but we can use the same
 rmap code that we do everywhere else, which provides improved stability
 without growing the code base even more.  Note that in-memory btree
 blocks are always page sized.
 
 This patchset modifies the kernel xfs buffer cache to be capable of
 using a xfile (aka a shmem file) as a backing device.  It then augments
 the btree code to support creating btree cursors with buffers that come
 from a buftarg other than the data device (namely an xfile-backed
 buftarg).  For the userspace xfs buffer cache, we instead use a memfd or
 an O_TMPFILE file as a backing device.
 
 This has been running on the djcloud for months with no problems.  Enjoy!
 
 Signed-off-by: Darrick J. Wong <djwong@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQQ2qTKExjcn+O1o2YRKO3ySh0YRpgUCZdlBYgAKCRBKO3ySh0YR
 pqHwAQCjqXCvbqrWgMC/oYWuuDkYul79mqZw7PF58tdWpsOnzwD/dkyt7FBpAaAA
 GCaCIpWwOPErZ0LMT2fH6CqZyndSxw8=
 =vdwY
 -----END PGP SIGNATURE-----

Merge tag 'in-memory-btrees-6.9_2024-02-23' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-6.9-mergeC

xfs: support in-memory btrees

Online repair of the reverse-mapping btrees presens some unique
challenges.  To construct a new reverse mapping btree, we must scan the
entire filesystem, but we cannot afford to quiesce the entire filesystem
for the potentially lengthy scan.

For rmap btrees, therefore, we relax our requirements of totally atomic
repairs.  Instead, repairs will scan all inodes, construct a new reverse
mapping dataset, format a new btree, and commit it before anyone trips
over the corruption.  This is exactly the same strategy as was used in
the quotacheck and nlink scanners.

Unfortunately, the xfarray cannot perform key-based lookups and is
therefore unsuitable for supporting live updates.  Luckily, we already a
data structure that maintains an indexed rmap recordset -- the existing
rmap btree code!  Hence we port the existing btree and buffer target
code to be able to create a btree using the xfile we developed earlier.
Live hooks keep the in-memory btree up to date for any resources that
have already been scanned.

This approach is not maximally memory efficient, but we can use the same
rmap code that we do everywhere else, which provides improved stability
without growing the code base even more.  Note that in-memory btree
blocks are always page sized.

This patchset modifies the kernel xfs buffer cache to be capable of
using a xfile (aka a shmem file) as a backing device.  It then augments
the btree code to support creating btree cursors with buffers that come
from a buftarg other than the data device (namely an xfile-backed
buftarg).  For the userspace xfs buffer cache, we instead use a memfd or
an O_TMPFILE file as a backing device.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>

* tag 'in-memory-btrees-6.9_2024-02-23' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: launder in-memory btree buffers before transaction commit
  xfs: support in-memory btrees
  xfs: add a xfs_btree_ptrs_equal helper
  xfs: support in-memory buffer cache targets
  xfs: teach buftargs to maintain their own buffer hashtable
History 2024-02-24 10:18:39 +05:30
arch - Correct the minimum CPU family for Transmeta Crusoe in Kconfig so that 2024-02-11 11:41:51 -08:00
block blk-iocost: Fix an UBSAN shift-out-of-bounds warning 2024-02-08 10:11:39 -07:00
certs This update includes the following changes: 2023-11-02 16:15:30 -10:00
crypto crypto: algif_hash - Remove bogus SGL free on zero-length error path 2024-02-02 18:08:12 +08:00
Documentation xfs: support in-memory btrees 2024-02-22 12:43:35 -08:00
drivers block-6.8-2024-02-10 2024-02-10 08:02:48 -08:00
fs xfs: launder in-memory btree buffers before transaction commit 2024-02-22 12:43:36 -08:00
include shmem: move shmem_mapping out of line 2024-02-21 11:36:50 +05:30
init Kconfig: Disable -Wstringop-overflow for GCC globally 2024-02-01 14:57:17 -08:00
io_uring io_uring/net: fix sr->len for IORING_OP_RECV with MSG_WAITALL and buffers 2024-02-01 06:42:36 -07:00
ipc shm: Slim down dependencies 2023-12-20 19:26:31 -05:00
kernel - Make sure a warning is issued when a hrtimer gets queued after the 2024-02-11 11:44:14 -08:00
lib linux_kselftest-kunit-fixes-6.8-rc3 2024-01-30 15:12:58 -08:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm shmem: document how to "persist" data when using shmem_*file_setup 2024-02-21 11:36:51 +05:30
net Some fscrypt-related fixups (sparse reads are used only for encrypted 2024-02-09 17:05:02 -08:00
rust Rust changes for v6.8 2024-01-11 13:05:41 -08:00
samples work around gcc bugs with 'asm goto' with outputs 2024-02-09 15:57:48 -08:00
scripts Kbuild fixes for v6.8 2024-02-01 11:57:42 -08:00
security lsm/stable-6.8 PR 20240131 2024-02-01 10:00:28 -08:00
sound ASoC: Fixes for v6.8 2024-02-01 19:40:42 +01:00
tools 21 hotfixes. 12 are cc:stable and the remainder pertain to post-6.7 2024-02-10 15:28:07 -08:00
usr Kbuild updates for v6.8 2024-01-18 17:57:07 -08:00
virt Generic: 2024-01-17 13:03:37 -08:00
.clang-format clang-format: Update with v6.7-rc4's for_each macro list 2023-12-08 23:54:38 +01:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.editorconfig Add .editorconfig file for basic formatting 2023-12-28 16:22:47 +09:00
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore Add .editorconfig file for basic formatting 2023-12-28 16:22:47 +09:00
.mailmap MAINTAINERS: Leo Yan has moved 2024-02-07 21:20:36 -08:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: supplement of zswap maintainers update 2024-01-25 23:52:21 -08:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS 21 hotfixes. 12 are cc:stable and the remainder pertain to post-6.7 2024-02-10 15:28:07 -08:00
Makefile Linux 6.8-rc4 2024-02-11 12:18:13 -08:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.