kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-14 05:55:17 +00:00
Code Issues Releases Packages Activity

xfs: CPU usage optimizations for realtime allocator [v2.3]

Browse source 
This is version 2 of [Omar's] XFS realtime allocator opimization patch
 series.
 
 Changes since v1 [1]:
 
 - Fixed potential overflow in patch 4.
 - Changed deprecated typedefs to normal struct names
 - Fixed broken indentation
 - Used xfs_fileoff_t instead of xfs_fsblock_t where appropriate.
 - Added calls to xfs_rtbuf_cache_relse anywhere that the cache is used
   instead of relying on the buffers being dirtied and thus attached to
   the transaction.
 - Clarified comments and commit messages in a few places.
 - Added Darrick's Reviewed-bys.
 
 Cover letter from v1:
 
 Our distributed storage system uses XFS's realtime device support as a
 way to split an XFS filesystem between an SSD and an HDD -- we configure
 the HDD as the realtime device so that metadata goes on the SSD and data
 goes on the HDD.
 
 We've been running this in production for a few years now, so we have
 some fairly fragmented filesystems. This has exposed various CPU
 inefficiencies in the realtime allocator. These became even worse when
 we experimented with using XFS_XFLAG_EXTSIZE to force files to be
 allocated contiguously.
 
 This series adds several optimizations that don't change the realtime
 allocator's decisions, but make them happen more efficiently, mainly by
 avoiding redundant work. We've tested these in production and measured
 ~10%% lower CPU utilization. Furthermore, it made it possible to use
 XFS_XFLAG_EXTSIZE to force contiguous allocations -- without these
 patches, our most fragmented systems would become unresponsive due to
 high CPU usage in the realtime allocator, but with them, CPU utilization
 is actually ~4-6%% lower than before, and disk I/O utilization is 15-20%%
 lower.
 
 Patches 2 and 3 are preparations for later optimizations; the remaining
 patches are the optimizations themselves.
 
 1: https://lore.kernel.org/linux-xfs/cover.1687296675.git.osandov@osandov.com/
 
 v2.1: djwong rebased everything atop his own cleanups, added dave's rtalloc_args
 v2.2: rebase with new apis and clean them up too
 v2.3: move struct definition around for lolz
 
 With a bit of luck, this should all go splendidly.
 
 Signed-off-by: Darrick J. Wong <djwong@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQQ2qTKExjcn+O1o2YRKO3ySh0YRpgUCZTFTEQAKCRBKO3ySh0YR
 pkbpAQD52XiQ7GXAiZSfJDStw1cr3C664AhmX/KdGtzHPmqxIQD/e8I0u1GEPezK
 6qnipiiFBq/px/hqakx5t8VgHfucwwY=
 =XdBg
 -----END PGP SIGNATURE-----

Merge tag 'rtalloc-speedups-6.7_2023-10-19' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-6.7-mergeA

xfs: CPU usage optimizations for realtime allocator [v2.3]

This is version 2 of [Omar's] XFS realtime allocator opimization patch
series.

Changes since v1 [1]:

- Fixed potential overflow in patch 4.
- Changed deprecated typedefs to normal struct names
- Fixed broken indentation
- Used xfs_fileoff_t instead of xfs_fsblock_t where appropriate.
- Added calls to xfs_rtbuf_cache_relse anywhere that the cache is used
  instead of relying on the buffers being dirtied and thus attached to
  the transaction.
- Clarified comments and commit messages in a few places.
- Added Darrick's Reviewed-bys.

Cover letter from v1:

Our distributed storage system uses XFS's realtime device support as a
way to split an XFS filesystem between an SSD and an HDD -- we configure
the HDD as the realtime device so that metadata goes on the SSD and data
goes on the HDD.

We've been running this in production for a few years now, so we have
some fairly fragmented filesystems. This has exposed various CPU
inefficiencies in the realtime allocator. These became even worse when
we experimented with using XFS_XFLAG_EXTSIZE to force files to be
allocated contiguously.

This series adds several optimizations that don't change the realtime
allocator's decisions, but make them happen more efficiently, mainly by
avoiding redundant work. We've tested these in production and measured
~10%% lower CPU utilization. Furthermore, it made it possible to use
XFS_XFLAG_EXTSIZE to force contiguous allocations -- without these
patches, our most fragmented systems would become unresponsive due to
high CPU usage in the realtime allocator, but with them, CPU utilization
is actually ~4-6%% lower than before, and disk I/O utilization is 15-20%%
lower.

Patches 2 and 3 are preparations for later optimizations; the remaining
patches are the optimizations themselves.

1: https://lore.kernel.org/linux-xfs/cover.1687296675.git.osandov@osandov.com/

v2.1: djwong rebased everything atop his own cleanups, added dave's rtalloc_args
v2.2: rebase with new apis and clean them up too
v2.3: move struct definition around for lolz

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>

* tag 'rtalloc-speedups-6.7_2023-10-19' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: don't look for end of extent further than necessary in xfs_rtallocate_extent_near()
  xfs: don't try redundant allocations in xfs_rtallocate_extent_near()
  xfs: limit maxlen based on available space in xfs_rtallocate_extent_near()
  xfs: return maximum free size from xfs_rtany_summary()
  xfs: invert the realtime summary cache
  xfs: simplify rt bitmap/summary block accessor functions
  xfs: simplify xfs_rtbuf_get calling conventions
  xfs: cache last bitmap block in realtime allocator
  xfs: consolidate realtime allocation arguments
This commit is contained in:
Chandan Babu R 2023-10-23 10:59:22 +05:30
commit 9fa8753aa1
5 changed files with 556 additions and 560 deletions
Download patch file Download diff file Expand all files Collapse all files