kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-11-10 20:48:12 +00:00
Code Issues Releases Packages Activity
No description
1,189,160 commits 117 branches 5,827 tags 6.3 GiB
  • C 96.8%
  • Assembly 0.9%
  • Rust 0.7%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Mike Kravetz 9a1a43a0e7 hugetlb: do not clear hugetlb dtor until allocating vmemmap 
commit 32c877191e upstream.

Patch series "Fix hugetlb free path race with memory errors".

In the discussion of Jiaqi Yan's series "Improve hugetlbfs read on
HWPOISON hugepages" the race window was discovered.
https://lore.kernel.org/linux-mm/20230616233447.GB7371@monkey/

Freeing a hugetlb page back to low level memory allocators is performed
in two steps.
1) Under hugetlb lock, remove page from hugetlb lists and clear destructor
2) Outside lock, allocate vmemmap if necessary and call low level free
Between these two steps, the hugetlb page will appear as a normal
compound page.  However, vmemmap for tail pages could be missing.
If a memory error occurs at this time, we could try to update page
flags non-existant page structs.

A much more detailed description is in the first patch.

The first patch addresses the race window.  However, it adds a
hugetlb_lock lock/unlock cycle to every vmemmap optimized hugetlb page
free operation.  This could lead to slowdowns if one is freeing a large
number of hugetlb pages.

The second path optimizes the update_and_free_pages_bulk routine to only
take the lock once in bulk operations.

The second patch is technically not a bug fix, but includes a Fixes tag
and Cc stable to avoid a performance regression.  It can be combined with
the first, but was done separately make reviewing easier.


This patch (of 2):

Freeing a hugetlb page and releasing base pages back to the underlying
allocator such as buddy or cma is performed in two steps:
- remove_hugetlb_folio() is called to remove the folio from hugetlb
  lists, get a ref on the page and remove hugetlb destructor.  This
  all must be done under the hugetlb lock.  After this call, the page
  can be treated as a normal compound page or a collection of base
  size pages.
- update_and_free_hugetlb_folio() is called to allocate vmemmap if
  needed and the free routine of the underlying allocator is called
  on the resulting page.  We can not hold the hugetlb lock here.

One issue with this scheme is that a memory error could occur between
these two steps.  In this case, the memory error handling code treats
the old hugetlb page as a normal compound page or collection of base
pages.  It will then try to SetPageHWPoison(page) on the page with an
error.  If the page with error is a tail page without vmemmap, a write
error will occur when trying to set the flag.

Address this issue by modifying remove_hugetlb_folio() and
update_and_free_hugetlb_folio() such that the hugetlb destructor is not
cleared until after allocating vmemmap.  Since clearing the destructor
requires holding the hugetlb lock, the clearing is done in
remove_hugetlb_folio() if the vmemmap is present.  This saves a
lock/unlock cycle.  Otherwise, destructor is cleared in
update_and_free_hugetlb_folio() after allocating vmemmap.

Note that this will leave hugetlb pages in a state where they are marked
free (by hugetlb specific page flag) and have a ref count.  This is not
a normal state.  The only code that would notice is the memory error
code, and it is set up to retry in such a case.

A subsequent patch will create a routine to do bulk processing of
vmemmap allocation.  This will eliminate a lock/unlock cycle for each
hugetlb page in the case where we are freeing a large number of pages.

Link: https://lkml.kernel.org/r/20230711220942.43706-1-mike.kravetz@oracle.com
Link: https://lkml.kernel.org/r/20230711220942.43706-2-mike.kravetz@oracle.com
Fixes: ad2fa3717b ("mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page")
Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
Reviewed-by: Muchun Song <songmuchun@bytedance.com>
Tested-by: Naoya Horiguchi <naoya.horiguchi@nec.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: James Houghton <jthoughton@google.com>
Cc: Jiaqi Yan <jiaqiyan@google.com>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
History 2023-08-16 18:32:20 +02:00
arch riscv: mm: fix 2 instances of -Wmissing-variable-declarations 2023-08-16 18:32:18 +02:00
block blk-mq: Fix stall due to recursive flush plug 2023-08-03 10:25:41 +02:00
certs KEYS: Add missing function documentation 2023-04-24 16:15:52 +03:00
crypto crypto: jitter - correct health test during initialization 2023-07-19 16:36:19 +02:00
Documentation Documentation: kdump: Add va_kernel_pa_offset for RISCV64 2023-08-11 12:14:19 +02:00
drivers accel/ivpu: Add set_pages_array_wc/uc for internal buffers 2023-08-16 18:32:20 +02:00
fs nilfs2: fix use-after-free of nilfs_root in dirtying inodes via iput 2023-08-16 18:32:20 +02:00
include tpm: Disable RNG for all AMD fTPMs 2023-08-16 18:32:16 +02:00
init init, x86: Move mem_encrypt_init() into arch_cpu_finalize_init() 2023-08-08 20:04:49 +02:00
io_uring io_uring: correct check for O_TMPFILE 2023-08-16 18:32:19 +02:00
ipc Merge branch 'work.namespace' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2023-02-24 19:20:07 -08:00
kernel bpf: Disable preemption in bpf_event_output 2023-08-11 12:14:22 +02:00
lib debugobjects: Recheck debug_objects_enabled before reporting 2023-08-11 12:14:25 +02:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm hugetlb: do not clear hugetlb dtor until allocating vmemmap 2023-08-16 18:32:20 +02:00
net mptcp: fix disconnect vs accept race 2023-08-16 18:32:17 +02:00
rust rust: allocator: Prevent mis-aligned allocation 2023-08-11 12:14:18 +02:00
samples samples: ftrace: Save required argument registers in sample trampolines 2023-07-23 13:54:09 +02:00
scripts kbuild: rust: avoid creating temporary files 2023-07-27 08:57:06 +02:00
security security: keys: Modify mismatched function name 2023-07-27 08:56:59 +02:00
sound ASoC: wm8904: Fill the cache for WM8904_ADC_TEST_0 register 2023-08-03 10:26:12 +02:00
tools radix tree test suite: fix incorrect allocation size for pthreads 2023-08-16 18:32:19 +02:00
usr initramfs: Check negative timestamp to prevent broken cpio archive 2023-04-16 17:37:01 +09:00
virt KVM: Grab a reference to KVM for VM and vCPU stats file descriptors 2023-08-03 10:26:01 +02:00
.clang-format cxl for v6.4 2023-04-30 11:51:51 -07:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for *.dtso files 2023-02-26 15:28:23 +09:00
.gitignore linux-kselftest-kunit-6.4-rc1 2023-04-24 12:31:32 -07:00
.mailmap mailmap: add entries for Ben Dooks 2023-06-19 13:19:35 -07:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: sctp: move Neil to CREDITS 2023-05-12 08:51:32 +01:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS Networking fixes for 6.4-rc8, including fixes from ipsec, bpf, 2023-06-22 17:59:51 -07:00
Makefile Linux 6.4.10 2023-08-11 12:14:29 +02:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.