kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-18 19:12:56 +00:00
Code Issues Releases Packages Activity
No description
1,074,815 commits 116 branches 5,798 tags 5.6 GiB
  • C 96.9%
  • Assembly 0.9%
  • Rust 0.6%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Sasha Levin 7495280866 fs: Prevent file descriptor table allocations exceeding INT_MAX 
commit 04a2c4b451 upstream.

When sysctl_nr_open is set to a very high value (for example, 1073741816
as set by systemd), processes attempting to use file descriptors near
the limit can trigger massive memory allocation attempts that exceed
INT_MAX, resulting in a WARNING in mm/slub.c:

  WARNING: CPU: 0 PID: 44 at mm/slub.c:5027 __kvmalloc_node_noprof+0x21a/0x288

This happens because kvmalloc_array() and kvmalloc() check if the
requested size exceeds INT_MAX and emit a warning when the allocation is
not flagged with __GFP_NOWARN.

Specifically, when nr_open is set to 1073741816 (0x3ffffff8) and a
process calls dup2(oldfd, 1073741880), the kernel attempts to allocate:
- File descriptor array: 1073741880 * 8 bytes = 8,589,935,040 bytes
- Multiple bitmaps: ~400MB
- Total allocation size: > 8GB (exceeding INT_MAX = 2,147,483,647)

Reproducer:
1. Set /proc/sys/fs/nr_open to 1073741816:
   # echo 1073741816 > /proc/sys/fs/nr_open

2. Run a program that uses a high file descriptor:
   #include <unistd.h>
   #include <sys/resource.h>

   int main() {
       struct rlimit rlim = {1073741824, 1073741824};
       setrlimit(RLIMIT_NOFILE, &rlim);
       dup2(2, 1073741880);  // Triggers the warning
       return 0;
   }

3. Observe WARNING in dmesg at mm/slub.c:5027

systemd commit a8b627a introduced automatic bumping of fs.nr_open to the
maximum possible value. The rationale was that systems with memory
control groups (memcg) no longer need separate file descriptor limits
since memory is properly accounted. However, this change overlooked
that:

1. The kernel's allocation functions still enforce INT_MAX as a maximum
   size regardless of memcg accounting
2. Programs and tests that legitimately test file descriptor limits can
   inadvertently trigger massive allocations
3. The resulting allocations (>8GB) are impractical and will always fail

systemd's algorithm starts with INT_MAX and keeps halving the value
until the kernel accepts it. On most systems, this results in nr_open
being set to 1073741816 (0x3ffffff8), which is just under 1GB of file
descriptors.

While processes rarely use file descriptors near this limit in normal
operation, certain selftests (like
tools/testing/selftests/core/unshare_test.c) and programs that test file
descriptor limits can trigger this issue.

Fix this by adding a check in alloc_fdtable() to ensure the requested
allocation size does not exceed INT_MAX. This causes the operation to
fail with -EMFILE instead of triggering a kernel warning and avoids the
impractical >8GB memory allocation request.

Fixes: 9cfe015aa4 ("get rid of NR_OPEN and introduce a sysctl_nr_open")
Cc: stable@vger.kernel.org
Signed-off-by: Sasha Levin <sashal@kernel.org>
Link: https://lore.kernel.org/20250629074021.1038845-1-sashal@kernel.org
Signed-off-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
History 2025-08-28 16:24:15 +02:00
arch MIPS: mm: tlb-r4k: Uniquify TLB entries on init 2025-08-28 16:24:14 +02:00
block block: default BLOCK_LEGACY_AUTOLOAD to y 2025-06-27 11:05:34 +01:00
certs certs/blacklist_hashes.c: fix const confusion in certs blacklist 2022-06-22 14:22:01 +02:00
crypto crypto: xts - Only add ecb if it is not already there 2025-06-27 11:05:10 +01:00
Documentation f2fs: doc: fix wrong quota mount option description 2025-08-28 16:24:12 +02:00
drivers sunvdc: Balance device refcount in vdc_port_mpgroup_check 2025-08-28 16:24:15 +02:00
fs
include io_uring: don't use int for ABI 2025-08-28 16:24:15 +02:00
init sched/isolation: Make CONFIG_CPU_ISOLATION depend on CONFIG_SMP 2025-05-02 07:44:36 +02:00
io_uring io_uring: fix possible deadlock in io_register_iowq_max_workers() 2024-11-17 15:06:25 +01:00
ipc ipc: fix to protect IPCS lookups using RCU 2025-06-27 11:05:26 +01:00
kernel perf/core: Prevent VMA split of buffer mappings 2025-08-28 16:24:14 +02:00
lib lib: bitmap: Introduce node-aware alloc API 2025-08-28 16:24:02 +02:00
LICENSES LICENSES/dual/CC-BY-4.0: Git rid of "smart quotes" 2021-07-15 06:31:24 -06:00
mm mm/hmm: move pmd_to_hmm_pfn_flags() to the respective #ifdeffery 2025-08-28 16:24:14 +02:00
net netlink: avoid infinite retry looping in netlink_unicast() 2025-08-28 16:24:15 +02:00
samples samples: mei: Fix building on musl libc 2025-08-28 16:24:07 +02:00
scripts kconfig: qconf: fix ConfigList::updateListAllforAll() 2025-08-28 16:24:12 +02:00
security apparmor: ensure WB_HISTORY_SIZE value is a power of 2 2025-08-28 16:24:11 +02:00
sound ALSA: usb-audio: Validate UAC3 cluster segment descriptors 2025-08-28 16:24:15 +02:00
tools selftests/perf_events: Add a mmap() correctness test 2025-08-28 16:24:14 +02:00
usr kbuild: hdrcheck: fix cross build with clang 2025-06-27 11:05:22 +01:00
virt KVM: Fix a data race on last_boosted_vcpu in kvm_vcpu_on_spin() 2024-10-22 15:40:41 +02:00
.clang-format clang-format: Update with the latest for_each macro list 2021-05-12 23:32:39 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore Remove *.orig pattern from .gitignore 2024-10-17 15:11:10 +02:00
.mailmap mailmap: add Andrej Shadura 2021-10-18 20:22:03 -10:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Move Daniel Drake to credits 2021-09-21 08:34:58 +03:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS trace: Relocate event helper files 2024-04-10 16:19:24 +02:00
Makefile Linux 5.15.189 2025-07-17 18:30:55 +02:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.