kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-11-13 07:31:07 +00:00
Code Issues Releases Packages Activity
No description
1,320,559 commits 117 branches 5,827 tags 6.3 GiB
  • C 96.8%
  • Assembly 0.9%
  • Rust 0.7%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Sasha Levin d4f9351243 fs: Prevent file descriptor table allocations exceeding INT_MAX 
commit 04a2c4b451 upstream.

When sysctl_nr_open is set to a very high value (for example, 1073741816
as set by systemd), processes attempting to use file descriptors near
the limit can trigger massive memory allocation attempts that exceed
INT_MAX, resulting in a WARNING in mm/slub.c:

  WARNING: CPU: 0 PID: 44 at mm/slub.c:5027 __kvmalloc_node_noprof+0x21a/0x288

This happens because kvmalloc_array() and kvmalloc() check if the
requested size exceeds INT_MAX and emit a warning when the allocation is
not flagged with __GFP_NOWARN.

Specifically, when nr_open is set to 1073741816 (0x3ffffff8) and a
process calls dup2(oldfd, 1073741880), the kernel attempts to allocate:
- File descriptor array: 1073741880 * 8 bytes = 8,589,935,040 bytes
- Multiple bitmaps: ~400MB
- Total allocation size: > 8GB (exceeding INT_MAX = 2,147,483,647)

Reproducer:
1. Set /proc/sys/fs/nr_open to 1073741816:
   # echo 1073741816 > /proc/sys/fs/nr_open

2. Run a program that uses a high file descriptor:
   #include <unistd.h>
   #include <sys/resource.h>

   int main() {
       struct rlimit rlim = {1073741824, 1073741824};
       setrlimit(RLIMIT_NOFILE, &rlim);
       dup2(2, 1073741880);  // Triggers the warning
       return 0;
   }

3. Observe WARNING in dmesg at mm/slub.c:5027

systemd commit a8b627a introduced automatic bumping of fs.nr_open to the
maximum possible value. The rationale was that systems with memory
control groups (memcg) no longer need separate file descriptor limits
since memory is properly accounted. However, this change overlooked
that:

1. The kernel's allocation functions still enforce INT_MAX as a maximum
   size regardless of memcg accounting
2. Programs and tests that legitimately test file descriptor limits can
   inadvertently trigger massive allocations
3. The resulting allocations (>8GB) are impractical and will always fail

systemd's algorithm starts with INT_MAX and keeps halving the value
until the kernel accepts it. On most systems, this results in nr_open
being set to 1073741816 (0x3ffffff8), which is just under 1GB of file
descriptors.

While processes rarely use file descriptors near this limit in normal
operation, certain selftests (like
tools/testing/selftests/core/unshare_test.c) and programs that test file
descriptor limits can trigger this issue.

Fix this by adding a check in alloc_fdtable() to ensure the requested
allocation size does not exceed INT_MAX. This causes the operation to
fail with -EMFILE instead of triggering a kernel warning and avoids the
impractical >8GB memory allocation request.

Fixes: 9cfe015aa4 ("get rid of NR_OPEN and introduce a sysctl_nr_open")
Cc: stable@vger.kernel.org
Signed-off-by: Sasha Levin <sashal@kernel.org>
Link: https://lore.kernel.org/20250629074021.1038845-1-sashal@kernel.org
Signed-off-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
History 2025-08-20 18:30:15 +02:00
arch LoongArch: Avoid in-place string operation on FDT content 2025-08-20 18:30:15 +02:00
block block: ensure discard_granularity is zero when discard is not supported 2025-08-15 12:14:05 +02:00
certs sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3 2024-09-20 19:52:48 +03:00
crypto crypto: ecdsa - Harden against integer overflows in DIV_ROUND_UP() 2025-07-17 18:37:24 +02:00
Documentation fscrypt: Don't use problematic non-inline crypto engines 2025-08-20 18:30:15 +02:00
drivers clk: samsung: gs101: fix alternate mout_hsi0_usb20_ref parent clock 2025-08-20 18:30:15 +02:00
fs fs: Prevent file descriptor table allocations exceeding INT_MAX 2025-08-20 18:30:15 +02:00
include PCI: Extend isolated function probing to LoongArch 2025-08-20 18:30:14 +02:00
init io_uring: fix breakage in EXPERT menu 2025-08-15 12:13:32 +02:00
io_uring io_uring/poll: fix POLLERR handling 2025-07-24 08:56:23 +02:00
ipc ipc: fix to protect IPCS lookups using RCU 2025-06-27 11:11:22 +01:00
kernel perf/core: Prevent VMA split of buffer mappings 2025-08-15 12:14:09 +02:00
lib lib/alloc_tag: do not acquire non-existent lock in alloc_tag_top_users() 2025-07-17 18:37:14 +02:00
LICENSES LICENSES: add 0BSD license text 2024-09-01 20:43:24 -07:00
mm mm: swap: fix potential buffer overflow in setup_clusters() 2025-08-15 12:14:14 +02:00
net netlink: avoid infinite retry looping in netlink_unicast() 2025-08-20 18:30:13 +02:00
rust rust: use #[used(compiler)] to fix build and modpost with Rust >= 1.89.0 2025-07-24 08:56:37 +02:00
samples samples: mei: Fix building on musl libc 2025-08-15 12:13:38 +02:00
scripts kconfig: qconf: fix ConfigList::updateListAllforAll() 2025-08-15 12:14:03 +02:00
security apparmor: Fix unaligned memory accesses in KUnit test 2025-08-15 12:13:59 +02:00
sound ALSA: hda/realtek: Add Framework Laptop 13 (AMD Ryzen AI 300) to quirks 2025-08-20 18:30:12 +02:00
tools selftests/bpf: Fix build error with llvm 19 2025-08-15 12:14:10 +02:00
usr kbuild: hdrcheck: fix cross build with clang 2025-03-13 13:02:18 +01:00
virt KVM: Allow CPU to reschedule while setting per-page memory attributes 2025-07-17 18:37:08 +02:00
.clang-format clang-format: Update with v6.11-rc1's for_each macro list 2024-08-02 13:20:31 +02:00
.clippy.toml rust: give Clippy the minimum supported Rust version 2025-08-01 09:48:44 +01:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.editorconfig .editorconfig: remove trim_trailing_whitespace option 2024-06-13 16:47:52 +02:00
.get_maintainer.ignore Add Jeff Kirsher to .get_maintainer.ignore 2024-03-08 11:36:54 +00:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore rust: introduce .clippy.toml 2025-03-13 13:01:42 +01:00
.mailmap mailmap: add entry for Thorsten Blum 2024-11-07 14:14:59 -08:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Remove self from DSA entry 2024-11-03 12:52:38 -08:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS MAINTAINERS: Update Alexey Makhalov's email address 2025-05-22 14:29:46 +02:00
Makefile Linux 6.12.42 2025-08-15 12:14:14 +02:00
README README: Fix spelling 2024-03-18 03:36:32 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the reStructuredText markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.