linux

mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-11-10 20:48:12 +00:00

No description

C 96.8%
Assembly 0.9%
Rust 0.7%
Shell 0.6%
Python 0.5%
Other 0.3%

Find a file

Toke Høiland-Jørgensen 4f51fb0d25 page_pool: Track DMA-mapped pages and unmap them when destroying the pool [ Upstream commit `ee62ce7a1d` ] When enabling DMA mapping in page_pool, pages are kept DMA mapped until they are released from the pool, to avoid the overhead of re-mapping the pages every time they are used. This causes resource leaks and/or crashes when there are pages still outstanding while the device is torn down, because page_pool will attempt an unmap through a non-existent DMA device on the subsequent page return. To fix this, implement a simple tracking of outstanding DMA-mapped pages in page pool using an xarray. This was first suggested by Mina[0], and turns out to be fairly straight forward: We simply store pointers to pages directly in the xarray with xa_alloc() when they are first DMA mapped, and remove them from the array on unmap. Then, when a page pool is torn down, it can simply walk the xarray and unmap all pages still present there before returning, which also allows us to get rid of the get/put_device() calls in page_pool. Using xa_cmpxchg(), no additional synchronisation is needed, as a page will only ever be unmapped once. To avoid having to walk the entire xarray on unmap to find the page reference, we stash the ID assigned by xa_alloc() into the page structure itself, using the upper bits of the pp_magic field. This requires a couple of defines to avoid conflicting with the POINTER_POISON_DELTA define, but this is all evaluated at compile-time, so does not affect run-time performance. The bitmap calculations in this patch gives the following number of bits for different architectures: - 23 bits on 32-bit architectures - 21 bits on PPC64 (because of the definition of ILLEGAL_POINTER_VALUE) - 32 bits on other 64-bit architectures Stashing a value into the unused bits of pp_magic does have the effect that it can make the value stored there lie outside the unmappable range (as governed by the mmap_min_addr sysctl), for architectures that don't define ILLEGAL_POINTER_VALUE. This means that if one of the pointers that is aliased to the pp_magic field (such as page->lru.next) is dereferenced while the page is owned by page_pool, that could lead to a dereference into userspace, which is a security concern. The risk of this is mitigated by the fact that (a) we always clear pp_magic before releasing a page from page_pool, and (b) this would need a use-after-free bug for struct page, which can have many other risks since page->lru.next is used as a generic list pointer in multiple places in the kernel. As such, with this patch we take the position that this risk is negligible in practice. For more discussion, see[1]. Since all the tracking added in this patch is performed on DMA map/unmap, no additional code is needed in the fast path, meaning the performance overhead of this tracking is negligible there. A micro-benchmark shows that the total overhead of the tracking itself is about 400 ns (39 cycles(tsc) 395.218 ns; sum for both map and unmap[2]). Since this cost is only paid on DMA map and unmap, it seems like an acceptable cost to fix the late unmap issue. Further optimisation can narrow the cases where this cost is paid (for instance by eliding the tracking when DMA map/unmap is a no-op). The extra memory needed to track the pages is neatly encapsulated inside xarray, which uses the 'struct xa_node' structure to track items. This structure is 576 bytes long, with slots for 64 items, meaning that a full node occurs only 9 bytes of overhead per slot it tracks (in practice, it probably won't be this efficient, but in any case it should be an acceptable overhead). [0] https://lore.kernel.org/all/CAHS8izPg7B5DwKfSuzz-iOop_YRbk3Sd6Y4rX7KBG9DcVJcyWg@mail.gmail.com/ [1] https://lore.kernel.org/r/20250320023202.GA25514@openwall.com [2] https://lore.kernel.org/r/ae07144c-9295-4c9d-a400-153bb689fe9e@huawei.com Reported-by: Yonglong Liu <liuyonglong@huawei.com> Closes: https://lore.kernel.org/r/8743264a-9700-4227-a556-5f931c720211@huawei.com Fixes: `ff7d6b27f8` ("page_pool: refurbish version of page_pool code") Suggested-by: Mina Almasry <almasrymina@google.com> Reviewed-by: Mina Almasry <almasrymina@google.com> Reviewed-by: Jesper Dangaard Brouer <hawk@kernel.org> Tested-by: Jesper Dangaard Brouer <hawk@kernel.org> Tested-by: Qiuling Ren <qren@redhat.com> Tested-by: Yuying Ma <yuma@redhat.com> Tested-by: Yonglong Liu <liuyonglong@huawei.com> Acked-by: Jesper Dangaard Brouer <hawk@kernel.org> Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com> Link: https://patch.msgid.link/20250409-page-pool-track-dma-v9-2-6a9ef2e0cba8@redhat.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org>		History 2025-06-19 15:31:42 +02:00
arch	arm64/fpsimd: Do not discard modified SVE state	2025-06-19 15:31:38 +02:00
block	block: fix adding folio to bio	2025-06-10 07:12:59 -04:00
certs	sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3	2024-09-20 19:52:48 +03:00
crypto	crypto: api - Redo lookup on EEXIST	2025-06-19 15:31:30 +02:00
Documentation	dt-bindings: phy: imx8mq-usb: fix fsl,phy-tx-vboost-level-microvolt property	2025-06-10 07:13:00 -04:00
drivers	page_pool: Move pp_magic check into helper functions	2025-06-19 15:31:42 +02:00
fs	f2fs: fix to do sanity check on sbi->total_valid_block_count	2025-06-19 15:31:42 +02:00
include	page_pool: Track DMA-mapped pages and unmap them when destroying the pool	2025-06-19 15:31:42 +02:00
init	rust: clean Rust 1.88.0's `unnecessary_transmutes` lint	2025-05-18 08:24:51 +02:00
io_uring	io_uring: fix overflow resched cqe reordering	2025-05-29 11:03:17 +02:00
ipc	ipc: fix memleak if msg_init_ns failed in create_ipc_ns	2024-12-05 14:03:02 +01:00
kernel	bpf: Check link_create.flags parameter for multi_kprobe	2025-06-19 15:31:40 +02:00
lib	kunit/usercopy: Disable u64 test on 32-bit SPARC	2025-06-19 15:31:36 +02:00
LICENSES	LICENSES: add 0BSD license text	2024-09-01 20:43:24 -07:00
mm	page_pool: Move pp_magic check into helper functions	2025-06-19 15:31:42 +02:00
net	page_pool: Track DMA-mapped pages and unmap them when destroying the pool	2025-06-19 15:31:42 +02:00
rust	rust: clean Rust 1.88.0's `clippy::uninlined_format_args` lint	2025-05-18 08:24:57 +02:00
samples	samples/bpf: Fix compilation failure for samples/bpf on LoongArch Fedora	2025-05-29 11:02:01 +02:00
scripts	kbuild: Properly disable -Wunterminated-string-initialization for clang	2025-05-29 11:03:25 +02:00
security	smack: Revert "smackfs: Added check catlen"	2025-05-29 11:02:48 +02:00
sound	ALSA: core: fix up bus match const issues.	2025-06-19 15:31:33 +02:00
tools	selftests/bpf: Fix bpf_nf selftest failure	2025-06-19 15:31:40 +02:00
usr	kbuild: hdrcheck: fix cross build with clang	2025-03-13 13:02:18 +01:00
virt	KVM: Add member to struct kvm_gfn_range to indicate private/shared	2025-05-22 14:29:36 +02:00
.clang-format	clang-format: Update with v6.11-rc1's `for_each` macro list	2024-08-02 13:20:31 +02:00
.clippy.toml	rust: clean Rust 1.88.0's warning about `clippy::disallowed_macros` configuration	2025-05-18 08:24:51 +02:00
.cocciconfig	scripts: add Linux .cocciconfig for coccinelle	2016-07-22 12:13:39 +02:00
.editorconfig	.editorconfig: remove trim_trailing_whitespace option	2024-06-13 16:47:52 +02:00
.get_maintainer.ignore	Add Jeff Kirsher to .get_maintainer.ignore	2024-03-08 11:36:54 +00:00
.gitattributes	.gitattributes: set diff driver for Rust source code files	2023-05-31 17:48:25 +02:00
.gitignore	rust: introduce `.clippy.toml`	2025-03-13 13:01:42 +01:00
.mailmap	mailmap: add entry for Thorsten Blum	2024-11-07 14:14:59 -08:00
.rustfmt.toml	rust: add `.rustfmt.toml`	2022-09-28 09:02:20 +02:00
COPYING	COPYING: state that all contributions really are covered by this file	2020-02-10 13:32:20 -08:00
CREDITS	MAINTAINERS: Remove self from DSA entry	2024-11-03 12:52:38 -08:00
Kbuild	Kbuild updates for v6.1	2022-10-10 12:00:45 -07:00
Kconfig	kbuild: ensure full rebuild when the compiler is updated	2020-05-12 13:28:33 +09:00
MAINTAINERS	MAINTAINERS: Update Alexey Makhalov's email address	2025-05-22 14:29:46 +02:00
Makefile	Linux 6.12.33	2025-06-10 07:13:00 -04:00
README	README: Fix spelling	2024-03-18 03:36:32 -06:00

README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the reStructuredText markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.