kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-12 06:01:58 +00:00
Code Issues Releases Packages Activity
No description
1,235,884 commits 116 branches 5,791 tags 5.6 GiB
  • C 96.9%
  • Assembly 0.9%
  • Rust 0.6%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Hou Tao 1c842bfde7 bpf: Use c->unit_size to select target cache during free 
[ Upstream commit 7ac5c53e00 ]

At present, bpf memory allocator uses check_obj_size() to ensure that
ksize() of allocated pointer is equal with the unit_size of used
bpf_mem_cache. Its purpose is to prevent bpf_mem_free() from selecting
a bpf_mem_cache which has different unit_size compared with the
bpf_mem_cache used for allocation. But as reported by lkp, the return
value of ksize() or kmalloc_size_roundup() may change due to slab merge
and it will lead to the warning report in check_obj_size().

The reported warning happened as follows:
(1) in bpf_mem_cache_adjust_size(), kmalloc_size_roundup(96) returns the
object_size of kmalloc-96 instead of kmalloc-cg-96. The object_size of
kmalloc-96 is 96, so size_index for 96 is not adjusted accordingly.
(2) the object_size of kmalloc-cg-96 is adjust from 96 to 128 due to
slab merge in __kmem_cache_alias(). For SLAB, SLAB_HWCACHE_ALIGN is
enabled by default for kmalloc slab, so align is 64 and size is 128 for
kmalloc-cg-96. SLUB has a similar merge logic, but its object_size will
not be changed, because its align is 8 under x86-64.
(3) when unit_alloc() does kmalloc_node(96, __GFP_ACCOUNT, node),
ksize() returns 128 instead of 96 for the returned pointer.
(4) the warning in check_obj_size() is triggered.

Considering the slab merge can happen in anytime (e.g, a slab created in
a new module), the following case is also possible: during the
initialization of bpf_global_ma, there is no slab merge and ksize() for
a 96-bytes object returns 96. But after that a new slab created by a
kernel module is merged to kmalloc-cg-96 and the object_size of
kmalloc-cg-96 is adjust from 96 to 128 (which is possible for x86-64 +
CONFIG_SLAB, because its alignment requirement is 64 for 96-bytes slab).
So soon or later, when bpf_global_ma frees a 96-byte-sized pointer
which is allocated from bpf_mem_cache with unit_size=96, bpf_mem_free()
will free the pointer through a bpf_mem_cache in which unit_size is 128,
because the return value of ksize() changes. The warning for the
mismatch will be triggered again.

A feasible fix is introducing similar APIs compared with ksize() and
kmalloc_size_roundup() to return the actually-allocated size instead of
size which may change due to slab merge, but it will introduce
unnecessary dependency on the implementation details of mm subsystem.

As for now the pointer of bpf_mem_cache is saved in the 8-bytes area
(or 4-bytes under 32-bit host) above the returned pointer, using
unit_size in the saved bpf_mem_cache to select the target cache instead
of inferring the size from the pointer itself. Beside no extra
dependency on mm subsystem, the performance for bpf_mem_free_rcu() is
also improved as shown below.

Before applying the patch, the performances of bpf_mem_alloc() and
bpf_mem_free_rcu() on 8-CPUs VM with one producer are as follows:

kmalloc : alloc 11.69 ± 0.28M/s free 29.58 ± 0.93M/s
percpu  : alloc 14.11 ± 0.52M/s free 14.29 ± 0.99M/s

After apply the patch, the performance for bpf_mem_free_rcu() increases
9% and 146% for kmalloc memory and per-cpu memory respectively:

kmalloc: alloc 11.01 ± 0.03M/s free   32.42 ± 0.48M/s
percpu:  alloc 12.84 ± 0.12M/s free   35.24 ± 0.23M/s

After the fixes, there is no need to adjust size_index to fix the
mismatch between allocation and free, so remove it as well. Also return
NULL instead of ZERO_SIZE_PTR for zero-sized alloc in bpf_mem_alloc(),
because there is no bpf_mem_cache pointer saved above ZERO_SIZE_PTR.

Fixes: 9077fc228f ("bpf: Use kmalloc_size_roundup() to adjust size_index")
Reported-by: kernel test robot <oliver.sang@intel.com>
Closes: https://lore.kernel.org/bpf/202310302113.9f8fe705-oliver.sang@intel.com
Signed-off-by: Hou Tao <houtao1@huawei.com>
Link: https://lore.kernel.org/r/20231216131052.27621-2-houtao@huaweicloud.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
History 2024-01-25 15:44:54 -08:00
arch arm64: dts: qcom: sc8180x: Fix up PCIe nodes 2024-01-25 15:44:54 -08:00
block block: add check of 'minors' and 'first_minor' in device_add_disk() 2024-01-25 15:44:53 -08:00
certs This update includes the following changes: 2023-11-02 16:15:30 -10:00
crypto crypto: scomp - fix req->dst buffer overflow 2024-01-25 15:44:43 -08:00
Documentation dt-bindings: media: mediatek: mdp3: correct RDMA and WROT node with generic names 2024-01-25 15:44:50 -08:00
drivers soc: qcom: llcc: Fix LLCC_TRP_ATTR2_CFGn offset 2024-01-25 15:44:53 -08:00
fs pNFS: Fix the pnfs block driver's calculation of layoutget size 2024-01-25 15:44:44 -08:00
include virtio/vsock: send credit update during setting SO_RCVLOWAT 2024-01-25 15:44:52 -08:00
init As usual, lots of singleton and doubleton patches all over the tree and 2023-11-02 20:53:31 -10:00
io_uring io_uring/cmd: fix breakage in SOCKET_URING_OP_SIOC* implementation 2023-12-14 16:52:13 -07:00
ipc Many singleton patches against the MM code. The patch series which are 2023-11-02 19:38:47 -10:00
kernel bpf: Use c->unit_size to select target cache during free 2024-01-25 15:44:54 -08:00
lib test_bpf: Rename second ALU64_SMOD_X to ALU64_SMOD_K 2024-01-25 15:44:50 -08:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm mm/memory_hotplug: fix memmap_on_memory sysfs value retrieval 2024-01-20 11:53:02 +01:00
net virtio/vsock: send credit update during setting SO_RCVLOWAT 2024-01-25 15:44:52 -08:00
rust Kbuild updates for v6.7 2023-11-04 08:07:19 -10:00
samples Landlock updates for v6.7-rc1 2023-11-03 09:28:53 -10:00
scripts scripts/decode_stacktrace.sh: optionally use LLVM utilities 2024-01-20 11:53:02 +01:00
security selinux: Fix error priority for bind with AF_UNSPEC on PF_INET6 socket 2024-01-25 15:44:40 -08:00
sound ALSA: hda: cs35l41: Support more HP models without _DSD 2024-01-20 11:53:00 +01:00
tools selftests/net: fix grep checking for fib_nexthop_multiprefix 2024-01-25 15:44:52 -08:00
usr arch: Remove Itanium (IA-64) architecture 2023-09-11 08:13:17 +00:00
virt KVM/arm64 fixes for 6.7, part #2 2023-12-22 18:03:54 -05:00
.clang-format iommu: Add for_each_group_device() 2023-05-23 08:15:51 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore kbuild: rpm-pkg: generate kernel.spec in rpmbuild/SPECS/ 2023-10-03 20:49:09 +09:00
.mailmap 12 hotfixes. 2 are cc:stable and the remainder either address post-6.7 2024-01-05 13:46:18 -08:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS 12 hotfixes. 2 are cc:stable and the remainder either address post-6.7 2024-01-05 13:46:18 -08:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS 12 hotfixes. 2 are cc:stable and the remainder either address post-6.7 2024-01-05 13:46:18 -08:00
Makefile Linux 6.7.1 2024-01-20 11:53:02 +01:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.