No description
  • C 96.8%
  • Assembly 0.9%
  • Rust 0.7%
  • Shell 0.6%
  • Python 0.5%
  • Other 0.3%
Find a file
Ritesh Oedayrajsingh Varma ff34657aa7 bpf: optimize bpf_map_update_elem() for map-in-map types
Updating a BPF_MAP_TYPE_HASH_OF_MAPS or BPF_MAP_TYPE_ARRAY_OF_MAPS via
bpf_map_update_elem() is very expensive.

In one of our workloads, we're inserting ~1400 maps of type
BPF_MAP_TYPE_ARRAY into a BPF_MAP_TYPE_ARRAY_OF_MAPS. This takes ~21
seconds on a single thread, with an average of ~15ms per call:

Function Name:    map_update_elem
Number of calls:  1369
Total time:       21s 182ms 966µs
Maximum:          47ms 937µs
Average:          15ms 473µs
Minimum:          7µs

Profiling shows that nearly all of this time is going to synchronize_rcu(),
via maybe_wait_bpf_programs() in map_update_elem().

The call to synchronize_rcu() is done to ensure that after
bpf_map_update_elem() returns, no BPF programs are still looking at the old
value of the map, per commit 1ae80cf319 ("bpf: wait for running BPF
programs when updating map-in-map").

As discussed on the bpf mailing list, replace synchronize_rcu() with
synchronize_rcu_expedited(). This is 175x faster: it now takes an average
of 88 microseconds per call, for a total of 127 milliseconds in the same
benchmark:

Function Name:    map_update_elem
Number of calls:  1439
Total time:       127ms 626µs
Maximum:          445µs
Average:          88µs
Minimum:          10µs

Link: https://lore.kernel.org/bpf/CAH6OuBR=w2kybK6u7aH_35B=Bo1PCukeMZefR=7V4Z2tJNK--Q@mail.gmail.com/

Signed-off-by: Ritesh Oedayrajsingh Varma <ritesh@superluminal.eu>
Link: https://lore.kernel.org/r/20251128000422.20462-1-ritesh@superluminal.eu
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
History 2025-11-29 09:48:41 -08:00
arch bpf: specify the old and new poke_type for bpf_arch_text_poke 2025-11-24 09:47:03 -08:00
block block-6.18-20251031 2025-10-31 12:57:19 -07:00
certs sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3 2024-09-20 19:52:48 +03:00
crypto This push contains the following changes: 2025-10-10 08:56:16 -07:00
Documentation docs: bpf: map_array: Specify BPF_MAP_TYPE_PERCPU_ARRAY value size limit 2025-11-25 14:32:00 -08:00
drivers pci-v6.18-fixes-5 2025-11-14 15:45:31 -08:00
fs More NFS Client Bugfixes for Linux 6.18-rc 2025-11-14 13:44:23 -08:00
include rqspinlock: Enclose lock/unlock within lock entry acquisitions 2025-11-29 09:35:35 -08:00
init printk changes for 6.18 2025-10-04 11:13:11 -07:00
io_uring io_uring/rsrc: don't use blk_rq_nr_phys_segments() as number of bvecs 2025-11-12 08:25:33 -07:00
ipc namespace-6.18-rc1 2025-09-29 11:20:29 -07:00
kernel bpf: optimize bpf_map_update_elem() for map-in-map types 2025-11-29 09:48:41 -08:00
lib Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf after 6.18-rc5+ 2025-11-14 17:43:41 -08:00
LICENSES LICENSES: Replace the obsolete address of the FSF in the GFDL-1.2 2025-07-24 11:15:39 +02:00
mm slab fix for 6.18-rc6 2025-11-13 11:42:44 -08:00
net bpf: Remove smap argument from bpf_selem_free() 2025-11-18 16:20:25 -08:00
rust rust: Add -fno-isolate-erroneous-paths-dereference to bindgen_skip_c_flags 2025-11-10 08:37:06 +08:00
samples samples/bpf: Fix spelling typos in samples/bpf 2025-10-18 19:26:23 -07:00
scripts Rust fixes for v6.18 (2nd) 2025-11-14 15:36:15 -08:00
security integrity-v6.18 2025-10-05 10:48:33 -07:00
sound ALSA: usb-audio: Add native DSD quirks for PureAudio DAC series 2025-11-14 14:19:47 +01:00
tools selftests/bpf: do not hardcode target rate in test_tc_edt BPF program 2025-11-29 09:37:41 -08:00
usr gen_init_cpio: Ignore fsync() returning EINVAL on pipes 2025-10-07 09:53:05 -07:00
virt KVM: guest_memfd: Remove bindings on memslot deletion when gmem is dying 2025-11-04 09:16:53 -08:00
.clang-format memblock: drop for_each_free_mem_pfn_range_in_zone_from() 2025-09-14 08:49:03 +03:00
.clippy.toml rust: clean Rust 1.88.0's warning about clippy::disallowed_macros configuration 2025-05-07 00:11:47 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.editorconfig .editorconfig: remove trim_trailing_whitespace option 2024-06-13 16:47:52 +02:00
.get_maintainer.ignore MAINTAINERS: remove Alyssa Rosenzweig 2025-09-18 21:17:31 +02:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore .gitignore: ignore compile_commands.json globally 2025-08-12 15:53:55 -07:00
.mailmap KVM/arm654 fixes for 6.18, take #2 2025-11-09 08:07:55 +01:00
.pylintrc tools: docs: parse-headers.py: move it from sphinx dir 2025-08-29 15:54:42 -06:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: mark ISDN subsystem as orphan 2025-10-27 17:49:45 -07:00
Kbuild sched: Make migrate_{en,dis}able() inline 2025-09-25 09:57:16 +02:00
Kconfig io_uring: Rename KConfig to Kconfig 2025-02-19 14:53:27 -07:00
MAINTAINERS Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf after 6.18-rc5+ 2025-11-14 17:43:41 -08:00
Makefile Linux 6.18-rc5 2025-11-09 15:10:19 -08:00
README README: Fix spelling 2024-03-18 03:36:32 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the reStructuredText markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.