Commit graph

1,461,981 commits

Author SHA1 Message Date
Amery Hung
79c9dc93fc bpf: Zero kfunc arg meta before error paths can read it
check_kfunc_call() reads meta.func_name when bpf_fetch_kfunc_arg_meta()
returns -EACCES, but that error can come from fetch_kfunc_meta() (e.g.
fd_array_get_btf() rejecting BTF binding for a signed program) before
meta is memset(), leaving it uninitialized and risking a garbage deref
in verbose().

Move the memset() to the start of bpf_fetch_kfunc_arg_meta() so meta is
zeroed on every error return. The intended "not allowed" -EACCES path
still sets func_name first, so its message is unchanged.

Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-3-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-17 14:14:15 -07:00
Amery Hung
918787e8f5 bpf: Disable raw mode for bloom filter map_peek
For a bloom filter, the value argument of bpf_map_peek_elem() is always
an input. Therefore, the verifier should not allow passing uninitialized
stack memory to it to avoid information leak.

bpf_map_peek_elem() tags its value argument ARG_PTR_TO_MAP_VALUE |
MEM_UNINIT, telling the verifier the callee fills the buffer. This holds
for queue/stack maps, but not for a bloom filter, which reads the buffer
as an input to test set membership and never writes it.

As a result, a program can pass an uninitialized stack buffer to
bpf_map_peek_elem() on a bloom filter. The verifier accepts it and marks
the buffer initialized on return, letting the program read back leftover
kernel stack memory. Bloom maps require CAP_BPF to create, so this is a
CAP_BPF-gated stack infoleak that bypasses the boundary CAP_BPF is meant
to enforce (arbitrary kernel reads are gated behind CAP_PERFMON).

Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-2-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-17 14:12:46 -07:00
Eduard Zingerman
ba6cb45969 Merge branch 'fixes-for-tailcall-with-subprog-and-callback'
Pu Lehui says:

====================
Fixes for tailcall with subprog and callback

From: Pu Lehui <pulehui@huawei.com>

This series fix two issues related to tailcall:

patch 1: Fix tail_call_reachable state leak across call branches.
patch 2: Reject callback subprogs invoke tailcall.
patch 3: testcases for patch 1 and 2.

v3:
- Fix tail_call_reachable state leak across call branches. (Sashiko)
- Add testcase for the above issue.

v2:
https://lore.kernel.org/bpf/20260714024459.420075-1-pulehui@huaweicloud.com
- Remove redantant testcase.
- Add ACK tag by Eduard.

v1:
https://lore.kernel.org/bpf/20260711104727.4023420-1-pulehui@huaweicloud.com
====================

Link: https://patch.msgid.link/20260716120157.835937-1-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-16 17:46:26 -07:00
Pu Lehui
42bfd21a8b selftests/bpf: Add testcases for callback with tailcall
Add 2 testcases for callback with tailcall:
1. failure case: callback->subprog->tailcall.
2. success case: subprog with tailcall do not affect
   no-tailcall callback.

Signed-off-by: Pu Lehui <pulehui@huawei.com>
Link: https://patch.msgid.link/20260716120157.835937-4-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-16 17:46:25 -07:00
Pu Lehui
a41d0c30d7 bpf: Reject callback subprogs invoke tailcall
Some JIT compilers, such as x86_64, rely on a register to pass the TCC.
When subprograms of synchronous callback invoke tailcall, C helpers
invoking bpf callback clobber this register, and the corrupted TCC may
bypass the TCC limit, leading to infinite tailcall.

Fix this by rejecting tailcall inside all subprogs of sync callback.
This also cleanly consolidates the existing async and exception callback
checks into a single unified `is_cb` check.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Reported-by: Björn Töpel <bjorn@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260716120157.835937-3-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-16 17:46:25 -07:00
Pu Lehui
3513ea9dab bpf: Sync tail_call_reachable with callee state on entry
Currently in check_max_stack_depth_subprog, when the verifier enters a
new callee branch, the local tail_call_reachable is not properly
synchronized with the callee's state.

Consider a main prog branching into multiple subprogs:

       subprog0 -> tailcall
main <
       subprog1 -> subprog2

When the verifier finishes checking subprog0 and backtracks to main
prog, the local tail_call_reachable state is left as true. As it
proceeds to subprog1, this uncleared state leaks into the new branch,
falsely marking subprog1 and subprog2 as tailcall reachable.

Fix this by explicitly syncing tail_call_reachable with the callee's
has_tail_call state on entry. The caller's state is safely preserved and
restored via the existing backtracking logic.

Fixes: ebf7d1f508 ("bpf, x64: rework pro/epilogue and tailcall handling in JIT")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Link: https://patch.msgid.link/20260716120157.835937-2-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-16 17:46:25 -07:00
Avinash Duduskar
d1f4b56417
selftests/bpf: Fix fib_lookup VLAN tests on hosts with forwarding on
The VLAN tests assume the test namespace starts with IPv4 forwarding off,
but a new netns copies conf/all and conf/default from init_net
(devinet_init_net(), with net.core.devconf_inherit_init_net at its
default), so on a host with net.ipv4.conf.all.forwarding=1 the devices in
the netns come up with forwarding already enabled. IPv6 uses compiled
defaults at the same sysctl value, so only the IPv4 arms are affected.

Two arms break as a result.

The arms that expect BPF_FIB_LKUP_RET_FWD_DISABLED see the lookup pass the
forwarding check and return SUCCESS instead, so fib_lookup fails:

  test_fib_lookup:FAIL:fib_lookup_ret unexpected fib_lookup_ret: actual 0 != expected 5

Pin forwarding off in setup_netns() before the devices are created; the
existing per-device writes still enable it where the tests need it.

The netns arm has the opposite problem. It checks that a VLAN device in
another netns is not resolved and expects NOT_FWDED. The lookup runs
against the caller's FIB, which had no route to the destination, so a
kernel that resolved the moved device anyway also returned NOT_FWDED and
the arm passed regardless of the namespace check. On a forwarding-on host,
where the resolved device clears the forwarding gate, this makes the arm a
tautology. Add a route so a resolved device returns SUCCESS and the arm
can tell the two apart.

Verified by deleting the netns check from bpf_fib_vlan_input_dev(): with
the fix the arm fails on both a forwarding-off host (actual 5) and a
forwarding-on host (actual 0), where before it passed on the latter. The
real kernel passes the full suite on both.

Fixes: e54a87872e ("selftests/bpf: Add bpf_fib_lookup() VLAN flag tests")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260713163826.D70201F000E9@smtp.kernel.org/
Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Link: https://lore.kernel.org/bpf/20260715100349.2684391-1-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 13:10:41 +02:00
Kumar Kartikeya Dwivedi
c314bcaa9d
Merge branch 'unify-helper-and-kfunc-call_arg_meta'
Amery Hung says:

====================
Unify helper and kfunc call_arg_meta

Hi all,

I am working toward unifying helper and kfunc handling in the verifier.
Historically, kfunc verification was mostly done separately from helper.
This causes code/logic duplication and may introduce subtle bugs as the
same checks are implemnented differently.

To address this, I plan to unify function call verification, which will
operate on a common function call descriptor bpf_func_proto.

As the first step, this patchset merges bpf_{kfunc_}call_arg_meta of
kfunc and helepr. The structure is used as a scratch pad for different
helper/kfunc verification mechanisms. The previous object tracking
refactor patchset has unified some of them. This patch handles the rest
and finally merge the two different structures.

The next step will be generating bpf_func_proto for kfunc before the
main verification loop and align kfunc and helper argument types.

Then hopefully helper and kfunc can go throguh the same call verification
path.

Changelog

v1 -> v2
 - Fix patch 2: ARG_PTR_TO_MAP_VALUE | MEM_UNINIT should also accept raw
   mode
 - Add patch 3: making check_func_arg() less verbose
 - Add patch 5: a selftest testing a preexisting OOB access bug when
   0 is passed to a kfunc's {rdwr,rdonly}_buf_size argument
 - Add patch 6: dropping pkt_access from bpf_call_arg_meta

====================

Link: https://patch.msgid.link/20260715064047.1793790-1-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:49 +02:00
Amery Hung
bf9c1b911f
bpf: Unify helper and kfunc call argument meta
Helper and kfunc argument checking carried two separate meta structs: the
verifier-local struct bpf_call_arg_meta and bpf_kfunc_call_arg_meta.
Merge them into a single struct bpf_call_arg_meta. This is groundwork for
sharing argument checking between helpers and kfuncs.

While merging, drop the btf_id field from the helper meta since it is
never used.

No functional change.

Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-8-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:48 +02:00
Amery Hung
d55149ff8c
bpf: Drop redundant pkt_access from bpf_call_arg_meta
meta->pkt_access is only ever a copy of fn->pkt_access, assigned once in
check_helper_call() and read back in may_access_direct_pkt_data(). Have
may_access_direct_pkt_data() take the bpf_func_proto and read
fn->pkt_access directly, and drop the meta field along with its
assignment.

The only non-NULL caller, check_func_arg(), already has fn in scope; the
remaining callers pass NULL and are unaffected.

No functional change intended.

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-7-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:47 +02:00
Amery Hung
1e63cd6be0
selftests/bpf: Test kfunc returning zero-sized allocation buffer
Add a test passing an explicit rdwr_buf_size of 0 to
bpf_kfunc_call_test_get_rdwr_mem() and then reading the returned R0.
R0 should be a zero-sized PTR_TO_MEM, so the access must be rejected
with "min value is outside of the allowed memory range".

This covers the pre-existing bug where a zero size argument was
treated as "no size argument": the verifier fell through to
btf_resolve_size() and sized R0 after the pointed-to return type,
wrongly allowing the read.

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-6-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:47 +02:00
Amery Hung
8faaa93b9f
bpf: Unify helper and kfunc allocation-size argument handling
The constant "size of the PTR_TO_MEM returned in R0" argument is handled by
both helpers (ARG_CONST_ALLOC_SIZE_OR_ZERO) and kfuncs (__rdonly_buf_size /
__rdwr_buf_size), each with its own meta field (meta->mem_size,
meta->r0_size) and duplicated validation. Add struct arg_alloc_mem_desc
and a shared process_const_alloc_mem_size(), and replace both fields with
meta->arg_alloc_mem.

The desc records presence with a 'found' flag instead of using a non-zero
size as the sentinel. This also fixes a pre-existing bug on the kfunc
return path: "no size argument" was tested as r0_size == 0, so an explicit
__rdonly_buf_size/__rdwr_buf_size of 0 was treated as absent and fell
through to btf_resolve_size(), giving R0 the size of the pointed-to return
type instead of 0. With 'found', an explicit zero size is honored and
btf_resolve_size() is used only when no size argument was passed.

The size is stored in a u32, matching regs[R0].mem_size. The U32_MAX
check now apply to both helper and kfunc through
process_const_alloc_mem_size().

Fold bpf_session_cookie return size assignment into current kfunc return
size resolution path.

Note that verifier saves kfunc return size through r0_size instead of
mem_size. The later has no active readers so remove it.

Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-5-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:47 +02:00
Amery Hung
77a4974c17
bpf: Pass argno to callees in check_func_arg() instead of argno_from_reg(regno)
check_func_arg() only ever handles register arguments (the caller
loops over the first MAX_BPF_FUNC_REG_ARGS arguments), so a single
argno_t built from the register number identifies the argument for
every callee. Remove the duplicated argno_from_reg() calls to
simplify check_func_arg().

'regno' is still kept for the few places that need the raw register
number directly (register reads, verbose R%d messages) and for
referring to the neighbouring size/memory argument in the
ARG_CONST_SIZE{,_OR_ZERO} cases.

No functional change intended.

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-4-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:46 +02:00
Amery Hung
92ec8b1b6b
bpf: Factor out raw_mode-related fields in bpf_call_arg_meta
To prepare for unifying the helper and kfunc call_arg_meta, group the
scattered MEM_UNINIT ("raw") memory argument fields (raw_mode, regno and
access_size) into a new struct arg_raw_mem_desc. The intention is to
make it clear about when these are set and used instead of fields with
overly generic names.

Identify the raw argument once, up front, in check_raw_mode_ok() (like
check_proto_release_reg() does for release_regno), recording its regno.
check_stack_range_initialized() now recognizes the raw buffer by matching
that regno, so the separate raw_mode flag is no longer needed, and the
per-argument "meta->raw_mode = arg_type & MEM_UNINIT" assignments in
check_func_arg() go away with it.

A raw memory argument can be tagged either ARG_PTR_TO_MEM | MEM_UNINIT or
ARG_PTR_TO_MAP_VALUE | MEM_UNINIT (the output buffer of bpf_map_pop_elem()
and bpf_map_peek_elem()). Either may be passed as a PTR_TO_STACK, which
reaches check_stack_range_initialized() through check_helper_mem_access(),
so both must be treated as raw. Extend arg_type_is_raw_mem() to match the
map value case as well; otherwise check_raw_mode_ok() would not record the
regno for it and an uninitialized stack buffer passed to those helpers
would be wrongly rejected for programs without CAP_PERFMON.

No functional change intended. This patch does not enable raw_mode
memory access for kfunc (i.e., uninit stack will not be allowed to be
passed to kfunc for unprivileged programs). Existing kfuncs with arguments
tagged with __uninit are either priviledged or dynptr kfuncs, which take
another path to make sure the access is checked by check_mem_access().

Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-3-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:46 +02:00
Amery Hung
c28cbef2f8
bpf: Remove dynptr check in check_stack_range_initialized()
For a MEM_UNINIT ("raw mode") helper argument,
check_stack_range_initialized() open-coded a scan that rejected any
STACK_DYNPTR slot in the range with "potential write to dynptr". This
duplicated, and was stricter than, the handling that runs when the
buffer is actually marked initialized. check_helper_call() later replays
the write byte by byte via check_mem_access(), which goes through
destroy_if_dynptr_stack_slot(), which rejects overwritting a referenced
dynptr. Therefore drop the redundant scan and rely on check_mem_access().

Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-2-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 11:00:45 +02:00
Jianlin Shi
f68df52fba
docs: bpf: Document BPF_RB_OVERWRITE_POS in bpf_ringbuf_query
BPF_RB_OVERWRITE_POS is supported by bpf_ringbuf_query() but was missing
from the helper documentation. Add it to the flags list in both the
kernel UAPI header and its tools/ mirror.

Signed-off-by: Jianlin Shi <shijianlin11@foxmail.com>
Acked-by: Xu Kuohai <xukuohai@huawei.com>
Link: https://lore.kernel.org/bpf/tencent_22134645443B75ED907D2A85A47AD554A709@qq.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 10:58:18 +02:00
Kumar Kartikeya Dwivedi
074927b976
Merge branch 'bpf-bidirectional-vlan-support-for-bpf_fib_lookup'
Avinash Duduskar says:

====================
bpf: bidirectional VLAN support for bpf_fib_lookup()

This series adds VLAN awareness to bpf_fib_lookup() in both directions.
BPF_FIB_LOOKUP_VLAN resolves a VLAN egress to its underlying real device
plus the VLAN tag (XDP programs need this because VLAN devices have no
XDP xmit), and BPF_FIB_LOOKUP_VLAN_INPUT runs the lookup as if a tagged
frame had arrived on the matching VLAN subinterface, for iif policy
routing and VRF table selection.

BPF_FIB_LOOKUP_VLAN opts in to replacing params->ifindex, whose value
existing programs consume since d1c362e1dd ("bpf: Always return
target ifindex in bpf_fib_lookup"); without it the output is unchanged.

An egress that does not reduce to a real device plus one tag (a QinQ
stack, or a parent in another network namespace) returns
BPF_FIB_LKUP_RET_VLAN_FAILURE with params->ifindex left at the input;
repeating the lookup without the flag, with a re-initialized params,
returns the VLAN device's own ifindex. A VLAN on a bond reduces to the
bond, which picks its egress slave at xmit.

The new return code is appended after BPF_FIB_LKUP_RET_NO_SRC_ADDR
(nothing renumbered, tools/ mirror updated) and is returned only when
the flag is set, so no existing caller can observe it.

Changes v6 -> v7:

- Patch 1 (BPF_FIB_LOOKUP_VLAN: resolve a VLAN egress to its real
  device plus the tag): uapi doc clarified, repeating the lookup after
  BPF_FIB_LKUP_RET_VLAN_FAILURE needs a re-initialized params, since
  output fields overwrite the inputs they share storage with. No
  functional change.
- Patch 2 (BPF_FIB_LOOKUP_VLAN_INPUT: run the lookup as if the tagged
  frame arrived on the matching VLAN subinterface): no code change; a
  commit message correction (an invalid proto returns -EINVAL under
  !CONFIG_VLAN_8021Q too).
- Patch 3 (selftests for both flags, tc and XDP paths): local defines
  for the netns subtest addresses (Emil's review); the netns input arm
  brings the moved device up first, so the namespace check is the only
  condition it can fail on; the live-frames subtest uses its own netns
  name (no collision under test_progs -j) and counts only the test's
  TCP frames, so background traffic cannot satisfy the delivery
  assertion; a stale mtu comment corrected.

v6: https://lore.kernel.org/all/20260704092159.1256823-1-avinash.duduskar@gmail.com/
v5: https://lore.kernel.org/all/20260624030530.3342884-1-avinash.duduskar@gmail.com/
v4: https://lore.kernel.org/all/20260623025147.1001664-1-avinash.duduskar@gmail.com/
v3: https://lore.kernel.org/all/20260617224729.1428662-1-avinash.duduskar@gmail.com/
v2: https://lore.kernel.org/all/20260616223426.3568080-1-avinash.duduskar@gmail.com/
v1: https://lore.kernel.org/all/20260609172052.81613-1-avinash.duduskar@gmail.com/
====================

Link: https://patch.msgid.link/20260713162305.1237211-1-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 10:49:59 +02:00
Avinash Duduskar
e54a87872e
selftests/bpf: Add bpf_fib_lookup() VLAN flag tests
Cover both new VLAN flags in the fib_lookup test. BPF_FIB_LOOKUP_VLAN
reduces a VLAN egress to its physical parent plus the tag, and
BPF_FIB_LOOKUP_VLAN_INPUT scopes the lookup to a VLAN subinterface.

BPF_FIB_LOOKUP_VLAN is XDP-only, since VLAN devices have no XDP xmit; the
tc helper rejects it with -EINVAL, which the table runner asserts for
every flag arm, and the egress result is checked through
bpf_xdp_fib_lookup(). Non-VLAN cases run through both helpers and assert
the path-independent results match; the XDP loop also checks dmac and,
for the tot_len cases, the route mtu_result, so the VLAN-egress dmac and
frag-needed coverage stays even though the tc path no longer reaches it.

The egress arms pin the reduction (parent ifindex plus tag, including
via a neighbour on the VLAN device, in OUTPUT mode, over a bond, and
through a DIRECT|TBID table) and the failure contract: a stacked-VLAN
(QinQ) egress returns BPF_FIB_LKUP_RET_VLAN_FAILURE with params->ifindex
left at the input. That is distinct from a no-neighbour return, which
reports the egress ifindex; only VLAN_FAILURE rewinds params->ifindex,
and a guard arm whose input and egress devices differ pins the
distinction. The VLAN_FAILURE arms are IPv4; the IPv6 path reaches it
through the same shared code, so an IPv6 arm would only re-test that.

The input arms use an iif rule that routes one destination to two
gateways, so the asserted gateway reveals which device the lookup used
as ingress, including VRF table selection through the l3mdev rule and
l3mdev_fib_table_rcu(). The VRF arms are IPv4-only: the l3mdev match
and table resolution are family-independent core shared by both rule
paths, and the IPv6 iif feed is pinned by the IPv6 VLAN input arm. A
cross-netns subtest moves a VLAN device into a second netns while it
stays registered on its parent and checks both directions fail closed
at the boundary.

A live-frames subtest (test_fib_lookup_vlan_redirect, with
BPF_F_TEST_XDP_LIVE_FRAMES) drives real frames through the native
xdp_do_redirect() / xdp_do_flush() path: a reducible egress is
redirected to the parent and delivered to its peer, while a QinQ egress
is passed to the stack, since redirecting to the VLAN device would drop
the frame at flush (no ndo_xdp_xmit).

The remaining per-case assertions are in the test table: resolution
semantics, the -EINVAL and NOT_FWDED error arms, and the SRC/SKIP_NEIGH
combinations.

Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260713162305.1237211-4-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 10:49:58 +02:00
Avinash Duduskar
217828aad8
bpf: Add BPF_FIB_LOOKUP_VLAN_INPUT flag to bpf_fib_lookup() helper
BPF_FIB_LOOKUP_VLAN resolves a VLAN egress. The reverse is also
useful: an XDP program receiving a VLAN-tagged frame on a physical
device wants the lookup to behave as if the packet had arrived on the
corresponding VLAN subinterface, so iif-based policy routing and VRF
table selection use the right ingress.

Add BPF_FIB_LOOKUP_VLAN_INPUT. When set, params->h_vlan_proto and
params->h_vlan_TCI are read as an input VLAN tag and the matching VLAN
device of params->ifindex is resolved with __vlan_find_dev_deep_rcu().
The device must be up and in the same network namespace as
params->ifindex (a VLAN device can be moved to another netns while
registered on its parent; receive would deliver into that other
namespace, which a lookup here cannot represent). If params->ifindex
is itself a VLAN device, its inner (QinQ) subinterface is matched.
For a bond or team, a tag on a port matches no device and returns
NOT_FWDED; pass the master's ifindex.
The lookup then runs with the resolved device as the ingress;
params->ifindex itself is not modified on the input side. When the
resolved device is enslaved to a VRF, both the full lookup (via the
l3mdev rule) and BPF_FIB_LOOKUP_DIRECT (via l3mdev_fib_table_rcu())
select the VRF's table from the resolved ingress. That follows from
feeding the resolved device to the flow as the ingress
(fl4.flowi4_iif = dev->ifindex), which is what makes l3mdev resolve
the VRF master from the subinterface rather than from
params->ifindex.

The two failure classes get different treatment on purpose. A
h_vlan_proto other than 802.1Q/802.1ad is API misuse and returns
-EINVAL, since it would otherwise reach the WARN in vlan_proto_idx()
with a program-controlled value. An unmatched VID, a device that is
down, or one in another namespace is a data outcome and returns
BPF_FIB_LKUP_RET_NOT_FWDED, matching the DIRECT path when
fib_get_table() finds no table and mirroring real ingress, where the
receive path drops such frames. A VID of 0 (a priority tag) is looked
up literally and normally fails the same way; receive instead
processes such frames untagged, so callers should not set the flag for
priority tags. Proceeding on the physical device for any of these
would be fail-open for the policy-routing cases above.

The h_vlan fields share a union with tbid, so the flag cannot be
combined with BPF_FIB_LOOKUP_TBID. It describes ingress, so it also
cannot be combined with BPF_FIB_LOOKUP_OUTPUT. Both combinations
return -EINVAL; restricting now keeps a later relaxation backward
compatible. Combining with BPF_FIB_LOOKUP_VLAN is allowed: the tag is
consumed on the ingress side and the egress tag is written on
success.

Under !CONFIG_VLAN_8021Q the __vlan_find_dev_deep_rcu() stub returns
NULL, so every lookup with a valid proto returns NOT_FWDED, which is
correct since no VLAN device can exist.

Suggested-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260713162305.1237211-3-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 10:49:58 +02:00
Avinash Duduskar
35dac1daeb
bpf: Add BPF_FIB_LOOKUP_VLAN flag to bpf_fib_lookup() helper
bpf_fib_lookup() returns the FIB-resolved egress ifindex straight
from the fib result. When the egress is a VLAN device, the returned
ifindex is the VLAN netdev's, which has no XDP xmit handler; XDP
programs that want to forward the frame (e.g. xdp-forward) must
instead target the underlying physical device and push the VLAN tag
themselves. Today the program has no way to learn either the
underlying ifindex or the VLAN tag without maintaining its own
VLAN-to-ifindex map in userspace and refreshing it on netlink
events.

Add BPF_FIB_LOOKUP_VLAN. When the caller sets this flag and the fib
result is a VLAN device whose immediate parent is a real (non-VLAN)
device in the same network namespace, populate the existing output
fields params->h_vlan_proto and params->h_vlan_TCI from the VLAN
device and replace params->ifindex with the parent's ifindex.
params->h_vlan_TCI carries the VID only, with PCP and DEI bits zero; a
consumer wanting to set egress priority writes PCP itself.
params->smac is the VLAN device's own address, which can differ from
the parent's.

Only the immediate parent is resolved, via vlan_dev_priv(dev)->real_dev
and not vlan_dev_real_dev(), which walks to the bottom of a stack. When
the immediate parent is not a real device in the same namespace, the
lookup returns BPF_FIB_LKUP_RET_VLAN_FAILURE and leaves params->ifindex
at the input. This covers a stacked VLAN (QinQ), where the immediate
parent is itself a VLAN device and one h_vlan_proto/h_vlan_TCI pair
cannot describe two tags, and a parent in another network namespace (a
VLAN device can be moved while its parent stays), whose ifindex would
be meaningless in the caller's namespace. A program that wants the
VLAN device's own ifindex re-issues the lookup, with a re-initialized
params, without BPF_FIB_LOOKUP_VLAN, so the unreducible case stays
distinct from a physical egress. That distinction matters for XDP: a
program cannot xmit on a VLAN device, so a success carrying the VLAN
ifindex would make it redirect to a device with no ndo_xdp_xmit and
drop the frame at xdp_do_flush(). The swap and the vlan fields are
written only on the reduce path; other output fields keep their
existing behaviour, so a frag-needed result still reports the route
mtu in params->mtu_result.

BPF_FIB_LOOKUP_VLAN is only useful to XDP, which cannot redirect to a
VLAN device. A tc program can redirect to the VLAN device directly, so
bpf_skb_fib_lookup() rejects the flag with -EINVAL; bpf_xdp_fib_lookup()
accepts it. When the flag is not set, behaviour is unchanged:
h_vlan_proto and h_vlan_TCI are zeroed and ifindex is left at the FIB
result.

The new block is compiled only under CONFIG_VLAN_8021Q since
vlan_dev_priv() is not defined otherwise; without that config
is_vlan_dev() is constant false and the flag is accepted but never
acts. That is safe because no VLAN device can exist there, so every
egress is already physical.

This lets an XDP redirect target the physical device and learn the
tag to push in a single lookup, which xdp-forward's optional VLAN
mode (xdp-project/xdp-tools#504) wants from the kernel side.

The helper's input semantics are unchanged; the reverse direction
(supplying a tag as lookup input) is added in the following patch.

Suggested-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: David Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/bpf/20260713162305.1237211-2-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-15 10:49:57 +02:00
Kumar Kartikeya Dwivedi
9a3a07d06e
Merge branch 'bpf-fix-warning-in-bpf_trampoline_multi_detach'
Leon Hwang says:

====================
bpf: Fix WARNING in bpf_trampoline_multi_detach

Since tracing_multi link does not set ftrace_managed, it would fail to
release the tracing_multi link when attaching tracing_multi link and
then attaching fentry link.

[    3.714215] WARNING: kernel/bpf/trampoline.c:1727 at bpf_trampoline_multi_detach+0x20b/0x240, CPU#1: test_progs/97
...
[    3.733170]  bpf_tracing_multi_link_release+0x14/0x30
[    3.733890]  bpf_link_free+0x58/0x130
[    3.734414]  bpf_link_release+0x23/0x30

Fix it by setting 'ftrace_managed = true' in register_fentry_multi().

Note: the reason of targeting bpf-next tree is I'm planning to add
tracing_multi link support for bpf progs, that would rely on the fix and
have code conflict with the test.

Changes:
v1 -> v2:
* Collect Acked-by from Jiri, thanks.
* Check arg using tracing_multi_arg_check() in bpf prog in test.
  (per Sashiko)
* v1: https://lore.kernel.org/bpf/20260710144734.78203-1-leon.hwang@linux.dev/
====================

Link: https://patch.msgid.link/20260711124822.29406-1-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-12 01:56:22 +02:00
Leon Hwang
539d3edf8e
selftests/bpf: Test fentry link after tracing_multi link
Verify that there's no any issue to attach tracing_multi link, then attach
fentry link.

Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260711124822.29406-3-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-12 01:56:19 +02:00
Leon Hwang
30bdd6d138
bpf: Mark tracing_multi trampolines as ftrace managed
Since tracing_multi link does not set ftrace_managed, it would fail to
release the tracing_multi link when attaching tracing_multi link and
then attaching fentry link.

[    3.714215] WARNING: kernel/bpf/trampoline.c:1727 at bpf_trampoline_multi_detach+0x20b/0x240, CPU#1: test_progs/97
...
[    3.733170]  bpf_tracing_multi_link_release+0x14/0x30
[    3.733890]  bpf_link_free+0x58/0x130
[    3.734414]  bpf_link_release+0x23/0x30

Fix it by setting 'ftrace_managed = true' in register_fentry_multi().

Fixes: aef4dfa790 ("bpf: Add bpf_trampoline_multi_attach/detach functions")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260711124822.29406-2-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-12 01:56:14 +02:00
Kumar Kartikeya Dwivedi
25935519f7
Merge branch 'bpf-require-an-owned-cpumask-for-bpf_cpumask_populate'
Nicholas Dudar says:

====================
bpf: require an owned cpumask for bpf_cpumask_populate()

bpf_cpumask_populate() writes its destination with bitmap_copy() but
types it as struct cpumask *, so the verifier accepts a borrowed,
read-only cpumask (for example from scx_bpf_get_online_cpumask()) as a
writable destination. Fix it and add a selftest for coverage.

Changelog:
----------
v2 -> v3 (v1/v2 were a private report to security@kernel.org)

 * Split the fix and selftest into separate patches.
 * Target bpf-next instead of bpf.
====================

Link: https://patch.msgid.link/20260709182800.2037938-1-main.kalliope@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-12 01:52:55 +02:00
Nicholas Dudar
6267b83528
selftests/bpf: Test bpf_cpumask_populate() rejects a borrowed cpumask
bpf_cpumask_populate() now takes a struct bpf_cpumask *, so update the
kfunc declaration and drop the struct cpumask * casts in the existing
populate tests. Add test_populate_borrowed_destination, which passes a
borrowed task->cpus_ptr and asserts the verifier rejects it as a writable
destination.

Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260709182800.2037938-3-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-12 01:52:50 +02:00
Nicholas Dudar
8740156ad3
bpf: Require a BPF cpumask for bpf_cpumask_populate()
bpf_cpumask_populate() writes to its destination with bitmap_copy(), but
the destination is typed as struct cpumask *. That allows the verifier to
accept borrowed cpumask pointers returned by read-only kfuncs, such as
scx_bpf_get_online_cpumask(), as a writable destination.

Make the destination a struct bpf_cpumask * so populate follows the same
ownership rule as the other mutating cpumask kfuncs. Query kfuncs continue
to accept const struct cpumask * inputs.

Fixes: 950ad93df2 ("bpf: add kfunc for populating cpumask bits")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260709182800.2037938-2-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-12 01:52:36 +02:00
Naveed Khan
77f02c9926 libbpf: Fix double-free of distilled base BTF on .BTF.ext parse error
When btf_parse_elf() is called without a caller-supplied base_btf (i.e.
via the public btf__parse_elf()) and the object file carries a .BTF.base
(distilled base) section, a dist_base_btf object is created and used as
the base of the split BTF built from the .BTF section.  Because base_btf
is NULL, the relocation block that would otherwise free and clear
dist_base_btf is skipped, and ownership of dist_base_btf is instead
transferred to the split btf by setting btf->owns_base = true.

That ownership transfer was performed before the fallible btf_ext__new()
call that parses the .BTF.ext section.  If .BTF.ext is malformed,
btf_ext__new() fails and the function jumps to the error path, which
frees dist_base_btf directly and then frees btf.  Since owns_base is
already set, btf__free(btf) also frees btf->base_btf, which is the same
dist_base_btf object.  The result is a use-after-free read followed by a
double free of the base BTF, driven entirely by a crafted object file
(a .BTF + .BTF.base + malformed .BTF.ext combination) passed to
btf__parse_elf(), as used by bpftool, pahole and similar tools.

Transfer ownership only after .BTF.ext has been parsed successfully, so
that any earlier failure leaves dist_base_btf owned solely by the local
cleanup path and it is freed exactly once.

Signed-off-by: Naveed Khan <naveed@digiscrypt.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/178345549172.94179.7948304165383170781@digiscrypt.com
2026-07-10 14:53:29 -07:00
Emil Tsalapatis
e821c22387 selftests/bpf: veristat: Minimize map size during verification
The veristat tool verifies that BPF objects along with their
maps pass verification for a given kernel version. To do so,
veristat loads the objects and maps into the kernel in order
to pass them through the verifier.

Currently, veristat sizes the maps according to the max_entries
field provided by the program author. Depending on the map type
this field may be irrelevant to the verification process. However,
loading a large map can fail because of -ENOMEM errors.

This is a problem when the map is supposed to run on large machines,
but veristat tests it machines with significantly less RAM (e.g.,
CI). In that case veristat fails even if the program verifies.

Expand veristat to resize maps whose max_entries are not relevant
to verification. Set the max_entries value as low as possible to
avoid -ENOMEM errors.

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260708193239.5063-1-emil@etsalapatis.com
2026-07-10 14:43:50 -07:00
Mahe Tardy
30f77a0419
bpf: Fix unused nskb warning in bpf_icmp_send
Declare nskb inside the IPv4 and IPv6 case blocks so it is only present
when the corresponding code is built. The case braces are intentional to
scope the local declarations under the switch labels.

Fixes: f3603df9ae ("bpf: Add bpf_icmp_send kfunc")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Link: https://lore.kernel.org/bpf/20260710180915.7105-1-mahe.tardy@gmail.com
Closes: https://lore.kernel.org/oe-kbuild-all/202607110140.JeJZ6GIa-lkp@intel.com/
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-10 20:52:25 +02:00
Mahe Tardy
b1d4514ff1 selftests/bpf: Add bpf_icmp_send no route test
For normal live cgroup_skb paths, the skb should already be routed. The
exception is for test run via BPF_PROG_TEST_RUN with packets created
via bpf_prog_test_run_skb. Those lack dst route and thus the icmp_send
would quietly fail by returning early.

This test exercises this and makes sure the kfunc returns -ENETUNREACH.

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-6-mahe.tardy@gmail.com
2026-07-10 11:57:02 +02:00
Mahe Tardy
49d07ba673 selftests/bpf: Add bpf_icmp_send recursion test
This test is similar to test_icmp_send_unreach_cgroup but checks that,
in case of recursion, meaning that the BPF program calling the kfunc was
re-triggered by the icmp_send done by the kfunc, the kfunc will stop
early and return -EBUSY.

The test attaches to the root cgroup to ensure the ICMP packet generated
by the kfunc re-triggers the BPF program.

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-5-mahe.tardy@gmail.com
2026-07-10 11:56:56 +02:00
Mahe Tardy
340a40df94 selftests/bpf: Add bpf_icmp_send kfunc cgroup_skb IPv6 tests
This test extends the existing cgroup_skb tests with IPv6 support.

Note that we need to set IPV6_RECVERR on the socket for IPv6 in
connect_to_fd_nonblock otherwise the error will be ignored even if
we are in the middle of the TCP handshake. See in
net/ipv6/datagram.c:ipv6_icmp_error for more details.

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-4-mahe.tardy@gmail.com
2026-07-10 11:56:47 +02:00
Mahe Tardy
39b337a3d9 selftests/bpf: Add bpf_icmp_send kfunc cgroup_skb tests
This test opens a server and client, enters a new cgroup, attach a
cgroup_skb program on egress and calls the bpf_icmp_send function from
the client egress so that an ICMP unreach control message is sent back
to the client. It then fetches the message from the error queue to
confirm the correct ICMP unreach code has been sent.

Note that, for the client, we have to connect in non-blocking mode to
let the test execute faster. Otherwise, we need to wait for the TCP
three-way handshake to timeout in the kernel before reading the errno.

Also note that we don't set IP_RECVERR on the socket in
connect_to_fd_nonblock since the error will be transferred anyway in our
test because the connection is rejected at the beginning of the TCP
handshake. See in net/ipv4/tcp_ipv4.c:tcp_v4_err for more details.

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-3-mahe.tardy@gmail.com
2026-07-10 11:56:43 +02:00
Mahe Tardy
f3603df9ae bpf: Add bpf_icmp_send kfunc
This is needed in the context of Tetragon to provide improved feedback
(in contrast to just dropping packets) to east-west traffic when blocked
by policies using cgroup_skb programs.

This reuses concepts from netfilter reject target codepath with the
differences that:

* Packets are cloned since the BPF user can still let the packet pass
  (SK_PASS from the cgroup_skb progs for example) and the current skb
  need to stay untouched (cgroup_skb hooks only allow read-only skb
  payload).
* We protect against recursion since the kfunc, by generating an ICMP
  error message, could retrigger the BPF prog that invoked it.

Only ICMP_DEST_UNREACH and ICMPV6_DEST_UNREACH are currently supported.
The interface accepts a type parameter to facilitate future extension to
other ICMP control message types.

For normal cgroup_skb paths, the skb dst route should already be set.
However, bpf_prog_test_run_skb can create synthetic IPv4/IPv6 skbs
without an attached route. In that case, icmp_send returns early, and
the kfunc would otherwise report success despite no ICMP reply being
sent. This check also reject metadata dsts, which are not valid struct
rtable instances. While IPv6 would stricly require only rejecting
metadata dsts, same check is applied for API consistency.

Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-2-mahe.tardy@gmail.com
2026-07-10 11:56:23 +02:00
Kumar Kartikeya Dwivedi
a4553044d1
Merge branch 'bpf-bound-rdonly-rdwr_buf_size-kfunc-return-size'
Nicholas Dudar says:

====================
bpf: bound rdonly/rdwr_buf_size kfunc return size

check_kfunc_args() stores a kfunc's rdonly_buf_size/rdwr_buf_size
argument into a u64 that check_kfunc_call() later narrows into the
returned register's u32 mem_size, so a value above U32_MAX truncates
instead of being rejected. Fix it and add a selftest for coverage.

Changelog:
----------
v1 -> v2 (v1 was a private report to security@kernel.org)

 * Split the fix and selftest into separate patches.
 * Target bpf-next instead of bpf.
====================

Link: https://patch.msgid.link/20260709155837.1879230-1-main.kalliope@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 22:21:53 +02:00
Nicholas Dudar
12556c3198
selftests/bpf: Add test for oversized rdonly/rdwr_buf_size kfunc argument
Add a load-failure test to the kfunc_call suite using the existing
bpf_kfunc_call_test_get_rdwr_mem() test kfunc. Its rdwr_buf_size
argument is a const int, so the test uses a 64-bit immediate load in
inline asm to place 2^64 - 192 (0xffffffffffffff40) in the argument
register. The verifier records r0_size from the full 64-bit register
value, and the test asserts that BPF_PROG_LOAD rejects it with
"rdonly/rdwr_buf_size exceeds u32 max".

Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260709155837.1879230-3-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 22:21:45 +02:00
Nicholas Dudar
2aaf67f051
bpf: Reject rdonly/rdwr_buf_size kfunc arguments that exceed u32 max
check_kfunc_args() detects a kfunc argument named rdonly_buf_size or
rdwr_buf_size and stores reg->var_off.value into meta->r0_size, a u64,
and does not bound it. check_kfunc_call() later copies that value into
the returned register's mem_size field:

  meta->r0_size = reg->var_off.value;
  ...
  regs[BPF_REG_0].mem_size = meta.r0_size;

regs[BPF_REG_0].mem_size is u32. A constant whose upper 32 bits are set
gets truncated instead of causing a load-time rejection, so the verifier
records a PTR_TO_MEM register with an approximately 4 GiB mem_size for
whatever allocation the kfunc returned. A later access check against
that register uses the truncated, wrong bound.

Reject rdonly_buf_size/rdwr_buf_size values that exceed U32_MAX at the
point meta->r0_size is set.

Fixes: eb1f7f71c1 ("bpf/verifier: allow kfunc to return an allocated mem")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260709155837.1879230-2-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 22:18:21 +02:00
Daniel Borkmann
c3d5ef291a selftests/bpf: Add test for scalar id on sign-extending stack fill
Add a verifier test where a spilled scalar is filled once via a sign-
extending load (BPF_MEMSX) and once via a zero-extending load (BPF_MEM).
The two destination registers must not share a scalar id.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_scalar_ids
  [...]
  #643/1   verifier_scalar_ids/linked_regs_bpf_k:OK
  #643/2   verifier_scalar_ids/linked_regs_bpf_x_src:OK
  #643/3   verifier_scalar_ids/linked_regs_bpf_x_dst:OK
  #643/4   verifier_scalar_ids/linked_regs_broken_link:OK
  #643/5   verifier_scalar_ids/precision_many_frames:OK
  #643/6   verifier_scalar_ids/precision_stack:OK
  #643/7   verifier_scalar_ids/precision_two_ids:OK
  #643/8   verifier_scalar_ids/linked_regs_too_many_regs:OK
  #643/9   verifier_scalar_ids/linked_regs_broken_link_2:OK
  #643/10  verifier_scalar_ids/cjmp_no_linked_regs_trigger:OK
  #643/11  verifier_scalar_ids/check_ids_in_regsafe:OK
  #643/12  verifier_scalar_ids/check_ids_in_regsafe_2:OK
  #643/13  verifier_scalar_ids/no_scalar_id_for_const:OK
  #643/14  verifier_scalar_ids/no_scalar_id_for_const32:OK
  #643/15  verifier_scalar_ids/ignore_unique_scalar_ids_cur:OK
  #643/16  verifier_scalar_ids/ignore_unique_scalar_ids_old:OK
  #643/17  verifier_scalar_ids/two_nil_old_ids_one_cur_id:OK
  #643/18  verifier_scalar_ids/two_old_ids_one_cur_id:OK
  #643/19  verifier_scalar_ids/linked_regs_and_subreg_def:OK
  #643/20  verifier_scalar_ids/ldsx_fill_scalar_id_not_shared:OK
  #643     verifier_scalar_ids:OK
  Summary: 1/20 PASSED, 0 SKIPPED, 0 FAILED

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-09 12:19:26 -07:00
Daniel Borkmann
2cb5f4ca69 bpf: Drop scalar id on sign-extending narrowing stack fills
When a spilled scalar is filled back with a sign-extending narrowing load
(BPF_MEMSX), check_stack_read_fixed_off() copies the spilled register
including its scalar id, but coerce_reg_to_size_sx() then sign-extends the
filled register's value. If the same slot is also filled with a plain
zero-extending load (BPF_MEM), both destination registers share the id yet
hold different values. A later 'if <zext-reg> == const' then refines the
sign-extended register through sync_linked_regs() to a value it does not
have at runtime (e.g. the verifier believes 0x80000000 while the register
is 0xffffffff80000000), which can be turned into an out-of-bounds access.

Drop the shared scalar id at the sign-extension site in check_mem_access()
when sign extension actually changes the value, mirroring the BPF_MOVSX
handling in check_alu_op() (no_sext = reg_umax < 2^(size*8-1)).

Fixes: 3cd5c89065 ("bpf: Let the verifier assign ids on stack fills")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-09 12:19:26 -07:00
Daniel Borkmann
36ffa86c42
bpf: Fix security_bpf_map_create error handling
Commit 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
made the LSM hook wrappers for BPF object creation clean up the LSM
state internally upon denial, e.g. security_bpf_map_create() internally
calls security_bpf_map_free() when the bpf_map_create hook returns an
error. map_create() however still routes a denial to its free_map_sec
label, which invokes security_bpf_map_free() a second time, so the
bpf_map_free hook fires twice for a single denied map.

In-tree LSMs are unaffected in practice since the blob kfree() inside
security_bpf_map_free() is NULL-safe and idempotent and none of them
implement bpf_map_free, but a BPF LSM program attached to that hook
observes double invocations. Route the denial to free_map instead.

Fixes: 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260709073422.379247-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 10:16:34 +02:00
Eduard Zingerman
55db6a4759 Merge branch 'introduce-jit_required-to-prevent-a-kernel-panic'
Tiezhu Yang says:

====================
Introduce jit_required to prevent a kernel panic

This series introduces a 'jit_required' flag in struct bpf_prog
to track programs that strictly require JIT.

The aim is to prevent a kernel panic by rejecting programs with
inlined helpers when JIT is not available.

v8 -> v9:
  -- Initialize 'fp->jit_required' with
     'IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON)'
     in bpf_prog_alloc_no_stats()
  -- Simplify __bpf_prog_select_runtime()
  -- Simplify the commit description for patch 1
  -- Enhance the commit description for patch 2
====================

Link: https://patch.msgid.link/20260708101806.18885-1-yangtiezhu@loongson.cn
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-08 23:25:39 -07:00
Tiezhu Yang
f1c2792257 bpf: Reject programs with inlined helpers if JIT is not available
When an architecture (such as LoongArch, ARM64, and RISC-V) implements
bpf_jit_inlines_helper_call(), the verifier skips rewriting the helper
call offset (insn->imm) in bpf_do_misc_fixups(). This is because the
helper is expected to be inlined by the JIT compiler later. Therefore,
insn->imm remains as the raw helper enum ID.

However, if JIT is disabled at runtime (net.core.bpf_jit_enable=0) or
if JIT compilation fails dynamically (e.g., due to OOM), the program
falls back to the BPF interpreter.

When the interpreter executes (__bpf_call_base + insn->imm) with the
unpatched raw ID, it jumps into an invalid address space, triggering
an instruction alignment fault or a kernel panic.

Although these helpers have valid C implementations in the kernel, the
omission of offset rewriting makes runtime interpreter fallback fatal.

Fix this by setting 'prog->jit_required = 1' when helper call rewriting
is skipped for JIT inlining. This ensures that such programs are safely
rejected if JIT is not available, preventing the runtime kernel panic.

Fixes: 2ddec2c80b ("riscv, bpf: inline bpf_get_smp_processor_id()")
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-08 23:25:26 -07:00
Tiezhu Yang
9a6df65d5c bpf: Introduce jit_required flag and remove bpf_prog_has_kfunc_call()
Introduce a 'jit_required' bitfield flag in struct bpf_prog to track
whether a BPF program strictly requires the JIT compiler to run. This
prevents a dangerous runtime fallback to the interpreter for features
that are only implemented in the JIT compiler.

Currently, bpf_prog_has_kfunc_call() is used only for kernel function
calls, replace the kfunc-specific helper with the new 'jit_required'
flag. This makes it easy to support other JIT-only BPF features, such
as inlined helpers.

Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Suggested-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-07-08 23:24:41 -07:00
Sanghyun Park
47b079e211
bpf: Fix use-after-free on mm_struct in bpf_find_vma()
bpf_find_vma() reads task->mm and calls mmap_read_trylock(mm) without
holding a reference on the mm. On a foreign task, a concurrent exit_mm()
can free the mm_struct between the lockless read and the trylock,
resulting in a use-after-free. mm_struct is not SLAB_TYPESAFE_BY_RCU.

For the current task, task->mm is stable. For a foreign task, pin the mm
under task->alloc_lock and release it with mmput_async(), mirroring commit
d8e27d2d22 ("bpf: fix mm lifecycle in open-coded task_vma iterator").
Use spin_trylock() instead of get_task_mm() so BPF context does not block
on alloc_lock. Reject irqs-disabled contexts and !CONFIG_MMU on the
foreign-task path because dropping the mm reference is not safe there.

Race:

  CPU0 (BPF program)                  CPU1 (exiting task)
  ============================        ==========================
  bpf_find_vma(foreign_task):
    mm = task->mm
                                      exit_mm():
                                        task->mm = NULL
                                        mmput(mm) -> frees mm_struct
    mmap_read_trylock(mm)
        // UAF on mm

Fixes: 7c7e3d31e7 ("bpf: Introduce helper bpf_find_vma")
Signed-off-by: Sanghyun Park <sanghyun.park.cnu@gmail.com>
Reviewed-by: Puranjay Mohan <puranjay@kernel.org>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/bpf/20260708072106.199637-2-sanghyun.park.cnu@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 07:59:41 +02:00
Kumar Kartikeya Dwivedi
e318f9dd8b
Merge branch 'misc-bpf-fixes-from-sashiko-findings'
Daniel Borkmann says:

====================
Misc BPF fixes from sashiko findings

Addressing some misc/random findings from sashiko that are worth fixing
which popped up as pre-existing issues while working on the signed BPF
loader series.
====================

Link: https://patch.msgid.link/20260708211537.371874-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 07:00:03 +02:00
Daniel Borkmann
ff755b6007
bpf: Account scratch buffer in bpf_prog_calc_tag
bpf_prog_calc_tag() copies the instructions into a plain vmalloc() scratch
buffer to blind the map fds before hashing. The buffer scales with the
program, up to ~8MB at the 1M instruction limit, and is allocated on every
program load, but unlike the rest of the load-time scratch memory it is
not charged to the loader's memcg. Use GFP_KERNEL_ACCOUNT to account it
like the other allocations scoped to the verification/load.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-5-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 07:00:02 +02:00
Daniel Borkmann
42560699a8
bpf: Account insn_aux_data allocation in bpf_check
The insn_aux_data array is allocated with a plain vzalloc(), while every
other allocation scoped to the verification - verifier states, explored
states, the cfg/scc arrays, liveness masks, jump history - is charged to
the loader's memcg via GFP_KERNEL_ACCOUNT.

At 136 bytes per instruction it is one of the largest verification-time
buffers, in the range of ~130MB for a program at the 1M instruction limit
(worst case), and it lives across the whole verification. The buffer is
also inconsistent with itself: when instruction patching grows it, the
vrealloc() in bpf_patch_insn_data() already passes GFP_KERNEL_ACCOUNT.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 07:00:02 +02:00
Daniel Borkmann
5e5e94d87d
bpf: Give vmlinux BTF init its own mutex
bpf_get_btf_vmlinux() serializes the lazy vmlinux BTF parse with
bpf_verifier_lock, the same mutex bpf_check() holds across the whole
verification of an unprivileged program (if enabled; it's disabled
by default). The latter can potentially stall the mutex holder for
a long time (e.g. via userfaultfd), and therefore block first-time
bpf_get_btf_vmlinux() caller from any context, including privileged
program loads.

Give the vmlinux BTF initialization a dedicated btf_vmlinux_lock so
it is independent of the unprivileged verification mutex. The parse
only needs mutual exclusion against itself.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 07:00:01 +02:00
Daniel Borkmann
92863e6780
bpf: Fix vmlinux BTF prep race in bpf_get_btf_vmlinux
bpf_get_btf_vmlinux() lazily parses the vmlinux BTF under the
bpf_verifier_lock, but publishes the result through a plain store
and re-checks it through a plain lockless load. Nothing orders
the stores initializing the struct btf inside btf_parse_vmlinux()
against the store publishing the pointer: On a weakly ordered
arch, a concurrent first-time caller taking the lockless fast
path could in principle observe the pointer before the parsed
contents are visible. The mutex_unlock() does not help such a
reader given it only synchronizes with a later acquisition of the
same lock. Thus, publish the pointer with smp_store_release()
and read it on the fast path with smp_load_acquire().

Acquire semantics are needed rather than a dependency-ordered
READ_ONCE(): btf_parse_vmlinux() also populates globals outside
the returned object (e.g. bpf_ctx_convert.t). An address
dependency would only order accesses performed through the
pointer and not cover other globals.

Fixes: 8580ac9404 ("bpf: Process in-kernel BTF")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-09 06:59:20 +02:00
Maxim Khmelevskii
41ec7e4a17
selftests/bpf: Skip res_spin_lock_stress if no perf support
Probe PMU support before loading bpf_test_rqspinlock.ko,
otherwise the test fails with not obvious error without
proper perf event support:

    Failed to load bpf_test_rqspinlock.ko into the kernel: -2
    serial_test_res_spin_lock_stress:FAIL:load module AA
        unexpected error: -22 (errno 2)

Reported-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Maxim Khmelevskii <max@linux.ibm.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Link: https://lore.kernel.org/bpf/20260708104729.1248234-2-max@linux.ibm.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-07-08 21:55:23 +02:00