Avinash Duduskar says:
====================
bpf: bidirectional VLAN support for bpf_fib_lookup()
This series adds VLAN awareness to bpf_fib_lookup() in both directions.
BPF_FIB_LOOKUP_VLAN resolves a VLAN egress to its underlying real device
plus the VLAN tag (XDP programs need this because VLAN devices have no
XDP xmit), and BPF_FIB_LOOKUP_VLAN_INPUT runs the lookup as if a tagged
frame had arrived on the matching VLAN subinterface, for iif policy
routing and VRF table selection.
BPF_FIB_LOOKUP_VLAN opts in to replacing params->ifindex, whose value
existing programs consume since d1c362e1dd ("bpf: Always return
target ifindex in bpf_fib_lookup"); without it the output is unchanged.
An egress that does not reduce to a real device plus one tag (a QinQ
stack, or a parent in another network namespace) returns
BPF_FIB_LKUP_RET_VLAN_FAILURE with params->ifindex left at the input;
repeating the lookup without the flag, with a re-initialized params,
returns the VLAN device's own ifindex. A VLAN on a bond reduces to the
bond, which picks its egress slave at xmit.
The new return code is appended after BPF_FIB_LKUP_RET_NO_SRC_ADDR
(nothing renumbered, tools/ mirror updated) and is returned only when
the flag is set, so no existing caller can observe it.
Changes v6 -> v7:
- Patch 1 (BPF_FIB_LOOKUP_VLAN: resolve a VLAN egress to its real
device plus the tag): uapi doc clarified, repeating the lookup after
BPF_FIB_LKUP_RET_VLAN_FAILURE needs a re-initialized params, since
output fields overwrite the inputs they share storage with. No
functional change.
- Patch 2 (BPF_FIB_LOOKUP_VLAN_INPUT: run the lookup as if the tagged
frame arrived on the matching VLAN subinterface): no code change; a
commit message correction (an invalid proto returns -EINVAL under
!CONFIG_VLAN_8021Q too).
- Patch 3 (selftests for both flags, tc and XDP paths): local defines
for the netns subtest addresses (Emil's review); the netns input arm
brings the moved device up first, so the namespace check is the only
condition it can fail on; the live-frames subtest uses its own netns
name (no collision under test_progs -j) and counts only the test's
TCP frames, so background traffic cannot satisfy the delivery
assertion; a stale mtu comment corrected.
v6: https://lore.kernel.org/all/20260704092159.1256823-1-avinash.duduskar@gmail.com/
v5: https://lore.kernel.org/all/20260624030530.3342884-1-avinash.duduskar@gmail.com/
v4: https://lore.kernel.org/all/20260623025147.1001664-1-avinash.duduskar@gmail.com/
v3: https://lore.kernel.org/all/20260617224729.1428662-1-avinash.duduskar@gmail.com/
v2: https://lore.kernel.org/all/20260616223426.3568080-1-avinash.duduskar@gmail.com/
v1: https://lore.kernel.org/all/20260609172052.81613-1-avinash.duduskar@gmail.com/
====================
Link: https://patch.msgid.link/20260713162305.1237211-1-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Cover both new VLAN flags in the fib_lookup test. BPF_FIB_LOOKUP_VLAN
reduces a VLAN egress to its physical parent plus the tag, and
BPF_FIB_LOOKUP_VLAN_INPUT scopes the lookup to a VLAN subinterface.
BPF_FIB_LOOKUP_VLAN is XDP-only, since VLAN devices have no XDP xmit; the
tc helper rejects it with -EINVAL, which the table runner asserts for
every flag arm, and the egress result is checked through
bpf_xdp_fib_lookup(). Non-VLAN cases run through both helpers and assert
the path-independent results match; the XDP loop also checks dmac and,
for the tot_len cases, the route mtu_result, so the VLAN-egress dmac and
frag-needed coverage stays even though the tc path no longer reaches it.
The egress arms pin the reduction (parent ifindex plus tag, including
via a neighbour on the VLAN device, in OUTPUT mode, over a bond, and
through a DIRECT|TBID table) and the failure contract: a stacked-VLAN
(QinQ) egress returns BPF_FIB_LKUP_RET_VLAN_FAILURE with params->ifindex
left at the input. That is distinct from a no-neighbour return, which
reports the egress ifindex; only VLAN_FAILURE rewinds params->ifindex,
and a guard arm whose input and egress devices differ pins the
distinction. The VLAN_FAILURE arms are IPv4; the IPv6 path reaches it
through the same shared code, so an IPv6 arm would only re-test that.
The input arms use an iif rule that routes one destination to two
gateways, so the asserted gateway reveals which device the lookup used
as ingress, including VRF table selection through the l3mdev rule and
l3mdev_fib_table_rcu(). The VRF arms are IPv4-only: the l3mdev match
and table resolution are family-independent core shared by both rule
paths, and the IPv6 iif feed is pinned by the IPv6 VLAN input arm. A
cross-netns subtest moves a VLAN device into a second netns while it
stays registered on its parent and checks both directions fail closed
at the boundary.
A live-frames subtest (test_fib_lookup_vlan_redirect, with
BPF_F_TEST_XDP_LIVE_FRAMES) drives real frames through the native
xdp_do_redirect() / xdp_do_flush() path: a reducible egress is
redirected to the parent and delivered to its peer, while a QinQ egress
is passed to the stack, since redirecting to the VLAN device would drop
the frame at flush (no ndo_xdp_xmit).
The remaining per-case assertions are in the test table: resolution
semantics, the -EINVAL and NOT_FWDED error arms, and the SRC/SKIP_NEIGH
combinations.
Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260713162305.1237211-4-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
BPF_FIB_LOOKUP_VLAN resolves a VLAN egress. The reverse is also
useful: an XDP program receiving a VLAN-tagged frame on a physical
device wants the lookup to behave as if the packet had arrived on the
corresponding VLAN subinterface, so iif-based policy routing and VRF
table selection use the right ingress.
Add BPF_FIB_LOOKUP_VLAN_INPUT. When set, params->h_vlan_proto and
params->h_vlan_TCI are read as an input VLAN tag and the matching VLAN
device of params->ifindex is resolved with __vlan_find_dev_deep_rcu().
The device must be up and in the same network namespace as
params->ifindex (a VLAN device can be moved to another netns while
registered on its parent; receive would deliver into that other
namespace, which a lookup here cannot represent). If params->ifindex
is itself a VLAN device, its inner (QinQ) subinterface is matched.
For a bond or team, a tag on a port matches no device and returns
NOT_FWDED; pass the master's ifindex.
The lookup then runs with the resolved device as the ingress;
params->ifindex itself is not modified on the input side. When the
resolved device is enslaved to a VRF, both the full lookup (via the
l3mdev rule) and BPF_FIB_LOOKUP_DIRECT (via l3mdev_fib_table_rcu())
select the VRF's table from the resolved ingress. That follows from
feeding the resolved device to the flow as the ingress
(fl4.flowi4_iif = dev->ifindex), which is what makes l3mdev resolve
the VRF master from the subinterface rather than from
params->ifindex.
The two failure classes get different treatment on purpose. A
h_vlan_proto other than 802.1Q/802.1ad is API misuse and returns
-EINVAL, since it would otherwise reach the WARN in vlan_proto_idx()
with a program-controlled value. An unmatched VID, a device that is
down, or one in another namespace is a data outcome and returns
BPF_FIB_LKUP_RET_NOT_FWDED, matching the DIRECT path when
fib_get_table() finds no table and mirroring real ingress, where the
receive path drops such frames. A VID of 0 (a priority tag) is looked
up literally and normally fails the same way; receive instead
processes such frames untagged, so callers should not set the flag for
priority tags. Proceeding on the physical device for any of these
would be fail-open for the policy-routing cases above.
The h_vlan fields share a union with tbid, so the flag cannot be
combined with BPF_FIB_LOOKUP_TBID. It describes ingress, so it also
cannot be combined with BPF_FIB_LOOKUP_OUTPUT. Both combinations
return -EINVAL; restricting now keeps a later relaxation backward
compatible. Combining with BPF_FIB_LOOKUP_VLAN is allowed: the tag is
consumed on the ingress side and the egress tag is written on
success.
Under !CONFIG_VLAN_8021Q the __vlan_find_dev_deep_rcu() stub returns
NULL, so every lookup with a valid proto returns NOT_FWDED, which is
correct since no VLAN device can exist.
Suggested-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260713162305.1237211-3-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_fib_lookup() returns the FIB-resolved egress ifindex straight
from the fib result. When the egress is a VLAN device, the returned
ifindex is the VLAN netdev's, which has no XDP xmit handler; XDP
programs that want to forward the frame (e.g. xdp-forward) must
instead target the underlying physical device and push the VLAN tag
themselves. Today the program has no way to learn either the
underlying ifindex or the VLAN tag without maintaining its own
VLAN-to-ifindex map in userspace and refreshing it on netlink
events.
Add BPF_FIB_LOOKUP_VLAN. When the caller sets this flag and the fib
result is a VLAN device whose immediate parent is a real (non-VLAN)
device in the same network namespace, populate the existing output
fields params->h_vlan_proto and params->h_vlan_TCI from the VLAN
device and replace params->ifindex with the parent's ifindex.
params->h_vlan_TCI carries the VID only, with PCP and DEI bits zero; a
consumer wanting to set egress priority writes PCP itself.
params->smac is the VLAN device's own address, which can differ from
the parent's.
Only the immediate parent is resolved, via vlan_dev_priv(dev)->real_dev
and not vlan_dev_real_dev(), which walks to the bottom of a stack. When
the immediate parent is not a real device in the same namespace, the
lookup returns BPF_FIB_LKUP_RET_VLAN_FAILURE and leaves params->ifindex
at the input. This covers a stacked VLAN (QinQ), where the immediate
parent is itself a VLAN device and one h_vlan_proto/h_vlan_TCI pair
cannot describe two tags, and a parent in another network namespace (a
VLAN device can be moved while its parent stays), whose ifindex would
be meaningless in the caller's namespace. A program that wants the
VLAN device's own ifindex re-issues the lookup, with a re-initialized
params, without BPF_FIB_LOOKUP_VLAN, so the unreducible case stays
distinct from a physical egress. That distinction matters for XDP: a
program cannot xmit on a VLAN device, so a success carrying the VLAN
ifindex would make it redirect to a device with no ndo_xdp_xmit and
drop the frame at xdp_do_flush(). The swap and the vlan fields are
written only on the reduce path; other output fields keep their
existing behaviour, so a frag-needed result still reports the route
mtu in params->mtu_result.
BPF_FIB_LOOKUP_VLAN is only useful to XDP, which cannot redirect to a
VLAN device. A tc program can redirect to the VLAN device directly, so
bpf_skb_fib_lookup() rejects the flag with -EINVAL; bpf_xdp_fib_lookup()
accepts it. When the flag is not set, behaviour is unchanged:
h_vlan_proto and h_vlan_TCI are zeroed and ifindex is left at the FIB
result.
The new block is compiled only under CONFIG_VLAN_8021Q since
vlan_dev_priv() is not defined otherwise; without that config
is_vlan_dev() is constant false and the flag is accepted but never
acts. That is safe because no VLAN device can exist there, so every
egress is already physical.
This lets an XDP redirect target the physical device and learn the
tag to push in a single lookup, which xdp-forward's optional VLAN
mode (xdp-project/xdp-tools#504) wants from the kernel side.
The helper's input semantics are unchanged; the reverse direction
(supplying a tag as lookup input) is added in the following patch.
Suggested-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: David Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/bpf/20260713162305.1237211-2-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Leon Hwang says:
====================
bpf: Fix WARNING in bpf_trampoline_multi_detach
Since tracing_multi link does not set ftrace_managed, it would fail to
release the tracing_multi link when attaching tracing_multi link and
then attaching fentry link.
[ 3.714215] WARNING: kernel/bpf/trampoline.c:1727 at bpf_trampoline_multi_detach+0x20b/0x240, CPU#1: test_progs/97
...
[ 3.733170] bpf_tracing_multi_link_release+0x14/0x30
[ 3.733890] bpf_link_free+0x58/0x130
[ 3.734414] bpf_link_release+0x23/0x30
Fix it by setting 'ftrace_managed = true' in register_fentry_multi().
Note: the reason of targeting bpf-next tree is I'm planning to add
tracing_multi link support for bpf progs, that would rely on the fix and
have code conflict with the test.
Changes:
v1 -> v2:
* Collect Acked-by from Jiri, thanks.
* Check arg using tracing_multi_arg_check() in bpf prog in test.
(per Sashiko)
* v1: https://lore.kernel.org/bpf/20260710144734.78203-1-leon.hwang@linux.dev/
====================
Link: https://patch.msgid.link/20260711124822.29406-1-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Verify that there's no any issue to attach tracing_multi link, then attach
fentry link.
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260711124822.29406-3-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Since tracing_multi link does not set ftrace_managed, it would fail to
release the tracing_multi link when attaching tracing_multi link and
then attaching fentry link.
[ 3.714215] WARNING: kernel/bpf/trampoline.c:1727 at bpf_trampoline_multi_detach+0x20b/0x240, CPU#1: test_progs/97
...
[ 3.733170] bpf_tracing_multi_link_release+0x14/0x30
[ 3.733890] bpf_link_free+0x58/0x130
[ 3.734414] bpf_link_release+0x23/0x30
Fix it by setting 'ftrace_managed = true' in register_fentry_multi().
Fixes: aef4dfa790 ("bpf: Add bpf_trampoline_multi_attach/detach functions")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260711124822.29406-2-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Nicholas Dudar says:
====================
bpf: require an owned cpumask for bpf_cpumask_populate()
bpf_cpumask_populate() writes its destination with bitmap_copy() but
types it as struct cpumask *, so the verifier accepts a borrowed,
read-only cpumask (for example from scx_bpf_get_online_cpumask()) as a
writable destination. Fix it and add a selftest for coverage.
Changelog:
----------
v2 -> v3 (v1/v2 were a private report to security@kernel.org)
* Split the fix and selftest into separate patches.
* Target bpf-next instead of bpf.
====================
Link: https://patch.msgid.link/20260709182800.2037938-1-main.kalliope@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_cpumask_populate() now takes a struct bpf_cpumask *, so update the
kfunc declaration and drop the struct cpumask * casts in the existing
populate tests. Add test_populate_borrowed_destination, which passes a
borrowed task->cpus_ptr and asserts the verifier rejects it as a writable
destination.
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260709182800.2037938-3-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_cpumask_populate() writes to its destination with bitmap_copy(), but
the destination is typed as struct cpumask *. That allows the verifier to
accept borrowed cpumask pointers returned by read-only kfuncs, such as
scx_bpf_get_online_cpumask(), as a writable destination.
Make the destination a struct bpf_cpumask * so populate follows the same
ownership rule as the other mutating cpumask kfuncs. Query kfuncs continue
to accept const struct cpumask * inputs.
Fixes: 950ad93df2 ("bpf: add kfunc for populating cpumask bits")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260709182800.2037938-2-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
When btf_parse_elf() is called without a caller-supplied base_btf (i.e.
via the public btf__parse_elf()) and the object file carries a .BTF.base
(distilled base) section, a dist_base_btf object is created and used as
the base of the split BTF built from the .BTF section. Because base_btf
is NULL, the relocation block that would otherwise free and clear
dist_base_btf is skipped, and ownership of dist_base_btf is instead
transferred to the split btf by setting btf->owns_base = true.
That ownership transfer was performed before the fallible btf_ext__new()
call that parses the .BTF.ext section. If .BTF.ext is malformed,
btf_ext__new() fails and the function jumps to the error path, which
frees dist_base_btf directly and then frees btf. Since owns_base is
already set, btf__free(btf) also frees btf->base_btf, which is the same
dist_base_btf object. The result is a use-after-free read followed by a
double free of the base BTF, driven entirely by a crafted object file
(a .BTF + .BTF.base + malformed .BTF.ext combination) passed to
btf__parse_elf(), as used by bpftool, pahole and similar tools.
Transfer ownership only after .BTF.ext has been parsed successfully, so
that any earlier failure leaves dist_base_btf owned solely by the local
cleanup path and it is freed exactly once.
Signed-off-by: Naveed Khan <naveed@digiscrypt.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/178345549172.94179.7948304165383170781@digiscrypt.com
The veristat tool verifies that BPF objects along with their
maps pass verification for a given kernel version. To do so,
veristat loads the objects and maps into the kernel in order
to pass them through the verifier.
Currently, veristat sizes the maps according to the max_entries
field provided by the program author. Depending on the map type
this field may be irrelevant to the verification process. However,
loading a large map can fail because of -ENOMEM errors.
This is a problem when the map is supposed to run on large machines,
but veristat tests it machines with significantly less RAM (e.g.,
CI). In that case veristat fails even if the program verifies.
Expand veristat to resize maps whose max_entries are not relevant
to verification. Set the max_entries value as low as possible to
avoid -ENOMEM errors.
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260708193239.5063-1-emil@etsalapatis.com
For normal live cgroup_skb paths, the skb should already be routed. The
exception is for test run via BPF_PROG_TEST_RUN with packets created
via bpf_prog_test_run_skb. Those lack dst route and thus the icmp_send
would quietly fail by returning early.
This test exercises this and makes sure the kfunc returns -ENETUNREACH.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-6-mahe.tardy@gmail.com
This test is similar to test_icmp_send_unreach_cgroup but checks that,
in case of recursion, meaning that the BPF program calling the kfunc was
re-triggered by the icmp_send done by the kfunc, the kfunc will stop
early and return -EBUSY.
The test attaches to the root cgroup to ensure the ICMP packet generated
by the kfunc re-triggers the BPF program.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-5-mahe.tardy@gmail.com
This test extends the existing cgroup_skb tests with IPv6 support.
Note that we need to set IPV6_RECVERR on the socket for IPv6 in
connect_to_fd_nonblock otherwise the error will be ignored even if
we are in the middle of the TCP handshake. See in
net/ipv6/datagram.c:ipv6_icmp_error for more details.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-4-mahe.tardy@gmail.com
This test opens a server and client, enters a new cgroup, attach a
cgroup_skb program on egress and calls the bpf_icmp_send function from
the client egress so that an ICMP unreach control message is sent back
to the client. It then fetches the message from the error queue to
confirm the correct ICMP unreach code has been sent.
Note that, for the client, we have to connect in non-blocking mode to
let the test execute faster. Otherwise, we need to wait for the TCP
three-way handshake to timeout in the kernel before reading the errno.
Also note that we don't set IP_RECVERR on the socket in
connect_to_fd_nonblock since the error will be transferred anyway in our
test because the connection is rejected at the beginning of the TCP
handshake. See in net/ipv4/tcp_ipv4.c:tcp_v4_err for more details.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-3-mahe.tardy@gmail.com
This is needed in the context of Tetragon to provide improved feedback
(in contrast to just dropping packets) to east-west traffic when blocked
by policies using cgroup_skb programs.
This reuses concepts from netfilter reject target codepath with the
differences that:
* Packets are cloned since the BPF user can still let the packet pass
(SK_PASS from the cgroup_skb progs for example) and the current skb
need to stay untouched (cgroup_skb hooks only allow read-only skb
payload).
* We protect against recursion since the kfunc, by generating an ICMP
error message, could retrigger the BPF prog that invoked it.
Only ICMP_DEST_UNREACH and ICMPV6_DEST_UNREACH are currently supported.
The interface accepts a type parameter to facilitate future extension to
other ICMP control message types.
For normal cgroup_skb paths, the skb dst route should already be set.
However, bpf_prog_test_run_skb can create synthetic IPv4/IPv6 skbs
without an attached route. In that case, icmp_send returns early, and
the kfunc would otherwise report success despite no ICMP reply being
sent. This check also reject metadata dsts, which are not valid struct
rtable instances. While IPv6 would stricly require only rejecting
metadata dsts, same check is applied for API consistency.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-2-mahe.tardy@gmail.com
Nicholas Dudar says:
====================
bpf: bound rdonly/rdwr_buf_size kfunc return size
check_kfunc_args() stores a kfunc's rdonly_buf_size/rdwr_buf_size
argument into a u64 that check_kfunc_call() later narrows into the
returned register's u32 mem_size, so a value above U32_MAX truncates
instead of being rejected. Fix it and add a selftest for coverage.
Changelog:
----------
v1 -> v2 (v1 was a private report to security@kernel.org)
* Split the fix and selftest into separate patches.
* Target bpf-next instead of bpf.
====================
Link: https://patch.msgid.link/20260709155837.1879230-1-main.kalliope@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a load-failure test to the kfunc_call suite using the existing
bpf_kfunc_call_test_get_rdwr_mem() test kfunc. Its rdwr_buf_size
argument is a const int, so the test uses a 64-bit immediate load in
inline asm to place 2^64 - 192 (0xffffffffffffff40) in the argument
register. The verifier records r0_size from the full 64-bit register
value, and the test asserts that BPF_PROG_LOAD rejects it with
"rdonly/rdwr_buf_size exceeds u32 max".
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260709155837.1879230-3-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
check_kfunc_args() detects a kfunc argument named rdonly_buf_size or
rdwr_buf_size and stores reg->var_off.value into meta->r0_size, a u64,
and does not bound it. check_kfunc_call() later copies that value into
the returned register's mem_size field:
meta->r0_size = reg->var_off.value;
...
regs[BPF_REG_0].mem_size = meta.r0_size;
regs[BPF_REG_0].mem_size is u32. A constant whose upper 32 bits are set
gets truncated instead of causing a load-time rejection, so the verifier
records a PTR_TO_MEM register with an approximately 4 GiB mem_size for
whatever allocation the kfunc returned. A later access check against
that register uses the truncated, wrong bound.
Reject rdonly_buf_size/rdwr_buf_size values that exceed U32_MAX at the
point meta->r0_size is set.
Fixes: eb1f7f71c1 ("bpf/verifier: allow kfunc to return an allocated mem")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260709155837.1879230-2-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
When a spilled scalar is filled back with a sign-extending narrowing load
(BPF_MEMSX), check_stack_read_fixed_off() copies the spilled register
including its scalar id, but coerce_reg_to_size_sx() then sign-extends the
filled register's value. If the same slot is also filled with a plain
zero-extending load (BPF_MEM), both destination registers share the id yet
hold different values. A later 'if <zext-reg> == const' then refines the
sign-extended register through sync_linked_regs() to a value it does not
have at runtime (e.g. the verifier believes 0x80000000 while the register
is 0xffffffff80000000), which can be turned into an out-of-bounds access.
Drop the shared scalar id at the sign-extension site in check_mem_access()
when sign extension actually changes the value, mirroring the BPF_MOVSX
handling in check_alu_op() (no_sext = reg_umax < 2^(size*8-1)).
Fixes: 3cd5c89065 ("bpf: Let the verifier assign ids on stack fills")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Commit 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
made the LSM hook wrappers for BPF object creation clean up the LSM
state internally upon denial, e.g. security_bpf_map_create() internally
calls security_bpf_map_free() when the bpf_map_create hook returns an
error. map_create() however still routes a denial to its free_map_sec
label, which invokes security_bpf_map_free() a second time, so the
bpf_map_free hook fires twice for a single denied map.
In-tree LSMs are unaffected in practice since the blob kfree() inside
security_bpf_map_free() is NULL-safe and idempotent and none of them
implement bpf_map_free, but a BPF LSM program attached to that hook
observes double invocations. Route the denial to free_map instead.
Fixes: 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260709073422.379247-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Tiezhu Yang says:
====================
Introduce jit_required to prevent a kernel panic
This series introduces a 'jit_required' flag in struct bpf_prog
to track programs that strictly require JIT.
The aim is to prevent a kernel panic by rejecting programs with
inlined helpers when JIT is not available.
v8 -> v9:
-- Initialize 'fp->jit_required' with
'IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON)'
in bpf_prog_alloc_no_stats()
-- Simplify __bpf_prog_select_runtime()
-- Simplify the commit description for patch 1
-- Enhance the commit description for patch 2
====================
Link: https://patch.msgid.link/20260708101806.18885-1-yangtiezhu@loongson.cn
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
When an architecture (such as LoongArch, ARM64, and RISC-V) implements
bpf_jit_inlines_helper_call(), the verifier skips rewriting the helper
call offset (insn->imm) in bpf_do_misc_fixups(). This is because the
helper is expected to be inlined by the JIT compiler later. Therefore,
insn->imm remains as the raw helper enum ID.
However, if JIT is disabled at runtime (net.core.bpf_jit_enable=0) or
if JIT compilation fails dynamically (e.g., due to OOM), the program
falls back to the BPF interpreter.
When the interpreter executes (__bpf_call_base + insn->imm) with the
unpatched raw ID, it jumps into an invalid address space, triggering
an instruction alignment fault or a kernel panic.
Although these helpers have valid C implementations in the kernel, the
omission of offset rewriting makes runtime interpreter fallback fatal.
Fix this by setting 'prog->jit_required = 1' when helper call rewriting
is skipped for JIT inlining. This ensures that such programs are safely
rejected if JIT is not available, preventing the runtime kernel panic.
Fixes: 2ddec2c80b ("riscv, bpf: inline bpf_get_smp_processor_id()")
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Introduce a 'jit_required' bitfield flag in struct bpf_prog to track
whether a BPF program strictly requires the JIT compiler to run. This
prevents a dangerous runtime fallback to the interpreter for features
that are only implemented in the JIT compiler.
Currently, bpf_prog_has_kfunc_call() is used only for kernel function
calls, replace the kfunc-specific helper with the new 'jit_required'
flag. This makes it easy to support other JIT-only BPF features, such
as inlined helpers.
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Suggested-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
bpf_find_vma() reads task->mm and calls mmap_read_trylock(mm) without
holding a reference on the mm. On a foreign task, a concurrent exit_mm()
can free the mm_struct between the lockless read and the trylock,
resulting in a use-after-free. mm_struct is not SLAB_TYPESAFE_BY_RCU.
For the current task, task->mm is stable. For a foreign task, pin the mm
under task->alloc_lock and release it with mmput_async(), mirroring commit
d8e27d2d22 ("bpf: fix mm lifecycle in open-coded task_vma iterator").
Use spin_trylock() instead of get_task_mm() so BPF context does not block
on alloc_lock. Reject irqs-disabled contexts and !CONFIG_MMU on the
foreign-task path because dropping the mm reference is not safe there.
Race:
CPU0 (BPF program) CPU1 (exiting task)
============================ ==========================
bpf_find_vma(foreign_task):
mm = task->mm
exit_mm():
task->mm = NULL
mmput(mm) -> frees mm_struct
mmap_read_trylock(mm)
// UAF on mm
Fixes: 7c7e3d31e7 ("bpf: Introduce helper bpf_find_vma")
Signed-off-by: Sanghyun Park <sanghyun.park.cnu@gmail.com>
Reviewed-by: Puranjay Mohan <puranjay@kernel.org>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/bpf/20260708072106.199637-2-sanghyun.park.cnu@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Daniel Borkmann says:
====================
Misc BPF fixes from sashiko findings
Addressing some misc/random findings from sashiko that are worth fixing
which popped up as pre-existing issues while working on the signed BPF
loader series.
====================
Link: https://patch.msgid.link/20260708211537.371874-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_prog_calc_tag() copies the instructions into a plain vmalloc() scratch
buffer to blind the map fds before hashing. The buffer scales with the
program, up to ~8MB at the 1M instruction limit, and is allocated on every
program load, but unlike the rest of the load-time scratch memory it is
not charged to the loader's memcg. Use GFP_KERNEL_ACCOUNT to account it
like the other allocations scoped to the verification/load.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-5-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The insn_aux_data array is allocated with a plain vzalloc(), while every
other allocation scoped to the verification - verifier states, explored
states, the cfg/scc arrays, liveness masks, jump history - is charged to
the loader's memcg via GFP_KERNEL_ACCOUNT.
At 136 bytes per instruction it is one of the largest verification-time
buffers, in the range of ~130MB for a program at the 1M instruction limit
(worst case), and it lives across the whole verification. The buffer is
also inconsistent with itself: when instruction patching grows it, the
vrealloc() in bpf_patch_insn_data() already passes GFP_KERNEL_ACCOUNT.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_get_btf_vmlinux() serializes the lazy vmlinux BTF parse with
bpf_verifier_lock, the same mutex bpf_check() holds across the whole
verification of an unprivileged program (if enabled; it's disabled
by default). The latter can potentially stall the mutex holder for
a long time (e.g. via userfaultfd), and therefore block first-time
bpf_get_btf_vmlinux() caller from any context, including privileged
program loads.
Give the vmlinux BTF initialization a dedicated btf_vmlinux_lock so
it is independent of the unprivileged verification mutex. The parse
only needs mutual exclusion against itself.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_get_btf_vmlinux() lazily parses the vmlinux BTF under the
bpf_verifier_lock, but publishes the result through a plain store
and re-checks it through a plain lockless load. Nothing orders
the stores initializing the struct btf inside btf_parse_vmlinux()
against the store publishing the pointer: On a weakly ordered
arch, a concurrent first-time caller taking the lockless fast
path could in principle observe the pointer before the parsed
contents are visible. The mutex_unlock() does not help such a
reader given it only synchronizes with a later acquisition of the
same lock. Thus, publish the pointer with smp_store_release()
and read it on the fast path with smp_load_acquire().
Acquire semantics are needed rather than a dependency-ordered
READ_ONCE(): btf_parse_vmlinux() also populates globals outside
the returned object (e.g. bpf_ctx_convert.t). An address
dependency would only order accesses performed through the
pointer and not cover other globals.
Fixes: 8580ac9404 ("bpf: Process in-kernel BTF")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Probe PMU support before loading bpf_test_rqspinlock.ko,
otherwise the test fails with not obvious error without
proper perf event support:
Failed to load bpf_test_rqspinlock.ko into the kernel: -2
serial_test_res_spin_lock_stress:FAIL:load module AA
unexpected error: -22 (errno 2)
Reported-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Maxim Khmelevskii <max@linux.ibm.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Link: https://lore.kernel.org/bpf/20260708104729.1248234-2-max@linux.ibm.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The signature_enforced, signature_too_large, signature_zero_size and
signature_bad_keyring subtests load a program that must be rejected,
but leave the fd open if the kernel unexpectedly accepts the load:
test_progs asserts record the failure and continue, so the fd would
linger for the rest of the run. Just close it.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708184107.369182-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Daniel Borkmann says:
====================
Verify BPF signed loader at load time
The BPF signing scheme signs a light skeleton's loader program and lets
the loader vouch for everything else: bpftool bakes the SHA256 of the
metadata map into the loader's instructions, signs the instructions, and
the loader compares the (frozen, exclusive) map against that hash from
within BPF once it runs. The construction is sound as a trusted hash
chain, but the kernel itself never attests the metadata, and that split
has been the recurring objection from the LSM / integrity side since the
scheme was proposed.
This proposal closes both gaps by having the kernel verify the metadata
at BPF_PROG_LOAD time, before the LSM admission hook and before the
verifier, /without/ growing the UAPI. A signed loader binds its metadata
map(s) through the existing fd_array/fd_array_cnt, and exclusive maps
are already bound to the loader's digest via excl_prog_hash. When a
signature is present, the kernel collects the exclusive maps from the
fd_array and appends their frozen contents to the instructions before
PKCS#7 verification, so the signature covers ...
insns || metadata_0 || metadata_1 || [...]
... in fd_array order. The in-loader hash check is dropped from the
gen_loader entirely: generated loaders carry no verification logic
anymore, and signing or verifying a skeleton becomes an ordinary CMS
operation over bytes that sit verbatim in the skeleton, reproducible
offline. A signed program is either BPF_SIG_UNSIGNED or BPF_SIG_VERIFIED
with nothing in between.
There is no new UAPI, we now have a single signature scheme, no LSM
code reaching into BPF internals, no new LSM hook, and unsigned loads
are completely unaffected. It is also less complex since the loader
does not need to deal with BTF, an extra kfunc, etc, as proposed in
an earlier series [0]. Tested against full BPF CI which came back
green. For more details and examples, see the documentation patch in
this series.
[0] https://lore.kernel.org/bpf/20260522023234.3778588-1-kpsingh@kernel.org/
v5 -> v6:
- Added bpf_map_write_active check to close potential race on
arm64 (sashiko)
- Rebased & rest stays as-is
v4 -> v5:
- Squashed patch 2/9 and 3/9 together and dropped Nack (Paul)
- Added Anton's Acked-by on the fd_array cache patch
- Add !attr->signature_size test to reject invalid param
and added BPF selftest for this case
v3 -> v4:
- Fix upper limit in MAX_FD_ARRAY_CNT (Anton)
- Reject !fd_array && attr->fd_array_cnt (Anton)
- Add bpftool patch wrt ignored return value of EVP_Digest() (sashiko)
- Fix setting of gen_loader_fixture_init (sashiko)
- Fix unused map_fd cleanup branch in selftest (bot+bpf-ci)
- Remove now unused map->excl member and adjust selftests
- Added more BPF signed_loader corner case selftest coverage
- Added Paul's Nack wrt bpf_prog_load LSM hook dispute
- Added patch 2 to move bigger allocations below fd_array
resolution (Paul)
v2 -> v3:
- Added first commit to cache and work on objects in fd_array
which was the most recent issue sashiko rightfully complained
- Added more BPF signed_loader selftest coverage to cover that
usage of sparse fd_array or map fds gets rejected
- I left the security_bpf_prog_load as in v2 given preference
from BPF side over adding new hook
v1 -> v2:
- Addressed both sashiko complaints, the TOCTOU bug regarding
fd_array processing, as well as exclusive map checking to
only allow array maps. The validation is now moved into the
verifier before the main verification work happens. This also
gives the opportunity to utilize the verifier log.
====================
Link: https://patch.msgid.link/20260708075343.358712-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Describe the BPF signing design end to end: why a trusted loader is
needed, the signature(insns || metadata) contract, load-time
verification via fd_array (exclusive + frozen maps), the binary
BPF_SIG_{UNSIGNED,VERIFIED} verdict, and how [BPF] LSMs can enforce
policy on it.
This writes down the contract on the discussion points with the LSM /
integrity folks [0][1]: by the time security_bpf_prog_load() is
called, signature verification has fully completed and covers the
instructions plus the frozen contents of every bound exclusive map;
there is no intermediate "loader verified, payload pending" state
to reason about; and what BPF_SIG_VERIFIED means at each hook is
spelled out explicitly.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/bc823ddbaf63e0e177eb46d1cc15076e4e2e689d.camel@HansenPartnership.com [0]
Link: https://lore.kernel.org/bpf/CAHC9VhSDkwGgPfrBUh7EgBKEJj_JjnY68c0YAmuuLT_i--GskQ@mail.gmail.com [1]
Link: https://lore.kernel.org/bpf/20260708075343.358712-9-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
With write-only excl member removed from struct bpf_map, ops moves
to offset 32 and inner_map_meta to offset 40. Update the expected
verifier message for the former and retarget the latter at the sha
byte array, so the beyond-member-end rejection path stays covered:
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_map_ptr
[...]
#619/5 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected:OK
#619/6 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected @unpriv:OK
#619/7 verifier_map_ptr/bpf_map_ptr: read beyond sha field rejected:OK
#619/8 verifier_map_ptr/bpf_map_ptr: read beyond sha field rejected @unpriv:OK
#619/9 verifier_map_ptr/bpf_map_ptr: read ops field accepted:OK
#619/10 verifier_map_ptr/bpf_map_ptr: read ops field accepted @unpriv:OK
[...]
#620 verifier_map_ptr_mixing:OK
Summary: 2/20 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708075343.358712-7-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpftool_prog_sign() signed only the loader instructions. The metadata
blob the loader installs was left to an in-loader hash check, which
the kernel now performs at load time over insns || metadata.
Sign that same concatenation: pass the metadata blob (gen_loader_opts
data) through to bpftool_prog_sign() and feed insns || metadata to
CMS_final(). The excl_prog_hash stays a digest of the instructions
alone; it binds the metadata map to the loader and is matched against
prog->digest by the verifier, independent of what the signature covers.
The signed artifact is now plain data: both bytes the signature
covers are embedded verbatim in the generated skeleton, so signing
and verifying an lskel is an ordinary CMS operation that a signer or
auditor can perform (or reproduce) offline, without analyzing loader
bytecode to establish what the signature actually attests to.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Quentin Monnet <qmo@kernel.org>
Link: https://lore.kernel.org/bpf/20260708075343.358712-6-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpftool_prog_sign() ignores the return value of EVP_Digest(). If the
digest computation fails (context allocation failure, or a digest
fetch failure under OpenSSL), EVP_Digest() returns 0 and leaves the
output buffer untouched, but the function still reports success.
Fixes: 40863f4d6e ("bpftool: Add support for signing BPF programs")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Quentin Monnet <qmo@kernel.org>
Link: https://lore.kernel.org/bpf/20260708075343.358712-5-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The signed gen_loader used to police its own metadata map from within
BPF: emit_signature_match() read the kernel-cached map->sha[] back
through hardcoded struct bpf_map offsets and compared it against a hash
that compute_sha_update_offsets() baked into the signed instructions,
after a BPF_OBJ_GET_INFO_BY_FD round-trip to populate map->sha[].
The kernel now verifies the metadata at BPF_PROG_LOAD time by folding
the frozen contents of the loader's exclusive fd_array maps into the
signature, so the loader no longer checks anything itself. Generated
loaders thus carry no verification logic of their own anymore: Nothing
in the signing chain depends on emitted loader bytecode doing the right
thing.
On the loading side, skel_internal.h now sets fd_array_cnt for a signed
load so the kernel scans fd_array for the exclusive metadata map -
still frozen, as the kernel requires - and the BPF_OBJ_GET_INFO_BY_FD
round-trip to populate map->sha[] is gone. The struct bpf_map layout
BUILD_BUG_ON()s on the kernel side are removed as well: they only
pinned the ABI for the in-BPF read of map->sha[] that is no longer
needed. Same for the map->excl member. Note: gen_hash is retained; it
still marks a loader as signed so an untrusted host cannot re-dimension
maps or override initial values now covered by the signature.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708075343.358712-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
A signed gen_loader program carries the programs, maps and relocations
it installs in a metadata array map. The loader instructions are covered
by the PKCS#7 signature, but the metadata map is not: Today the loader
compares the map contents from within BPF against a hash baked into its
(signed) instructions, using the kernel-cached map hash. The kernel
itself never actually attests that the metadata the loader installs is
the metadata that was signed.
This split is the core of the long-standing objection to the BPF signing
scheme from the LSM / integrity side: the integrity check of a light
skeleton only completes once the loader program runs, that is, after the
security_bpf_prog_load() hook, so at admission time an LSM observes a
program whose payload has not yet been verified. Auditing the chain
link is also not a purely cryptographic operation: whoever signs or
reviews an lskel has to disassemble the loader's preamble to convince
themselves that the embedded hash check is present and correct [0][1].
Two acceptable fixes were identified in those threads: Complete the
integrity check before the admission hook fires, or add a second hook
that collects the verification result after the loader ran [2]. Covering
both the loader and its maps directly with the PKCS#7 signature is what
Blaise Boscaccy's patchsets proposed in several forms. Let's implement
the former, without growing the UAPI, and in particular as a single
unified scheme where the signature spans the raw bytes rather than
derived hashes.
A signed loader binds its metadata map(s) through the existing fd_array,
and an exclusive map is already bound to a program digest (excl_prog_hash).
So when a signature is present, collect the exclusive maps from fd_array
and append their frozen contents to the instructions before verification:
The signature now covers insns || metadata_0 || metadata_1 || [...] in
the fd_array order, and verification completes in bpf_check(), once the
fd_array maps are resolved into used_maps, before the LSM admission hook
and the rest of verification. A program is either BPF_SIG_UNSIGNED or
BPF_SIG_VERIFIED, with nothing in between. While folding the fd_array
maps, a non-exclusive map bound to a signed program is rejected, so every
map folded into the signature is exclusive. A signed loader that fails
to cover its metadata thus does not load, and BPF_SIG_VERIFIED always
means the instructions and every exclusive map are authentic. The maps
must be frozen so the hashed bytes cannot change before the loader runs;
the map <-> program digest binding is enforced by the verifier for every
used map. Binding maps through fd_array_cnt makes the verifier resolve
and excl-check them (excl_prog_sha vs prog->digest) before it would
otherwise compute the digest, so compute prog->digest up front in
bpf_check(), over the unmodified instructions the signature covers, for
a load that folds metadata.
Unsigned programs are not affected by the signature path; for them the
LSM admission hook merely moves below fd_array resolution, with minimal
bounded work in between. Note, signed loaders generated by older libbpf/
bpftool versions need to be regenerated; some of the recent fixes we've
had on the signed loader side require the latter already to close gaps.
Finally, some remarks around the security_bpf_prog_load() placement
given there was discussion on whether a new hook is needed or the existing
security_bpf_prog() hook should be reused [3]: For a new hook it would
mean that just for loading a single BPF program it has to pass through
four layers of LSM hooks:
1) security_bpf (cmd=PROG_LOAD): for gating various bpf subcmds
2) security_bpf_prog_load: historical admission hook (CAP/token,
prog_type, attach point), pre-verification
3) security_bpf_prog_verify_signature: newly asked admission hook,
same role as 2), plus the BPF signature verdict
4) security_bpf_prog: gate handing the prog fd back to userspace,
verification done & signature verified
The use-cases of 2) and 3) conflate, thus BPF community prefers to just
keep a total of 3 LSM hooks (as-is today): 3) makes 2) incoherent given
they are the /same class/ of hook, that is, access-control admission on
the load and split only by _what_ they can see. Worse, with the split,
for a signed BPF program security_bpf_prog_load 2) admits a program whose
signature has not been checked, so a policy gating at 2) is structurally
unable to express "admit only verified" and every such policy is forced
onto 3) *anyway*. In other words, one doesn't get two complementary hooks,
but rather, one real admission hook aka 3) plus a now-degraded /legacy/
hook 2) that can't answer the question operators actually want to ask.
Reusing security_bpf_prog() 4) for admission is no alternative either:
it fires only after the entire verifier (and JIT) pipeline ran, so
denying a not-yet-verified program at that point burns exactly the
work a denial is supposed to avoid, and by then the program has an id
assigned and the kallsyms/perf/audit load events fired. Policies are
free to also consume the signature verdict at 4), but admission control
belongs into security_bpf_prog_load(). Hence the latter remains the only
admission hook, merely moved past signature verification; with moving
large allocations further down into the BPF verifier, there is now only
minimal work between the old and new location: The preparation work in
bpf_check() is reordered such that only the minimally necessary setup
happens up front: Allocating the env, initializing the verifier log and
resolving the fd_array that a signed BPF metadata map needs. The worst
case allocation up until security_bpf_prog_load() is ~90K which is the
env itself (~54K) plus the continuous fd_array cache (at most 32K). The
insn_aux_data array is moved into a later stage in the verification.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/2f71d6c03698eb17d51f7247efde777627ee578a.camel@HansenPartnership.com [0]
Link: https://lore.kernel.org/lkml/ecf0521ed302db672672ebfbc670ecfba36a6e00.camel@HansenPartnership.com [1]
Link: https://lore.kernel.org/bpf/88703f00d5b7a779728451008626efa45e42db3d.camel@HansenPartnership.com [2]
Link: https://lore.kernel.org/bpf/DJOFY21DYUI4.19WKQ3NPZ4H5R@gmail.com [3]
Link: https://lore.kernel.org/bpf/20260708075343.358712-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The fd_array passed to BPF_PROG_LOAD carries the map and module BTF file
descriptors a program binds. The verifier reads it more than once during
a load: process_fd_array() walks it to bind the maps and BTFs, and
check_and_resolve_insns() and the kfunc BTF resolver later read it again
to resolve the program's BPF_PSEUDO_MAP_IDX* and module kfunc refs.
For signed BPF, we need these upfront in memory, thus resolve each fd to
its object once and cache it by fd_array index, then bind that cached
object for the rest of the load. env->fd_array becomes a small per-slot
{map, btf} cache rather than a bpfptr_t; every later reference is then
an in-bounds lookup of an already-resolved object, and an index outside
the cache is rejected instead of read from user memory:
- continuous (fd_array_cnt given): the caller declares the length and
every entry is resolved and bound up front (used also by the BPF
signed loader)
- sparse (no fd_array_cnt): left as the legacy path with no fd_array
cache; each reference reads its fd from the caller's fd_array and
resolves it on the spot. Deduplication in used_maps and the kfunc BTF
table keeps this correct, and only unsigned programs use this shape.
Split these into separate helpers to make it easier to follow.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
Link: https://lore.kernel.org/bpf/20260708075343.358712-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Currently each build with resolve_btfids enabled unnecessarily prints
the line 'INSTALL libsubcmd_headers' from libsubcmd.
Use the libcmd headers from source tree instead, without installation.
The same was done for objtool in commit ac99992677 ("objtool: Include
libsubcmd headers directly from source tree"), albeit for a different
reason.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Tested-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://patch.msgid.link/20260702-libsubcmd-spam-v1-1-300ec142a62f@linutronix.de
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add a TCP congestion-control struct_ops load test for a write through a
BTF pointer produced by bpf_rdonly_cast().
The test expects the verifier to reject the program before the TCP CA
btf_struct_access callback can whitelist the tcp_sock field write.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
check_ptr_to_btf_access() lets program-type btf_struct_access callbacks
validate writes before the default BTF access path rejects non-read
accesses. That bypasses the read-only policy for untrusted BTF pointers
created by helpers such as bpf_rdonly_cast().
Reject non-read accesses through PTR_UNTRUSTED BTF pointers at the
common entry point, before the callback branch to handle all cases.
Fixes: 282de143ea ("bpf: Introduce allocated objects support")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
When building bpf selftest with latest bpf-next, I got the following failure:
In file included from /home/yhs/work/bpf-next/tools/testing/selftests/bpf/libarena/selftests/test_parallel_bitmap.bpf.c:8:
/home/yhs/work/bpf-next/tools/testing/selftests/bpf/libarena/include/libarena/bitmap.h:11:8: error: redefinition of
'bitmap'
11 | struct bitmap {
| ^
/home/yhs/work/bpf-next/tools/testing/selftests/bpf/tools/include/vmlinux.h:51320:8: note: previous definition is here
51320 | struct bitmap {
| ^
The vmlinux.h struct bitmap comes from drivers/md/md-bitmap.c:
struct bitmap {
struct bitmap_counts { ... }
...
}
To fix the issue, I renamed libarena struct bitmap to arena_bitmap to avoid the conflict.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260707220136.910374-1-yonghong.song@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
