bpf: Fix infinite loop in pcpu_freelist push with one possible CPU

Browse source

__pcpu_freelist_push() can loop forever when only one CPU is possible
and an NMI re-enters pcpu_freelist_push() while the interrupted context
holds that CPU's freelist lock.

After the current-CPU fast path fails, the fallback loop walks
cpu_possible_mask while skipping the current CPU. With CONFIG_SMP=n, or
when an SMP kernel is limited to one possible CPU with nr_cpus=1 or
possible_cpus=1, there are no other possible CPUs to examine. The loop
therefore makes no lock acquisition attempt and can never make progress.

The following stack was observed on a UP system:

  NMI context:
    pcpu_freelist_push
    free_htab_elem
    htab_map_delete_elem
    [perf-event BPF program]
    __perf_event_overflow
    perf_event_nmi_handler
    exc_nmi

  Interrupted context:
    __pcpu_freelist_push
    pcpu_freelist_push
    free_htab_elem
    htab_map_delete_elem
    [raw_tp/sys_enter BPF program]
    __bpf_trace_sys_enter
    do_syscall_64

raw_res_spin_lock() detects the same-CPU recursive acquisition and
returns -EDEADLK, but the subsequent fallback loop has no candidate head
on a system with one possible CPU.

Restore the extra fallback head that existed before the rqspinlock
conversion. Keep the current-CPU fast path, then try the other possible
CPUs and finally the extra head. The additional head lets a push, which
cannot fail without losing a preallocated element, make progress when the
only per-CPU head is held by the interrupted context.

Also check the extra head from the pop path so that nodes placed there
can be reused.

Fixes: f2ac0e5d1c ("bpf: Convert percpu_freelist.c to rqspinlock")
Signed-off-by: Hui Su <sh_def@163.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260806175600.1993595-1-sh_def@163.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>

...

This commit is contained in:

Hui Su 2026-08-07 01:56:00 +08:00 • committed by Kumar Kartikeya Dwivedi

parent 7ee2f20bf2

commit efebf64966

No known key found for this signature in database

GPG key ID: 472D377B63542F83

2 changed files with 28 additions and 8 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

35

kernel/bpf/percpu_freelist.c

View file

1

kernel/bpf/percpu_freelist.h

View file