Augment selected Resource Lifetime Safety failures with structured diagnostics
while preserving the existing verifier messages.
Report unreleased references from check_reference_leak() using
reference-scoped diagnostic history, and add state reports for dynptr,
iterator, lock, and IRQ-flag lifetime misuse.
IRQ restore mismatch and out-of-order diagnostics use IRQ context-scoped
history when an IRQ-disabled region is active, so retained save/restore context
is still visible after per-state history removal.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-11-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Augment selected register-state verifier failures with Register Type Safety
reports. The existing verbose verifier messages remain in place; the new
reports add reason, source context, causal path, and suggestions.
Cover invalid pointer dereferences, unreadable registers, missing outgoing
stack arguments for bpf2bpf and kfunc calls, and rejected pointer arithmetic.
Use scoped diagnostic history so reports start from the latest relevant value
change and then show later branch outcomes.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-9-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Record verifier context transitions in the diagnostic history so later reports
can anchor causal paths to the critical section that made an operation invalid.
This covers lock, IRQ, RCU, and preempt regions without adding any new
verifier error reports. Category-specific commits decide where those recorded
events should be rendered.
Use context depth when selecting scoped history so nested regions anchor at the
outer active region, and fall back to the earliest retained event when the
matching entry was pruned.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-8-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add reference acquire and release events to diagnostic history so Resource
Lifetime Safety reports can show the lifetime of a specific reference id along
the path.
Record acquisitions after the verifier assigns the reference id. Record
releases only after release_reference_nomark() succeeds, including the
kptr_xchg RCU conversion path and owning-to-non-owning conversion path that
consume an owning reference.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-7-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Record material register and outgoing stack argument changes so diagnostics can
explain how a value reached its current type, bounds, or unreadable state.
Store old and new register types, scalar ranges, tnum value and mask, map and
BTF type identity, and basic operand metadata in the environment-owned
diagnostic event stream.
Record invalidations when packet data moves, references are released, or
borrowed references leave their protected region. Register-scoped history
starts at the latest matching modification and then shows later branch
outcomes.
Also record fixed stack spills and overwrites, and tag register fills from
stack so register-scoped history can follow value flow through spilled stack
slots.
The type_is_map_ptr() helper previously lived as a static function in
kernel/bpf/log.c since commit 0c95c9fdb6 ("bpf: emit map name in register
state if applicable and available"). Move it verbatim to
include/linux/bpf_verifier.h as a static inline, next to the other type
classifiers, so diagnostics.c can reuse it without duplicating the case list.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-6-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Save the diagnostic event-log position with each verifier stack entry and
reset the environment-owned stream together with the normal verifier log
when a queued state is popped. Also reset the diagnostic stream after
successful subprogram verification even when level-2 logging preserves the
normal verifier log.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-5-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add an environment-owned diagnostic history for verifier reports. Event
payloads keep the user-facing branch history shape, while storage lives
in bpf_verifier_env and follows the active verifier path.
Grow the event array geometrically up to a 64 MiB limit. Once storage
reaches the limit, or an allocation fails, overwrite the oldest event so
diagnostics retain the newest useful suffix without adding per-event
metadata.
Represent saved positions as absolute logical sequence numbers. A restore
truncates to a retained position. If its prefix has already been evicted,
clear the abandoned suffix and preserve the missing-history position. This
keeps marks stable across rotation without increasing their size.
Add the branch event renderer and branch recording.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-4-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Teach verifier diagnostics to annotate an instruction with BTF source
line information and nearby BPF instructions. The renderer keeps source
text in a fixed-width lane and prints instructions in a stable right-hand
gutter.
Wrap annotation text under the source line so long error labels remain
readable while the source and instruction lanes keep their fixed layout.
Keeping source and instruction context in one commit preserves the visual
layout contract that later diagnostic reports rely on.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-3-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add the initial diagnostics renderer for verifier reports and wire it into
the BPF build. The helper emits the common failure header through the
verifier log.
Later patches add prose wrapping, reusable report sections, and source and
instruction context for category-specific diagnostics.
Gate the helpers on normal verifier log output from the start, so
BPF_LOG_STATS-only loads do not collect or render diagnostics.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-2-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
In principle, the desired performance level can be set in MSR_HWP_REQUEST
to indicate to the processor what performance level the OS would like
the given CPU to run at, but if the Dynamic Efficiency Control (DEC)
feature is enabled in the processor, doing so may result in confusing
the processor firmware. It is then better to let the processor firmware
figure out the most suitable performance level by itself.
Accordingly, make intel_pstate always set the desired performance level
to zero (which means "no preference") when running on a platform with
DEC enabled.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
Link: https://patch.msgid.link/4758098.LvFx2qVVIh@rafael.j.wysocki
After previous changes, intel_pstate_hybrid_hwp_adjust() does not do
much and its name and kerneldoc comment (which is not really necessary
because the function is static) have become a bit confusing. Moreover,
the initialization of P-states on systems with HWP enabled is divided
between it and a direct conditional statement branch in
intel_pstate_get_cpu_pstates() which is not super-easy to follow.
Address this by introducing intel_pstate_get_hwp_pstates() for the
entire HWP-specific initialization of P-states and moving the code from
intel_pstate_hybrid_hwp_adjust() into it along with some HWP-related
code from intel_pstate_get_cpu_pstates().
No intentional functional impact.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Link: https://patch.msgid.link/6021518.DvuYhMxLoT@rafael.j.wysocki
rate_limit_us is an unsigned int, while NSEC_PER_USEC is defined as
1000L. On 32-bit systems, the multiplication is therefore performed
using 32-bit unsigned arithmetic before the result is assigned to
freq_update_delay_ns.
For example, writing 4294968 to rate_limit_us wraps the delay from
4294968000 ns to 704 ns. This makes schedutil update far more often
than configured.
Add sugov_update_rate_limit_us() to widen rate_limit_us to s64 before
converting it to nanoseconds. Use the helper when updating the tunable
through sysfs and when starting the governor, so both paths perform the
conversion without overflow.
Fixes: 9bdcb44e39 ("cpufreq: schedutil: New governor based on scheduler utilization data")
Signed-off-by: Hui Su <sh_def@163.com>
Reviewed-by: Zhongqiu Han <zhongqiu.han@oss.qualcomm.com>
Cc: All applicable <stable@vger.kernel.org>
Link: https://patch.msgid.link/20260806142304.1761454-1-sh_def@163.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
These are three last-minute fixes for the 7.2 release,
though nothing alarming:
- one error handling fix for optee firmware
- incorrect i2c data for the apple M3 that was added in 7.2
- a boot time warning fix for nvidia tegra
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAmp/kkEACgkQmmx57+YA
GNn0sRAAnDh3M8YkrmeGdU4aLCEIFDa+SB4+5OF2ANtjunGy3kUlPj1f9y9Sk7aq
CEA2foLgAOThE7anz3VrV5RC+OHBqfe3OD39kMqdlTJlqZNLpVEbqFsyoMr8owAw
zpfwDl8n1u9ROjGEyQ7t6QAE512ZLbwZ3wducoQPFGXZfjTpiuF4P00lpXa729xF
H+Pb0WOMz3zytIfGKvKx4RWkiH0/xatiLoy0AO+73b2lCQgkJF4bhdkM+G5XAIvG
XBhzO488Pu9vuoXpNnY0lizE2xZiJdhBJkR8/Ad0KrTlaLsjwJkF237ozWQ89F8U
/00wS8zE9FXsEQNw9uJj7EjW7nKItxTloq3MELjoXbGV7xW5sEsjWZM3xsvFX5rD
uJo4laNmjp7LZgTvlOZ9T5HM3XIMes3MNTqa4qx0bmrpxwy8BiaTuED9ugrUAIkv
xR2xejKQdFEIEfJRga9YSVbVhF1fDH0dFG+8BVchx6npFWwFRk/RI0JycFGqUul7
8CNXuweWNaVb0Q/zfwWz99pVNqdDySJpAMh4Pl4GKB++jYRvnHU4sCSqDYrkqWRo
UGMpZhuXHpwzVWu3B/gPdk2YW/Mo1jfGN6dPFP3JsfdQx1BaUx/2j4ggJn20nTUw
v7bBU2i0ljMvNkClCqY8sIfgWDxiI1vsFzabW5o2PRzm8R+jm94=
=xgY+
-----END PGP SIGNATURE-----
Merge tag 'soc-fixes-7.2-3' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
Pull SoC fixes from Arnd Bergmann:
"These are three last-minute fixes for the 7.2 release, though nothing
alarming:
- one error handling fix for optee firmware
- incorrect i2c data for the apple M3 that was added in 7.2
- a boot time warning fix for nvidia tegra"
* tag 'soc-fixes-7.2-3' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc:
arm64: tegra: Add EL2 virtual timer interrupt for Tegra194
arm64: dts: apple: t8122: Fix I2C resources
optee: ffa: Add NULL check in optee_ffa_lend_protmem
A bug fix found by researchers.
* Ensure to mask all privileged bits when restoring the supervisor
register from sigreturn.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE2cRzVK74bBA6Je/xw7McLV5mJ+QFAmqABmQACgkQw7McLV5m
J+TL9A/7BmXvg+A5KT0xKxYrC//OCvD5dGwwKVhc5tEtNhpcWZU4fQo9WioRUpcz
OfbBcKN4RGrpm9Do685mqc2+nOGFcK9SId6eN47ri2G1ZJ1+wXE1IVKLIddlpBzp
rBJJyStBAKG2ICQfgoKM26JGbRttH19hJWSDITLzA2vAMp2gxpV7I1tSiLMrbOym
e/fKuclQ2tlH3DX+CJrRia/azeKRMjhlTCsSCH+FQSG+oegtCd6PP+rWi+i7cIas
pdSrDes+ES6LcGncK+uaNQfIi4D9tm37POYOK1zq4cyQknl4yl0mCWzHDegHocM3
EMAeZXgIxWR2sDn/xGl42gXsEjsIxgwUn9eufvAaNNVomFKOE8cVTsTAI7sgOCsj
9eiLLTCAQdGt8wiKYZLfnSm/tkIrtUs+Uc0v1OWslWUN9pVOuhiE5RYVwt6oOgyf
/16K0OFVggElHSyCkk9WPWryER1fg2n/eaOHflyOCO7YaQj9LL2e6TkVae7jQ+re
vZ1ZgNuSO9ZchcemvtQ0gH28sceRy+R2ICQV7ytuR+7Isxk1kQwXuKFhjNkKcy35
pTVFaFY1d/PwrtgUglk45/9ITFLJZNiz4ToE8jrP8P9+DlQhdp7tmZ2gNmtPb3Ll
H0fOU2Bh4QeAQm4T0XDGlm1U2eGA5gy3HFUUNMXomLdNfpT2vIU=
=p9Ll
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of https://github.com/openrisc/linux
Pull OpenRISC fix from Stafford Horne:
"A bug fix found by researchers:
- mask all privileged bits when restoring the supervisor register
from sigreturn"
* tag 'for-linus' of https://github.com/openrisc/linux:
openrisc: signal: do not restore privileged SR bits on sigreturn
The sample parsing test only validates grouped read values when
PERF_FORMAT_LOST is present.
For PERF_FORMAT_GROUP without PERF_FORMAT_LOST, the contents of
read.group.values[] are not validated, allowing corruption of the parsed
value and id fields to go undetected.
The values are also handed to the synthesis as a plain array of struct
sample_read_value, which always has a 24-byte stride, while
read.group.values is expected to be packed according to read_format --
evsel__parse_sample() points it into the event data. Without
PERF_FORMAT_LOST the stride is 16, so both the synthesis and the
comparison walk overlapping bytes and the test passes regardless of the
contents.
Validate value and id for grouped reads and continue to validate lost
when PERF_FORMAT_LOST is present, walking the entries with
next_sample_read_value(). Also build the input packed using
sample_read_value_size() so the compared fields are the real ones.
Verified with a deliberate stride bug in copy_read_group_values(): the
test still passes without this change and fails at read_format 0xc with
it applied.
Signed-off-by: PVS Narasimha Rao <venkatasuryapala@gmail.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
dso__read_symbol() asserts that len <= jited_prog_len, where len comes
from sym->end - sym->start (parsed from PERF_RECORD_KSYMBOL in
perf.data). Both values originate from untrusted file input.
With NDEBUG (production builds), the assert is compiled out, allowing
an out-of-bounds heap read when the BPF program buffer is accessed.
Without NDEBUG, a crafted perf.data crashes perf with an assertion
failure.
Replace the assert with a runtime bounds check that returns NULL with
an appropriate error code, matching the existing error handling
pattern in this function.
Fixes: aa04707f50 ("perf dso: Support BPF programs in dso__read_symbol()")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Cc: Song Liu <song@kernel.org>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
dso_cache__memcpy() computes cache_offset = offset - cache->offset,
then cache_size = min(cache->size - cache_offset, size). The RB tree
lookup in __dso_cache__find() matches using the full
DSO__DATA_CACHE_SIZE window, but cache->size reflects the actual pread
return value from dso_cache__populate().
A short pread (e.g. near end-of-file) makes cache->size smaller than
DSO__DATA_CACHE_SIZE. If a subsequent access targets an offset past
cache->offset + cache->size but within the DSO__DATA_CACHE_SIZE
window, the cache entry is found but cache_offset exceeds cache->size.
Since both are u64, the subtraction cache->size - cache_offset wraps
to a large value, min() selects the caller's size, and memcpy reads
out of bounds.
Return 0 for an offset past the valid cached data. For a regular
file a short pread only happens at end-of-file, so 0 is what a direct
pread() at that offset would return: cached_io() stops its read loop
as on EOF. Re-reading from the backing file would not help — a
second pread at the same offset returns the same short count.
Fixes: 366df72657 ("perf dso: Refactor dso_cache__read()")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
file_read() and file_size() use ret = -errno when
dso__data(dso)->fd is negative after try_to_open_dso() fails. By this
point errno has been through mutex_lock(), nsinfo__mountns_enter(), and
multiple open() attempts inside try_to_open_dso() — it no longer
reflects the actual open failure. If errno happens to be 0, ret = 0
looks like EOF rather than an error, and file_size() callers like
dso__data_size() would then report a zero-sized file instead of
failing.
dso__data(dso)->fd is always negative on failure — -errno from
__open_dso() when no filename could be built (e.g. -EINVAL, -ENOENT),
or -1 when do_open() itself failed — and never 0, so use it directly
instead of reading the stale global errno.
No assert() or comment is needed after the assignment: the enclosing
if (dso__data(dso)->fd < 0) already guarantees ret < 0
[Namhyung Kim review].
Fixes: 33bdedcea2 ("perf tools: Protect dso cache fd with a mutex")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
dso__decompress_kmodule_path() unconditionally calls close(fd) on the
return value of decompress_kmodule(). When decompression fails or the
DSO is not compressed, decompress_kmodule() returns -1. close(-1)
fails with EBADF and clobbers errno, which callers up the chain
(dso__get_filename → __open_dso) depend on for error propagation.
Guard the close() call with fd >= 0 so only valid file descriptors are
closed.
Fixes: 42b3fa6708 ("perf tools: Introduce dso__decompress_kmodule_{fd,path}")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
__open_dso() computes fd = -errno when dso__get_filename() returns NULL.
Some failure paths in dso__get_filename() (e.g. binary type mismatch)
return NULL without making a syscall, leaving errno at 0 from a prior
successful call. fd = -0 = 0, which is stdin — subsequent code treats
it as a valid file descriptor.
Fall back to ENOENT when errno is 0, ensuring fd is always negative on
failure.
The forced ENOENT stays in errno for the callers that check it after a
negative fd. It must not misdirect the try_to_open_dso() fallback
loop, though: dso__get_filename()'s chroot fallback used to accept a
stale ENOENT even when stat() succeeded on a non-regular file (e.g. a
directory). Re-stat() there and only take the chroot path when
stat() actually failed with ENOENT [sashiko-bot review of PATCH 1/5].
Fixes: eba5102d2f ("perf tools: Add global list of opened dso objects")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
sched_class->balance() is gone from sched_ext and what balance_one() does is
run dispatch to produce something pickable. Update the balance-era names to
dispatch terms:
- balance_one() -> dispatch_one()
- SCX_RQ_IN_BALANCE -> SCX_RQ_IN_DISPATCH
No BPF scheduler reads the flag. The enum autogen headers gain the new name
with the old entry retained like other removed enumerators, zero-filling at
load time. No functional changes.
Signed-off-by: Tejun Heo <tj@kernel.org>
The fixup demoting a keep verdict when @prev is not on ext_sched_class
guarded against the rq-level SCX_RQ_BAL_KEEP flag going stale back when
balancing and picking were separate operations.
The verdict now travels in the return value, created and consumed in one
invocation against the @prev it evaluated, and every keep decision tests
SCX_TASK_QUEUED under the rq lock, which implies ext_sched_class as a class
switch dequeues first. Drop the fixup along with dispatch_core_pick()'s
copy.
Signed-off-by: Tejun Heo <tj@kernel.org>
kick_sync_wait_bal_cb() assumes it runs on the rq's CPU from the
__schedule() tail: the snapshots it compares against live in that CPU's
percpu area and the busy-wait runs with the rq lock dropped and IRQs
enabled.
However, dispatch can now drop the rq lock while the callback sits queued,
and rq lock takers in that window (the sched class change paths, the scx
task iterator) flush pending balance callbacks on release, running the
callback on a foreign CPU. Such a run compares against unrelated snapshots
and can deadlock when the executing CPU is itself a wait target.
Bail on a foreign CPU and leave the wait state alone. The wait only observes
progress that the resched kicks already guarantee and the rq's next wait
picks up the stale cpus_to_sync bits.
Fixes: 4c95380701 ("sched/ext: Fold balance_scx() into pick_task_scx()")
Cc: stable@vger.kernel.org # v6.19+
Signed-off-by: Tejun Heo <tj@kernel.org>
kernel/sched/ext/arena.c uses the gen_pool allocator, which is built only
when GENERIC_ALLOCATOR is set. SCHED_CLASS_EXT doesn't select it, so on
configs where nothing else does, the build fails to link:
build_policy.o: undefined reference to `gen_pool_create'
build_policy.o: undefined reference to `gen_pool_for_each_chunk'
build_policy.o: undefined reference to `gen_pool_destroy'
Fixes: 9eca087deb ("sched_ext: Sub-allocator over kernel-claimed BPF arena pages")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202608151315.tvN3X0Oq-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202608151632.3p91bTQj-lkp@intel.com/
Signed-off-by: Tejun Heo <tj@kernel.org>
fcg_dispatch() true-ups the current cgroup's cvtime when its slice
expires or its DSQ runs empty while the slice is still active:
__sync_fetch_and_add(&cgc->cvtime_delta,
(cpuc->cur_at + cgrp_slice_ns - now) *
FCG_HWEIGHT_ONE / (cgc->hweight ?: 1));
The true-up should be actual minus charged: on CNS_EXPIRE, the overrun
(now - cur_at - cgrp_slice_ns) should be added; on CNS_EMPTY, the
unused portion of the slice should be subtracted. The expression above
has the sign inverted, and in the CNS_EXPIRE case now is already past
cur_at + cgrp_slice_ns, so the u64 subtraction wraps. The
multiplication preserves the two's complement encoding but the
unsigned division by hweight destroys it, adding roughly 2^64/hweight
per expiry instead of a small correction.
Under saturation the hweight budget clamp in cgrp_cap_budget() masks
most of the garbage, so the weight distribution barely moves, but the
accounting is broken all the same. Compute the delta as a signed value
and use fetch_and_add()/fetch_and_sub() so that the dividends stay
positive, as BPF division is unsigned.
Instrumented the true-up and ran a saturated three-leaf cgroup tree on
a 4-CPU VM: without the fix, each expiry added ~5e15 (2^64/hweight
territory) to cvtime_delta; with it, the corrections are back to
slice scale, with the overrun added and the unused portion subtracted.
Fixes: a4103eacc2 ("sched_ext: Add a cgroup scheduler which uses flattened hierarchy")
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_bpf_dsq_reenq() queues a deferred reenq (dru) that runs from
run_deferred(), not ops.dispatch(). If the DSQ is destroyed before the dru
runs, process_deferred_reenq_users() sees dsq->id == SCX_DSQ_INVALID and
hits the BUG_ON. destroy_dsq() doesn't flush pending drus, so just skip.
tj: Read dsq->id once with READ_ONCE(). Reading it separately in the INVALID
check and the BUG_ON would leave a window where destroy_dsq() can
invalidate the id between the two reads and still trigger the BUG_ON.
Fixes: 84b1a0ea0b ("sched_ext: Implement scx_bpf_dsq_reenq() for user DSQs")
Cc: stable@vger.kernel.org # v7.1+
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
libbpf strips the last ___flavor suffix when resolving kfunc externs, so
the bare ___v2 declaration resolves to scx_bpf_dsq_move_to_local, whose
BTF proto lacks @enq_flags. The extern never matches, bpf_ksym_exists()
returns false on every kernel that has the ___v2 kfunc, and the macro
falls back to ___v1, silently dropping @enq_flags.
Add the trailing ___compat suffix used by the other versioned externs in
this file (scx_bpf_dsq_insert___v2, scx_bpf_reenqueue_local___v2).
Any caller passing non-zero @enq_flags through the compat macro silently
loses them.
Fixes: 860683763e ("sched_ext: Add enq_flags to scx_bpf_dsq_move_to_local()")
Cc: stable@vger.kernel.org # v7.1+
Assisted-by: Z.ai:glm-5.2
Signed-off-by: fangqiurong <fangqiurong@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_bpf_events() always reads the root scheduler's event counters, so a
sub-scheduler program querying its own events silently gets the root's
instead and has no BPF-visible way to read its own (the per-scheduler sysfs
"events" file is the only interface). Resolve the scheduler from the calling
program with scx_prog_sched(). Unassociated programs follow the usual
scx_prog_sched() resolution: the root scheduler under a pre-sub-attach
compat root and zeroed counters otherwise.
Also fix up the malformed comment into proper kerneldoc.
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_root_disable() invalidates each rq's clock before taking the rq lock.
scx_rq_clock_invalidate() is a plain read-modify-write of rq->scx.flags and
every other writer of the word runs under the rq lock, so the unlocked
update can race a concurrent flags update and lose one side's bits.
The invalidation doesn't matter in the first place. The cached clock is read
only by scx_bpf_now() from a loaded scheduler's BPF programs, nothing can
re-validate the clock while sched_ext is disabled as scx_rq_clock_update()
is gated on scx_enabled() too, and the usual rq lock cycles under the next
scheduler refresh or invalidate it before it's practically observable. Drop
the invalidation instead of fixing the locking.
v2: Description and comment updated - the invalidation is unnecessary rather
than subsumed by the rq lock cycle below.
Fixes: 3a9910b590 ("sched_ext: Implement scx_bpf_now()")
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Changwoo Min <changwoo@igalia.com>
Fix missing pci_disable_device() in probe and remove.
Use pcim_enable_device(), pcim_request_region(), devm_ioremap(),
devm_ioremap_uc() and devm_ioremap_wc() for the PCI path. Convert
aux_start to devm_request_mem_region(). Guard atyfb_remove() to
only unmap/release for non-PCI (Atari) devices. Keep iounmap for
sprite.addr outside the guard since it uses raw ioremap().
Signed-off-by: Shixiong Ou <oushixiong@kylinos.cn>
Signed-off-by: Helge Deller <deller@gmx.de>
Fix missing pci_disable_device() in probe and remove.
Use pcim_enable_device(), devm_request_mem_region(),
devm_ioremap() and devm_ioremap_wc() to replace manual resource
management. Remove all release_mem_region() and iounmap() calls.
Use devm_request_mem_region() instead of pcim_request_region()
because the requested sizes (16384 for MMIO, maxvram for FB) do
not match the full PCI BAR sizes.
Signed-off-by: Shixiong Ou <oushixiong@kylinos.cn>
Signed-off-by: Helge Deller <deller@gmx.de>
Fix missing pci_disable_device() in probe and remove.
Use pcim_enable_device(), pcim_request_all_regions(),
devm_ioremap() and devm_ioremap_wc() to replace manual resource
management. Remove all pci_release_regions() and iounmap() calls.
Merge failed_init label into failed_enable.
Signed-off-by: Shixiong Ou <oushixiong@kylinos.cn>
Signed-off-by: Helge Deller <deller@gmx.de>
Fix missing pci_disable_device() in probe and remove.
Use pcim_enable_device(), pcim_request_all_regions(),
devm_ioremap() and devm_ioremap_wc() to replace manual resource
management. Remove all pci_release_regions() and iounmap() calls.
Signed-off-by: Shixiong Ou <oushixiong@kylinos.cn>
Signed-off-by: Helge Deller <deller@gmx.de>
Fix missing pci_disable_device() in probe and remove.
Use pcim_enable_device(), pcim_request_region(), devm_ioremap()
and devm_ioremap_wc() to replace manual resource management.
Remove all release_mem_region() and iounmap() calls.
Signed-off-by: Shixiong Ou <oushixiong@kylinos.cn>
Signed-off-by: Helge Deller <deller@gmx.de>
Add LANDLOCK_RESTRICT_SELF_NO_NEW_PRIVS to the default flag setting.
Gate the flag on the ABI version, but do not expose any userspace
control over this flag as it has no practical effect on the resulting
sandbox.
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Link: https://patch.msgid.link/20260809154544.1253100-6-utilityemal77@gmail.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Document setting no_new_privs with ruleset enforcement, following the
same compatibility section style as previous ABI additions.
Include a section explaining the tradeoffs of setting no_new_privs
through any means for privileged users of Landlock.
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Link: https://patch.msgid.link/20260809154544.1253100-5-utilityemal77@gmail.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Check that a successful landlock_restrict_self(2) call with
LANDLOCK_RESTRICT_SELF_NO_NEW_PRIVS sets no_new_privs without a prior
prctl(2) call nor CAP_SYS_ADMIN, that a failed call from both an
invalid ruleset and hitting the layer maximum leaves the attribute
unchanged, and that LANDLOCK_RESTRICT_SELF_TSYNC extends it to sibling
threads. Also check that this flag requires a ruleset.
Turn the multi_threaded_success test into a multi_threaded fixture with
success, no_new_privs, and no_new_privs_max_layers variants to factor
out the threading code.
Finally, rename restrict_self_fd_logging_flags to
restrict_self_fd_flags, and restrict_self_logging_flags to
restrict_self_flags to indicate that non-logging flags are now tested.
Test coverage for security/landlock is 91.8% of 2373 lines according to
LLVM 22.
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Link: https://patch.msgid.link/20260809154544.1253100-4-utilityemal77@gmail.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Add a landlock_restrict_self(2) flag to set the no_new_privs attribute
of the calling thread only after enforcement of the ruleset:
no_new_privs is set if and only if the call succeeds. This removes the
need for a prior prctl(2) PR_SET_NO_NEW_PRIVS call and guarantees that
a failed enforcement leaves the attribute unchanged.
Because no_new_privs is set by the call itself, the no_new_privs /
CAP_SYS_ADMIN requirement of landlock_restrict_self(2) is fulfilled by
construction, and the related EPERM check is skipped.
Unlike LANDLOCK_RESTRICT_SELF_LOG_SUBDOMAINS_OFF, this flag always
requires a valid ruleset: with a ruleset_fd of -1, such a call would be
nothing more than a Landlock-flavored prctl(2) PR_SET_NO_NEW_PRIVS, and
there is no valid use case for setting no_new_privs (possibly with
LANDLOCK_RESTRICT_SELF_TSYNC) without also enforcing Landlock
restrictions. Rejecting these calls also keeps the option of giving
them a meaning later.
The attribute is only set past the last point of failure, just before
committing the new credentials. When combined with
LANDLOCK_RESTRICT_SELF_TSYNC, no_new_privs is set on the sibling threads
as well, in their commit phase, with the same ordering.
Bump the Landlock ABI version to 11, and include the minimal related
test changes to keep the tests bisectable.
Cc: Mickaël Salaün <mic@digikod.net>
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Link: https://patch.msgid.link/20260809212459.2427878-1-utilityemal77@gmail.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
landlock_restrict_self(2) currently checks the no_new_privs /
CAP_SYS_ADMIN requirement before validating the flags argument. An
unprivileged caller without no_new_privs thus gets EPERM even when the
passed flags are invalid, hiding the EINVAL error.
Move the no_new_privs / CAP_SYS_ADMIN check just after the flags check
so that malformed calls consistently error out with EINVAL whatever the
caller's privileges, the same way seccomp(2) validates its flags before
checking no_new_privs.
Update the restrict_self_checks_ordering test accordingly.
Cc: Mickaël Salaün <mic@digikod.net>
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Link: https://patch.msgid.link/20260809154544.1253100-2-utilityemal77@gmail.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
The documentation embeds the canonical erratum documentation from the
header file, which is already a self-contained description of the issue.
Signed-off-by: Günther Noack <gnoack@google.com>
Link: https://patch.msgid.link/20260813093157.1436894-7-gnoack@google.com
[mic: Update the documentation date]
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Even though OverlayFS uses vfs_rename() with RENAME_WHITEOUT on its
backing directories, and even though RENAME_WHITEOUT requires
LANDLOCK_ACCESS_FS_MAKE_REG, a process that renames non-regular files in
an OverlayFS can do so without having the LANDLOCK_ACCESS_FS_MAKE_REG
right in that location.
This works, and is supposed to work, because the changes to the backing
directories are done by OverlayFS, not by the originator task that did
the original rename() on the OverlayFS mount. Therefore, the changes
done to backing directories are not subject to the originator task's
credentials.
Test coverage for security/landlock is 91.8% of 2368 lines according to
LLVM 22.
Signed-off-by: Günther Noack <gnoack@google.com>
Link: https://patch.msgid.link/20260813093157.1436894-6-gnoack@google.com
[mic: Add test coverage, reflow commit message]
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Add audit_layout1.make_whiteout: This test looks similar to
audit_layout1.make_char, but creates a whiteout object through mknod().
Since whiteout object creation is now guarded with
LANDLOCK_ACCESS_FS_MAKE_REG rather than LANDLOCK_ACCESS_FS_MAKE_CHAR, it
also needs to log the matching denial to audit.
Signed-off-by: Günther Noack <gnoack@google.com>
Link: https://patch.msgid.link/20260813093157.1436894-5-gnoack@google.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Add tests to check that whiteout object creation is guarded by
LANDLOCK_ACCESS_FS_MAKE_REG, in the cases where these are created from
userspace:
* Conventional creation with mknod()
* Linking or renaming an existing whiteout object
* renameat2() with RENAME_WHITEOUT,
which creates a new whiteout object in the source location
* renameat2() with RENAME_EXCHANGE,
with one of the renamed objects being a whiteout object
Signed-off-by: Günther Noack <gnoack@google.com>
Link: https://patch.msgid.link/20260813093157.1436894-4-gnoack@google.com
[mic: Update commit message as requested]
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Whiteout objects are used in the upper layer of an OverlayFS to
indicate that the file with this name does not exist in the unified
view, even if it is present in one of the lower layer file systems.
For the userspace implementations of OverlayFS (fuse-overlayfs),
whiteout objects can be created from userspace as well:
* mknod(2) with S_IFCHR and makedev(0, 0)
* renameat2(2) with RENAME_WHITEOUT,
creating the whiteout in the old place of the moved file.
This commit guards whiteout creation in both of these cases with
LANDLOCK_ACCESS_FS_MAKE_REG. Whiteout objects are *not* considered
character devices and are not bound to a driver.
LANDLOCK_ACCESS_FS_MAKE_REG describes the same permission class as a
whiteout object: creating one is the only S_IFCHR creation that the VFS
exempts from CAP_MKNOD, so it is as unprivileged as creating a regular
file, while LANDLOCK_ACCESS_FS_MAKE_CHAR and
LANDLOCK_ACCESS_FS_MAKE_BLOCK keep meaning the creation of devices that
expose a kernel interface [1].
For the mknod(2) case, introduce a Landlock erratum. The creation of
whiteout objects through mknod(2) was previously guarded using
LANDLOCK_ACCESS_FS_MAKE_CHAR, and it is now guarded using
LANDLOCK_ACCESS_FS_MAKE_REG.
For the renameat2(2) case, fix a bug: Before this commit, renameat2(2)
with RENAME_WHITEOUT would create a directory entry even when all
LANDLOCK_ACCESS_FS_MAKE_* rights were denied.
This does not affect normal renames within layered OverlayFS mounts:
When doing a regular rename() on a mounted fuse-overlayfs, it is the
fuse-overlayfs daemon that exercises renameat2() with RENAME_WHITEOUT,
and only the Landlock domain of that daemon is checked there.
Suggested-by: Christian Brauner <brauner@kernel.org>
Suggested-by: Mickaël Salaün <mic@digikod.net>
Cc: stable@vger.kernel.org
Fixes: cb2c7d1a17 ("landlock: Support filesystem access-control")
Depends-on: 49c9e09d96 ("landlock: Fix handling of disconnected directories")
Depends-on: fe72ce6710 ("landlock: Add errata documentation section")
Signed-off-by: Günther Noack <gnoack@google.com>
Link: https://patch.msgid.link/20260720.chow9ohYie5b@digikod.net [1]
Link: https://patch.msgid.link/20260813093157.1436894-3-gnoack@google.com
[mic: Record why LANDLOCK_ACCESS_FS_MAKE_REG is the matching right, and
add link(2) to the user doc]
Signed-off-by: Mickaël Salaün <mic@digikod.net>
By passing a (0, 0) device number, the audit test for
LANDLOCK_ACCESS_FS_MAKE_CHAR was accidentally creating a whiteout object
rather than a char device. In preparation to treating whiteout objects
differently, use an actual character device instead.
Signed-off-by: Günther Noack <gnoack@google.com>
Link: https://patch.msgid.link/20260813093157.1436894-2-gnoack@google.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Vineet Gupta says:
====================
bpf, x86: fix per-CPU address resolution into an extended register
The JIT resolves a per-CPU address with
add <dst>, gs:[this_cpu_off]
but builds the REX prefix with add_1mod(), which sets REX.B. The
destination is encoded in ModRM.reg, which REX.R extends, and the memory
operand is disp32 with no base, so REX.B does nothing and the high
register bit is dropped. Every extended destination therefore resolves
into whichever register shares the low three bits:
R5 -> RAX R7 -> RBP R8 -> RSI R9 -> RDI
The address is left unadjusted and an unrelated register is clobbered.
Patch 1 switches to add_2mod() so the bit goes through REX.R.
Clang reloads the address into R1 before each per-CPU access, so the
destination is never an extended register and the bug has been dormant
since v6.10. GCC keeps several per-CPU addresses live at once, which is
how it turned up: test_progs-bpf_gcc panics the kernel in
global_percpu_data/init, with the address of a .percpu variable in R5.
Patch 2 covers every register. A functional test only catches this if
the address happens to land in an extended register, so the test matches
the JITed add instead.
Changes in v3:
- Fold the five per-register programs into one that loads every
register, and drop the comment explaining the register choice
(Eduard Zingerman).
- Move the percpu_data declaration inside the arch guard, so other
targets no longer carry a .percpu section and an unused map (bpf-ci).
- Match the movabsq of each address as well as the add, so the matchers
stay on consecutive lines and the pair is checked to use the same
register.
- Restore the Reviewed-by on patch 1, dropped by mistake in v2.
Changes in v2:
- Add the selftest, patch 2/2 (Eduard Zingerman). It uses __jited()
rather than __xlated(): the xlated stream is identical for every
register, and the wrong prefix is only visible in the native encoding.
- No functional change to patch 1.
====================
Link: https://patch.msgid.link/20260814220254.3797467-1-vineet.gupta@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
An ld_imm64 of a per-CPU map value is followed by a mov_percpu_addr that
reuses the same register, so which register the address lands in decides
how the JIT encodes the add. Getting the REX prefix wrong there is
invisible to a functional test unless the address happens to land in an
extended register, which is why this went unnoticed.
Load a .percpu variable into every register in one program and match the
JITed add against the register each one must resolve into.
Signed-off-by: Vineet Gupta <vineet.gupta@linux.dev>
Link: https://patch.msgid.link/20260814220254.3797467-3-vineet.gupta@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
The destination of the per-CPU address MOV is encoded in ModRM.reg,
which is extended by REX.R, but the REX prefix is built with
add_1mod(), which sets REX.B. REX.B extends ModRM.rm and SIB.base, and
this instruction addresses memory as disp32 with no base, so the bit
has no effect at all and the high register bit is simply lost.
Every is_ereg() destination therefore resolves to the wrong register,
picking whichever one shares the low three bits:
R5 -> RAX R7 -> RBP R8 -> RSI R9 -> RDI
With BPF_REG_5, whose reg2hex is 0, the emitted
65 49 03 04 25 <off> add %gs:<off>,%rax
adds the per-CPU offset to RAX rather than R8. The destination keeps
the unadjusted address and RAX is clobbered, so the program goes on to
dereference a pointer that was never made per-CPU:
BUG: unable to handle page fault for address: 0000607e386a8894
RIP: bpf_prog_707837aafd2aa9ae_update_percpu_data+0x93/0xc9
Call Trace:
__bpf_prog_test_run_raw_tp+0x2dc/0x7d0
__flush_smp_call_function_queue+0x1e9/0xc80
Kernel panic - not syncing: Fatal exception in interrupt
R5 is the mildest of the four, aliasing a scratch register and faulting
at the store. R7 aliases RBP and would corrupt the frame pointer, R8
and R9 alias the argument registers.
Use add_2mod() so the register goes through REX.R, matching how
add_2reg() places it in ModRM.reg and how emit_priv_frame_ptr()
hardcodes 0x4c for the same instruction with R9. Encodings for the
non-extended registers are unchanged.
Problem showed up when trying to resurrect BPF_GCC CI (selftests built
with BPF_GCC).
This has gone unnoticed because clang reloads the address into R1
before each per-CPU access, so the destination is never an extended
register. GCC keeps several per-CPU addresses live at once, and
test_progs-bpf_gcc panics the kernel in global_percpu_data/init, where
the address of a .percpu variable ends up in R5.
Fixes: 7bdbf74463 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Signed-off-by: Vineet Gupta <vineet.gupta@linux.dev>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260814220254.3797467-2-vineet.gupta@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
