Benefit from recently introduced instance iteration and convert
reporters .dumpit generic netlink callback to use it.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Benefit from recently introduced instance iteration and convert
linecards .dumpit generic netlink callback to use it.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
As long as the reporter life time is protected by devlink instance
lock, the reference counting is no longer needed. Remove it.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Remove port-specific health reporter destroy function as it is
currently the same as the instance one so no longer needed. Inline
__devlink_health_reporter_destroy() as it is no longer called from
multiple places.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Similar to other devlink objects, rely on devlink instance lock
and remove object specific reporters_lock.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Similar to other devlink objects, protect the reporters list
by devlink instance lock. Alongside add unlocked versions
of health reporter create/destroy functions and use them in drivers
on call paths where the instance lock is held.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The MLX5E_LOCKED_FLOW flag is not checked anywhere now so remove it
entirely.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The fact that devlink instance lock is held over mlx5 auxiliary devices
probe and remove routines brought a need to conditionally take devlink
instance lock there. The code is checking a MLX5E_LOCKED_FLOW flag
in mlx5 priv struct.
This is racy and may lead to access devlink objects without holding
instance lock or deadlock.
To avoid this, the only lock-wise sane solution is to make the
devlink entities created by the auxiliary device independent on
the original pci devlink instance. Create devlink instance for the
auxiliary device and put the uplink port instance there alongside with
the port health reporters.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
As long as the linecard life time is protected by devlink instance
lock, the reference counting is no longer needed. Remove it.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Similar to other devlink objects, convert the linecards list to be
protected by devlink instance lock. Alongside with that rename the
create/destroy() functions to devl_* to indicate the devlink instance
lock needs to be held while calling them.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
In the am65_cpsw_init_serdes_phy() function, the error handling for the
call to the devm_of_phy_get() function misses the case where the return
value of devm_of_phy_get() is ERR_PTR(-EPROBE_DEFER). Proceeding without
handling this case will result in a crash when the "phy" pointer with
this value is dereferenced by phy_init() in am65_cpsw_enable_phy().
Fix this by adding appropriate error handling code.
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Fixes: dab2b265dd ("net: ethernet: ti: am65-cpsw: Add support for SERDES configuration")
Suggested-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Siddharth Vadapalli <s-vadapalli@ti.com>
Reviewed-by: Roger Quadros <rogerq@kernel.org>
Link: https://lore.kernel.org/r/20230118112136.213061-1-s-vadapalli@ti.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We want to return negative error codes here but the copy_to/from_user()
functions return the number of bytes remaining to be copied.
Fixes: c59e12a140 ("net: dsa: microchip: ptp: Initial hardware time stamping support")
Signed-off-by: Dan Carpenter <error27@gmail.com>
Reviewed-by: Vladimir Oltean <olteanv@gmail.com>
Link: https://lore.kernel.org/r/Y8fJxSvbl7UNVHh/@kili
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Russell King says:
====================
net: sfp: cleanup i2c / dt / acpi / fwnode / includes
This series cleans up the DT/fwnode/ACPI code in the SFP cage driver:
1. Use the newly introduced i2c_get_adapter_by_fwnode(), which removes
the need to know about ACPI handles to find the I2C device.
2. Use device_get_match_data() to get the match data, rather than
having to look up the matching DT device_id to get at the data.
3. Rename gpio_of_names, as this is not DT specific.
4. Remove acpi.h include which is no longer necessary.
5. Remove ctype.h include which, as far as I can tell, was never
necessary.
====================
Link: https://lore.kernel.org/r/Y8fH+Vqx6huYQFDU@shell.armlinux.org.uk
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
An include of linux/ctype.h was added in commit 1323061a01
("net: phy: sfp: Add HWMON support for module sensors") but nothing
was used from this header file. Remove this unnecessary include.
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Nothing in the sfp code now references anything from the ACPI header,
everything is done via fwnode APIs, so get rid of this header.
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
There's nothing DT specific about the gpio_of_names array, let's drop
the _of infix.
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Rather than using of_match_node() to get the matching of_device_id
to then retrieve the match data, use device_get_match_data() instead
to avoid firmware specific functions, and free the driver from having
firmware specific code.
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Use the newly introduced i2c_get_adapter_by_fwnode() API, so that we
can retrieve the I2C adapter in a firmware independent manner once we
have the fwnode handle for the adapter.
Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When use tools/testing/selftests/kselftest_install.sh to make the
kselftest-list.txt under tools/testing/selftests/kselftest_install.
Then use tools/testing/selftests/kselftest_install/run_kselftest.sh to run
all the kselftests in kselftest-list.txt, it will be blocked by case
"filesystems/fat: run_fat_tests.sh" with "Warning: file run_fat_tests.sh
is not executable", so grant executable permission to run_fat_tests.sh to
fix this issue.
Link: https://lkml.kernel.org/r/dfdbba6df8a1ab34bb1e81cd8bd7ca3f9ed5c369.1673424747.git.pengfei.xu@intel.com
Fixes: dd7c9be330 ("selftests/filesystems: add a vfat RENAME_EXCHANGE test")
Signed-off-by: Pengfei Xu <pengfei.xu@intel.com>
Reviewed-by: Javier Martinez Canillas <javierm@redhat.com>
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kconfig fix for panfrost and two fixes for i915 and fb-helper to
address some bugs with vga-switcheroo.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRcEzekXsqa64kGDp7j7w1vZxhRxQUCY8j9NQAKCRDj7w1vZxhR
xYDhAP4mPeqkQWEcKD0KOxd+vG3khWTcB8BrHS8gJfxZgDVUxQEA5hsRluQ9MsNV
o1Q/qYvpAJzNYKJiPkff/SCOuy8JpQE=
=1LL3
-----END PGP SIGNATURE-----
Merge tag 'drm-misc-fixes-2023-01-19' of git://anongit.freedesktop.org/drm/drm-misc into drm-fixes
A fix for vc4 to address a memory leak when allocating a buffer, a
Kconfig fix for panfrost and two fixes for i915 and fb-helper to
address some bugs with vga-switcheroo.
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Maxime Ripard <maxime@cerno.tech>
Link: https://patchwork.freedesktop.org/patch/msgid/20230119082059.h32bs7zqoxmjbcvn@houat
Jiri Olsa says:
====================
hi,
sending new version of [1] patchset posted originally by Zhen Lei.
It contains 2 changes that improove search performance for livepatch
and bpf.
v3 changes:
- fixed off by 1 issue, simplified condition, added acks [Song]
- added module attach as subtest [Andrii]
v2 changes:
- reworked the bpf change and meassured the performance
- adding new selftest to benchmark kprobe multi module attachment
- skipping patch 3 as requested by Zhen Lei
- added Reviewed-by for patch 1 [Petr Mladek]
thanks,
jirka
[1] https://lore.kernel.org/bpf/20221230112729.351-1-thunder.leizhen@huawei.com/
---
Jiri Olsa (2):
selftests/bpf: Add serial_test_kprobe_multi_bench_attach_kernel/module tests
bpf: Change modules resolving for kprobe multi link
====================
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
We currently use module_kallsyms_on_each_symbol that iterates all
modules/symbols and we try to lookup each such address in user
provided symbols/addresses to get list of used modules.
This fix instead only iterates provided kprobe addresses and calls
__module_address on each to get list of used modules. This turned
out to be simpler and also bit faster.
On my setup with workload (executed 10 times):
# test_progs -t kprobe_multi_bench_attach/modules
Current code:
Performance counter stats for './test.sh' (5 runs):
76,081,161,596 cycles:k ( +- 0.47% )
18.3867 +- 0.0992 seconds time elapsed ( +- 0.54% )
With the fix:
Performance counter stats for './test.sh' (5 runs):
74,079,889,063 cycles:k ( +- 0.04% )
17.8514 +- 0.0218 seconds time elapsed ( +- 0.12% )
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Reviewed-by: Zhen Lei <thunder.leizhen@huawei.com>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Link: https://lore.kernel.org/r/20230116101009.23694-4-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Currently we traverse all symbols of all modules to find the specified
function for the specified module. But in reality, we just need to find
the given module and then traverse all the symbols in it.
Let's add a new parameter 'const char *modname' to function
module_kallsyms_on_each_symbol(), then we can compare the module names
directly in this function and call hook 'fn' after matching. If 'modname'
is NULL, the symbols of all modules are still traversed for compatibility
with other usage cases.
Phase1: mod1-->mod2..(subsequent modules do not need to be compared)
|
Phase2: -->f1-->f2-->f3
Assuming that there are m modules, each module has n symbols on average,
then the time complexity is reduced from O(m * n) to O(m) + O(n).
Reviewed-by: Petr Mladek <pmladek@suse.com>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Miroslav Benes <mbenes@suse.cz>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Link: https://lore.kernel.org/r/20230116101009.23694-2-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Use the dir_put_page() helper in sysv_rename() instead of open-coding two
kunmap() + put_page().
Cc: Al Viro <viro@zeniv.linux.org.uk>
Suggested-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Fabio M. De Francesco <fmdefrancesco@gmail.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Change the signature of dir_get_page() in order to prepare this function
to the conversion to the use of kmap_local_page(). Change also those call
sites which are required to adjust to the new signature.
Cc: Ira Weiny <ira.weiny@intel.com>
Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Fabio M. De Francesco <fmdefrancesco@gmail.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Use the offset_in_page() helper because it is more suitable than doing
explicit subtractions between pointers to directory entries and kernel
virtual addresses of mapped pages.
Cc: Ira Weiny <ira.weiny@intel.com>
Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Fabio M. De Francesco <fmdefrancesco@gmail.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
This cleans up the ISA string handling to more closely match a version
of the ISA spec. This is visible in /proc/cpuinfo and the ordering
changes may break something in userspace, but these orderings have
changed before without issues so with any luck that's still the case.
This also adds documentation so userspace has a better idea of what is
intended when it comes to compatibility for /proc/cpuinfo, which should
help everyone as this will likely keep changing.
* b4-shazam-merge:
Documentation: riscv: add a section about ISA string ordering in /proc/cpuinfo
RISC-V: resort all extensions in consistent orders
RISC-V: clarify ISA string ordering rules in cpu.c
Link: https://lore.kernel.org/r/20221205144525.2148448-1-conor.dooley@microchip.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
We do not need to writeout modified directory blocks immediately when
modifying them while the page is locked. It is enough to do the flush
somewhat later which has the added benefit that inode times can be
flushed as well. It also allows us to stop depending on
write_one_page() function.
Ported from an ext2 patch by Jan Kara.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Since commit 80b6093b55 ("kbuild: add -Wundef to KBUILD_CPPFLAGS
for W=1 builds"), building with W=1 detects misuse of #if.
$ make W=1 ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- arch/riscv/kernel/
[snip]
AS arch/riscv/kernel/head.o
arch/riscv/kernel/head.S:329:5: warning: "CONFIG_RISCV_BOOT_SPINWAIT" is not defined, evaluates to 0 [-Wundef]
329 | #if CONFIG_RISCV_BOOT_SPINWAIT
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
CONFIG_RISCV_BOOT_SPINWAIT is a bool option. #ifdef should be used.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Fixes: 2ffc48fc70 ("RISC-V: Move spinwait booting method to its own config")
Link: https://lore.kernel.org/r/20230106161213.2374093-1-masahiroy@kernel.org
Cc: stable@vger.kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
I remember being told "Just ping me on IRC" about patches, but googling
at the time was not helpful. #riscv on libera is not linux specific,
but a bunch of contributors etc do hang out there.
Add a link to the maintainers entry to help others find it in the future!
Signed-off-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://lore.kernel.org/r/20230106125344.1685266-1-conor@kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
On the non-assembler-side wrapping alternative-macros inside other macros
to prevent duplication of code works, as the end result will just be a
string that gets fed to the asm instruction.
In real assembler code, wrapping .macro blocks inside other .macro blocks
brings more restrictions on usage it seems and the optimization done by
commit 2ba8c7dc71 ("riscv: Don't duplicate __ALTERNATIVE_CFG in __ALTERNATIVE_CFG_2")
results in a compile error like:
../arch/riscv/lib/strcmp.S: Assembler messages:
../arch/riscv/lib/strcmp.S:15: Error: too many positional arguments
../arch/riscv/lib/strcmp.S:15: Error: backward ref to unknown label "886:"
../arch/riscv/lib/strcmp.S:15: Error: backward ref to unknown label "887:"
../arch/riscv/lib/strcmp.S:15: Error: backward ref to unknown label "886:"
../arch/riscv/lib/strcmp.S:15: Error: backward ref to unknown label "887:"
../arch/riscv/lib/strcmp.S:15: Error: backward ref to unknown label "886:"
../arch/riscv/lib/strcmp.S:15: Error: attempt to move .org backwards
Wrapping the variables containing assembler code in quotes solves this issue,
compilation and the code in question still works and objdump also shows sane
decompiled results of the affected code.
Fixes: 2ba8c7dc71 ("riscv: Don't duplicate __ALTERNATIVE_CFG in __ALTERNATIVE_CFG_2")
Signed-off-by: Heiko Stuebner <heiko.stuebner@vrull.eu>
Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Link: https://lore.kernel.org/r/20230105192610.1940841-1-heiko@sntech.de
Cc: stable@vger.kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
We do not need to writeout modified directory blocks immediately when
modifying them while the page is locked. It is enough to do the flush
somewhat later which has the added benefit that inode times can be
flushed as well. It also allows us to stop depending on
write_one_page() function.
Ported from an ext2 patch by Jan Kara.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
If minix_prepare_chunk fails, updating c/mtime and marking the
dir inode dirty is wrong, as the inode hasn't been modified. Also
propagate the error to the caller.
Note that this moves the dir_put_page call later, but that matches
other uses of this helper in the directory code.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
If minix_prepare_chunk fails, updating c/mtime and marking the
dir inode dirty is wrong, as the inode hasn't been modified.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Instead of consuming the page reference and kmap in the low-level
minix_delete_entry and minix_set_link helpers, do it in the callers
where that code can be shared with the error cleanup path.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Instrument nmi_trigger_cpumask_backtrace() to dump out diagnostics based
on evidence accumulated by exc_nmi(). These diagnostics are dumped for
CPUs that ignored an NMI backtrace request for more than 10 seconds.
[ paulmck: Apply Ingo Molnar feedback. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: <x86@kernel.org>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
CPUs ignoring NMIs is often a sign of those CPUs going bad, but there
are quite a few other reasons why a CPU might ignore NMIs. Therefore,
accumulate evidence within exc_nmi() as to what might be preventing a
given CPU from responding to an NMI.
[ paulmck: Apply Peter Zijlstra feedback. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: <x86@kernel.org>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
- Prevent reading into undefined memory in the expression lexer,
accounting for a trailer backslash followed by the null byte.
- Fix file mode when copying files to the build id cache, the problem
happens when the cache directory is in a different file system than the
file being cached, otherwise the mode was preserved as only a hard link
would be done to save space.
- Fix a related build-id 'perf test' entry that checked that permission
when caching PE (Portable Executable) files, used when profiling
Windows executables under wine.
- Sync the tools/ copies of kvm headers, build_bug.h, socket.h and
arm64's cputype.h with the kernel sources.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQR2GiIUctdOfX2qHhGyPKLppCJ+JwUCY8l5UgAKCRCyPKLppCJ+
J5eQAP4o2h70ULZ6m7/AFDCG5GW/wAIKgsBmEwPQQcKNWY19QQD6AsSszQ8pu4Ct
q65WT0qvpaSFA1KdrAWJenaVrkJCrwk=
=MUmJ
-----END PGP SIGNATURE-----
Merge tag 'perf-tools-fixes-for-v6.2-3-2023-01-19' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux
Pull perf tools fixes from Arnaldo Carvalho de Melo:
- Prevent reading into undefined memory in the expression lexer,
accounting for a trailer backslash followed by the null byte.
- Fix file mode when copying files to the build id cache, the problem
happens when the cache directory is in a different file system than
the file being cached, otherwise the mode was preserved as only a
hard link would be done to save space.
- Fix a related build-id 'perf test' entry that checked that permission
when caching PE (Portable Executable) files, used when profiling
Windows executables under wine.
- Sync the tools/ copies of kvm headers, build_bug.h, socket.h and
arm64's cputype.h with the kernel sources.
* tag 'perf-tools-fixes-for-v6.2-3-2023-01-19' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux:
perf test build-id: Fix test check for PE file
perf buildid-cache: Fix the file mode with copyfile() while adding file to build-id cache
perf expr: Prevent normalize() from reading into undefined memory in the expression lexer
tools headers: Syncronize linux/build_bug.h with the kernel sources
perf beauty: Update copy of linux/socket.h with the kernel sources
tools headers arm64: Sync arm64's cputype.h with the kernel sources
tools kvm headers arm64: Update KVM header from the kernel sources
tools headers UAPI: Sync x86's asm/kvm.h with the kernel sources
tools headers UAPI: Sync linux/kvm.h with the kernel sources
Eduard Zingerman says:
====================
Struct bpf_reg_state is copied directly in several places including:
- check_stack_write_fixed_off() (via save_register_state());
- check_stack_read_fixed_off();
- find_equal_scalars().
However, a literal copy of this struct also copies the following fields:
struct bpf_reg_state {
...
struct bpf_reg_state *parent;
...
enum bpf_reg_liveness live;
...
};
This breaks register parentage chain and liveness marking logic.
The commit message for the first patch has a detailed example.
This patch-set replaces direct copies with a call to a function
copy_register_state(dst,src), which preserves 'parent' and 'live'
fields of the 'dst'.
The fix comes with a significant verifier runtime penalty for some
selftest binaries listed in tools/testing/selftests/bpf/veristat.cfg
and cilium BPF binaries (see [1]):
$ ./veristat -e file,prog,states -C -f 'states_diff>10' master-baseline.log current.log
File Program States (A) States (B) States (DIFF)
-------------------------- -------------------------------- ---------- ---------- ---------------
bpf_host.o tail_handle_ipv4_from_host 231 299 +68 (+29.44%)
bpf_host.o tail_handle_nat_fwd_ipv4 1088 1320 +232 (+21.32%)
bpf_host.o tail_handle_nat_fwd_ipv6 716 729 +13 (+1.82%)
bpf_host.o tail_nodeport_nat_ingress_ipv4 281 314 +33 (+11.74%)
bpf_host.o tail_nodeport_nat_ingress_ipv6 245 256 +11 (+4.49%)
bpf_lxc.o tail_handle_nat_fwd_ipv4 1088 1320 +232 (+21.32%)
bpf_lxc.o tail_handle_nat_fwd_ipv6 716 729 +13 (+1.82%)
bpf_lxc.o tail_ipv4_ct_egress 239 262 +23 (+9.62%)
bpf_lxc.o tail_ipv4_ct_ingress 239 262 +23 (+9.62%)
bpf_lxc.o tail_ipv4_ct_ingress_policy_only 239 262 +23 (+9.62%)
bpf_lxc.o tail_ipv6_ct_egress 181 195 +14 (+7.73%)
bpf_lxc.o tail_ipv6_ct_ingress 181 195 +14 (+7.73%)
bpf_lxc.o tail_ipv6_ct_ingress_policy_only 181 195 +14 (+7.73%)
bpf_lxc.o tail_nodeport_nat_ingress_ipv4 281 314 +33 (+11.74%)
bpf_lxc.o tail_nodeport_nat_ingress_ipv6 245 256 +11 (+4.49%)
bpf_overlay.o tail_handle_nat_fwd_ipv4 799 829 +30 (+3.75%)
bpf_overlay.o tail_nodeport_nat_ingress_ipv4 281 314 +33 (+11.74%)
bpf_overlay.o tail_nodeport_nat_ingress_ipv6 245 256 +11 (+4.49%)
bpf_sock.o cil_sock4_connect 47 70 +23 (+48.94%)
bpf_sock.o cil_sock4_sendmsg 45 68 +23 (+51.11%)
bpf_sock.o cil_sock6_post_bind 31 42 +11 (+35.48%)
bpf_xdp.o tail_lb_ipv4 4413 6457 +2044 (+46.32%)
bpf_xdp.o tail_lb_ipv6 6876 7249 +373 (+5.42%)
test_cls_redirect.bpf.o cls_redirect 4704 4799 +95 (+2.02%)
test_tcp_hdr_options.bpf.o estab 180 206 +26 (+14.44%)
xdp_synproxy_kern.bpf.o syncookie_tc 21059 21485 +426 (+2.02%)
xdp_synproxy_kern.bpf.o syncookie_xdp 21857 23122 +1265 (+5.79%)
-------------------------- -------------------------------- ---------- ---------- ---------------
I looked through verification log for bpf_xdp.o tail_lb_ipv4 program in
order to identify the reason for ~50% visited states increase.
The slowdown is triggered by a difference in handling of three stack slots:
fp-56, fp-72 and fp-80, with the main difference coming from fp-72.
In fact the following change removes all the difference:
@@ -3256,7 +3256,10 @@ static void save_register_state(struct bpf_func_state *state,
{
int i;
- copy_register_state(&state->stack[spi].spilled_ptr, reg);
+ if ((spi == 6 /*56*/ || spi == 8 /*72*/ || spi == 9 /*80*/) && size != BPF_REG_SIZE)
+ state->stack[spi].spilled_ptr = *reg;
+ else
+ copy_register_state(&state->stack[spi].spilled_ptr, reg);
For fp-56 I found the following pattern for divergences between
verification logs with and w/o this patch:
- At some point insn 1862 is reached and checkpoint is created;
- At some other point insn 1862 is reached again:
- with this patch:
- the current state is considered *not* equivalent to the old checkpoint;
- the reason for mismatch is the state of fp-56:
- current state: fp-56=????mmmm
- checkpoint: fp-56_rD=mmmmmmmm
- without this patch the current state is considered equivalent to the
checkpoint, the fp-56 is not present in the checkpoint.
Here is a fragment of the verification log for when the checkpoint in
question created at insn 1862:
checkpoint 1862: ... fp-56=mmmmmmmm ...
1862: ...
1863: ...
1864: (61) r1 = *(u32 *)(r0 +0)
1865: ...
1866: (63) *(u32 *)(r10 -56) = r1 ; R1_w=scalar(...) R10=fp0 fp-56=
1867: (bf) r2 = r10 ; R2_w=fp0 R10=fp0
1868: (07) r2 += -56 ; R2_w=fp-56
; return map_lookup_elem(&LB4_BACKEND_MAP_V2, &backend_id);
1869: (18) r1 = 0xffff888100286000 ; R1_w=map_ptr(off=0,ks=4,vs=8,imm=0)
1871: (85) call bpf_map_lookup_elem#1
- Without this patch:
- at insn 1864 r1 liveness is set to REG_LIVE_WRITTEN;
- at insn 1866 fp-56 liveness is set REG_LIVE_WRITTEN mark because
of the direct r1 copy in save_register_state();
- at insn 1871 REG_LIVE_READ is not propagated to fp-56 at
checkpoint 1862 because of the REG_LIVE_WRITTEN mark;
- eventually fp-56 is pruned from checkpoint at 1862 in
clean_func_state().
- With this patch:
- at insn 1864 r1 liveness is set to REG_LIVE_WRITTEN;
- at insn 1866 fp-56 liveness is *not* set to REG_LIVE_WRITTEN mark
because write size is not equal to BPF_REG_SIZE;
- at insn 1871 REG_LIVE_READ is propagated to fp-56 at checkpoint 1862.
Hence more states have to be visited by verifier with this patch compared
to current master.
Similar patterns could be found for both fp-72 and fp-80, although these
are harder to track trough the log because of a big number of insns between
slot write and bpf_map_lookup_elem() call triggering read mark, boils down
to the following C code:
struct ipv4_frag_id frag_id = {
.daddr = ip4->daddr,
.saddr = ip4->saddr,
.id = ip4->id,
.proto = ip4->protocol,
.pad = 0,
};
...
map_lookup_elem(..., &frag_id);
Where:
- .id is mapped to fp-72, write of size u16;
- .saddr is mapped to fp-80, write of size u32.
This patch-set is a continuation of discussion from [2].
Changes v1 -> v2 (no changes in the code itself):
- added analysis for the tail_lb_ipv4 verification slowdown;
- rebase against fresh master branch.
[1] git@github.com:anakryiko/cilium.git
[2] https://lore.kernel.org/bpf/517af2c57ee4b9ce2d96a8cf33f7295f2d2dfe13.camel@gmail.com/
====================
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
A testcase to check that verifier.c:copy_register_state() preserves
register parentage chain and livness information.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20230106142214.1040390-3-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Register range information is copied in several places. The intent is
to transfer range/id information from one register/stack spill to
another. Currently this is done using direct register assignment, e.g.:
static void find_equal_scalars(..., struct bpf_reg_state *known_reg)
{
...
struct bpf_reg_state *reg;
...
*reg = *known_reg;
...
}
However, such assignments also copy the following bpf_reg_state fields:
struct bpf_reg_state {
...
struct bpf_reg_state *parent;
...
enum bpf_reg_liveness live;
...
};
Copying of these fields is accidental and incorrect, as could be
demonstrated by the following example:
0: call ktime_get_ns()
1: r6 = r0
2: call ktime_get_ns()
3: r7 = r0
4: if r0 > r6 goto +1 ; r0 & r6 are unbound thus generated
; branch states are identical
5: *(u64 *)(r10 - 8) = 0xdeadbeef ; 64-bit write to fp[-8]
--- checkpoint ---
6: r1 = 42 ; r1 marked as written
7: *(u8 *)(r10 - 8) = r1 ; 8-bit write, fp[-8] parent & live
; overwritten
8: r2 = *(u64 *)(r10 - 8)
9: r0 = 0
10: exit
This example is unsafe because 64-bit write to fp[-8] at (5) is
conditional, thus not all bytes of fp[-8] are guaranteed to be set
when it is read at (8). However, currently the example passes
verification.
First, the execution path 1-10 is examined by verifier.
Suppose that a new checkpoint is created by is_state_visited() at (6).
After checkpoint creation:
- r1.parent points to checkpoint.r1,
- fp[-8].parent points to checkpoint.fp[-8].
At (6) the r1.live is set to REG_LIVE_WRITTEN.
At (7) the fp[-8].parent is set to r1.parent and fp[-8].live is set to
REG_LIVE_WRITTEN, because of the following code called in
check_stack_write_fixed_off():
static void save_register_state(struct bpf_func_state *state,
int spi, struct bpf_reg_state *reg,
int size)
{
...
state->stack[spi].spilled_ptr = *reg; // <--- parent & live copied
if (size == BPF_REG_SIZE)
state->stack[spi].spilled_ptr.live |= REG_LIVE_WRITTEN;
...
}
Note the intent to mark stack spill as written only if 8 bytes are
spilled to a slot, however this intent is spoiled by a 'live' field copy.
At (8) the checkpoint.fp[-8] should be marked as REG_LIVE_READ but
this does not happen:
- fp[-8] in a current state is already marked as REG_LIVE_WRITTEN;
- fp[-8].parent points to checkpoint.r1, parentage chain is used by
mark_reg_read() to mark checkpoint states.
At (10) the verification is finished for path 1-10 and jump 4-6 is
examined. The checkpoint.fp[-8] never gets REG_LIVE_READ mark and this
spill is pruned from the cached states by clean_live_states(). Hence
verifier state obtained via path 1-4,6 is deemed identical to one
obtained via path 1-6 and program marked as safe.
Note: the example should be executed with BPF_F_TEST_STATE_FREQ flag
set to force creation of intermediate verifier states.
This commit revisits the locations where bpf_reg_state instances are
copied and replaces the direct copies with a call to a function
copy_register_state(dst, src) that preserves 'parent' and 'live'
fields of the 'dst'.
Fixes: 679c782de1 ("bpf/verifier: per-register parent pointers")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20230106142214.1040390-2-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
For LoadPin to be used at all in a classic distro environment, it needs
to allow for switching filesystems (from the initramfs to the "real"
root filesystem). To allow for this, if the "enforce" mode is not set at
boot, reset the pinned filesystem tracking when the pinned filesystem
gets unmounted instead of invalidating further loads. Once enforcement
is set, it cannot be unset, and the pinning will stick.
This means that distros can build with CONFIG_SECURITY_LOADPIN=y, but with
CONFIG_SECURITY_LOADPIN_ENFORCE disabled, but after boot is running,
the system can enable enforcement:
$ sysctl -w kernel.loadpin.enforced=1
Cc: Paul Moore <paul@paul-moore.com>
Cc: James Morris <jmorris@namei.org>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: linux-security-module@vger.kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Serge Hallyn <serge@hallyn.com>
Link: https://lore.kernel.org/r/20221209195746.1366607-4-keescook@chromium.org
Refactor the pin reporting to be more cleanly outside the locking. It
was already, but moving it around helps clear the path for the root to
switch when not enforcing.
Cc: Paul Moore <paul@paul-moore.com>
Cc: James Morris <jmorris@namei.org>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: linux-security-module@vger.kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Serge Hallyn <serge@hallyn.com>
Link: https://lore.kernel.org/r/20221209195746.1366607-3-keescook@chromium.org
In preparation for shifting root mount when not enforcing, split sysctl
logic out into a separate helper, and unconditionally register the
sysctl, but only make it writable when the device is writable.
Cc: Paul Moore <paul@paul-moore.com>
Cc: James Morris <jmorris@namei.org>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: linux-security-module@vger.kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Serge Hallyn <serge@hallyn.com>
Link: https://lore.kernel.org/r/20221209195746.1366607-2-keescook@chromium.org
In preparation for allowing mounts to shift when not enforced, move
read-only checking into a separate helper.
Cc: Paul Moore <paul@paul-moore.com>
Cc: James Morris <jmorris@namei.org>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: linux-security-module@vger.kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Serge Hallyn <serge@hallyn.com>
Link: https://lore.kernel.org/r/20221209195746.1366607-1-keescook@chromium.org
