In policy_unpack.c:unpack_perms_table, the perms struct is allocated via
kcalloc, with the position being reset if the allocation fails. However,
the error path results in -EPROTO being retured instead of -ENOMEM. Fix
this to return the correct error code.
Reported-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Fixes: fd1b2b95a2 ("apparmor: add the ability for policy to specify a permission table")
Reviewed-by: Tyler Hicks <code@tyhicks.com>
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
If we are not in an atomic context in common_file_perm, then we don't have
to use the atomic versions, resulting in improved performance outside of
atomic contexts.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
with the previous changes to mmap the in_atomic flag is now always
false, so drop it.
Suggested-by: Tyler Hicks <code@tyhicks.com>
Reviewed-by: Georgia Garcia <georgia.garcia@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
The previous value of GFP_ATOMIC is an int and not a bool, potentially
resulting in UB when being assigned to a bool. In addition, the mmap hook
is called outside of locks (i.e. in a non-atomic context), so we can pass
a fixed constant value of false instead to common_mmap.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
This new field allows reliable identification of the binary that
triggered a denial since the existing field (comm) only gives the name of
the binary, not its path. Thus comm doesn't work for binaries outside of
$PATH or works unreliably when two binaries have the same name.
Additionally comm can be modified by a program, for example, comm="(tor)"
or comm=4143504920506F6C6C6572 (= ACPI Poller).
Signed-off-by: Maxime Bélair <maxime.belair@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
Add support for the per permission tag index for a given permission
set. This will be used by both meta-data tagging, to allow annotating
accept states with context and debug information. As well as by rule
tainting and triggers to specify the taint or trigger to be applied.
Since these are low frequency ancillary data items they are stored
in a tighter packed format to that allows for sharing and reuse of the
strings between permissions and accept states. Reducing the amount of
kernel memory use at the cost of having to go through a couple if
index based indirections.
The tags are just strings that has no meaning with out context. When
used as meta-data for auditing and debugging its entirely information
for userspace, but triggers, and tainting can be used to affect the
domain. However they all exist in the same packed data set and can
be shared between different uses.
Signed-off-by: John Johansen <john.johansen@canonical.com>
* Add a compatible to the reset-gpio driver, suppress the sysfs bind
attributes, and propagate GPIO API errors.
* Add support for the i.MX8ULP SIM LPAV reset controller.
* Add RZ/G3S USBPHY suspend/resume support.
* Enable reset-k230 by default on ARCH_CANAAN
* Add support for the SpacemiT K3 SoC reset controller.
* Merge the 'spacemit-clkrst-v6.20-3' tag, shared with the clk tree,
as a dependency for the SpacemiT changes.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQRRO6F6WdpH1R0vGibVhaclGDdiwAUCaXiRShcccC56YWJlbEBw
ZW5ndXRyb25peC5kZQAKCRDVhaclGDdiwDWkAP0foLlNQzqjNj7uT6ysYoHwbixE
+F9iaLCqJ9SAS5ajcwEAvJdJOJJCHGLR/4ImjH4AdCC1RqENUPYTqivvjZ3ozAU=
=Z0D4
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAml7J0sACgkQmmx57+YA
GNnDxhAAn5icxeT8YFUJjKnYCnNx3gOVP9sdSQRc+MOdTwSNen6LdVqaXhYJij2y
yqTk138Ulr3NUG84k3p75Rajz6TbMsVY1usuqa+vJqr9ruuPmAIx1xTEI/Hb6HVM
4k2NeSk2osYSx/o0xgOC3qPA8fGAw/sdduFkU69wrWPAfoxg5uK+OQyJ8VJRnhFQ
45WnIZ+Z8AtINhJiq88cowilf762PTEi2t6PRjsI/E5JJplQXYyi2Kpx48nYHmrp
Kfl0bmNIM7a7fVOAXd/fdQ7QYex/qhUZzZy1GZh7uo3VSVyjLJuN2ZHkxvmfjIjG
x2SyAhfuryvCWiN7GDJcreiPNJOOkBtzjvisCvBQE9OFYSsZd2YRhb2cluK7YUZt
mlSPxqqD0Z+Y4/03pXAdLmAiUYqq2ZZeLiO4u6ExwYrHu7lcJlhmSMXdcokiMVwo
1QRRaJiXc5qcr/s/xkKT91Kmy1ncwiulb0Py2WkUaGTPKjPCqvPgHQ2PDZ6za9Eo
ilsn6yyuBfn3Dlcat2o59frXDp5/GggHx0gcxq6CrEWUUwchUbvHy0IMfqUNdzo4
dTT2HKwse+GPDfK9Rm8O1UiRGaWMBSW68db2X4KNGeKj3PvakDyAz7CxdCa86E3g
HfFQDzTK8bzlSnXCvjH9/q99aqxwAxs171NkveE3nx+NPqxptXc=
=P4vq
-----END PGP SIGNATURE-----
Merge tag 'reset-for-v6.20' of https://git.pengutronix.de/git/pza/linux into soc/drivers
Reset controller updates for v6.20
* Add a compatible to the reset-gpio driver, suppress the sysfs bind
attributes, and propagate GPIO API errors.
* Add support for the i.MX8ULP SIM LPAV reset controller.
* Add RZ/G3S USBPHY suspend/resume support.
* Enable reset-k230 by default on ARCH_CANAAN
* Add support for the SpacemiT K3 SoC reset controller.
* Merge the 'spacemit-clkrst-v6.20-3' tag, shared with the clk tree,
as a dependency for the SpacemiT changes.
* tag 'reset-for-v6.20' of https://git.pengutronix.de/git/pza/linux:
reset: spacemit: Add SpacemiT K3 reset driver
reset: spacemit: Extract common K1 reset code
reset: Create subdirectory for SpacemiT drivers
dt-bindings: soc: spacemit: Add K3 reset support and IDs
reset: canaan: k230: drop OF dependency and enable by default
reset: rzg2l-usbphy-ctrl: Add suspend/resume support
reset: rzg2l-usbphy-ctrl: Propagate the return value of regmap_field_update_bits()
reset: gpio: check the return value of gpiod_set_value_cansleep()
reset: imx8mp-audiomix: Support i.MX8ULP SIM LPAV
reset: imx8mp-audiomix: Extend the driver usage
reset: imx8mp-audiomix: Switch to using regmap API
reset: imx8mp-audiomix: Drop unneeded macros
reset: gpio: suppress bind attributes in sysfs
clk: spacemit: k3: extract common header
reset: spacemit: fix auxiliary device id
clk: spacemit: prepare common ccu header
reset: gpio: add the "compatible" property
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Currently, dma_alloc_from_pool() unconditionally warns and dumps a stack
trace when an allocation fails, with the message "Failed to get suitable
pool".
This conflates two distinct failure modes:
1. Configuration error: No atomic pool is available for the requested
DMA mask (a fundamental system setup issue)
2. Resource Exhaustion: A suitable pool exists but is currently full (a
recoverable runtime state)
This lack of distinction prevents drivers from using __GFP_NOWARN to
suppress error messages during temporary pressure spikes, such as when
awaiting synchronous reclaim of descriptors.
Refactor the error handling to distinguish these cases:
- If no suitable pool is found, keep the unconditional WARN regarding
the missing pool.
- If a pool was found but is exhausted, respect __GFP_NOWARN and update
the warning message to explicitly state "DMA pool exhausted".
Fixes: 9420139f51 ("dma-pool: fix coherent pool allocations for IOMMU mappings")
Signed-off-by: Sai Sree Kartheek Adivi <s-adivi@ti.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Link: https://lore.kernel.org/r/20260128133554.3056582-1-s-adivi@ti.com
As discussed extensively in the changelog for the addition of this
syscall on x86 ("x86/shstk: Introduce map_shadow_stack syscall") the
existing mmap() and madvise() syscalls do not map entirely well onto the
security requirements for shadow stack memory since they lead to windows
where memory is allocated but not yet protected or stacks which are not
properly and safely initialised. Instead a new syscall map_shadow_stack()
has been defined which allocates and initialises a shadow stack page.
This patch implements this syscall for riscv. riscv doesn't require
tokens to be setup by kernel because user mode can do that by
itself. However to provide compatibility and portability with other
architectues, user mode can specify token set flag.
Signed-off-by: Deepak Gupta <debug@rivosinc.com>
Link: https://patch.msgid.link/20251112-v5_user_cfi_series-v23-10-b55691eacf4f@rivosinc.com
Link: https://lore.kernel.org/linux-riscv/aXfRPJvoSsOW8AwM@debug.ba.rivosinc.com/
[pjw@kernel.org: added allocate_shadow_stack() fix per Deepak; fixed bug found by sparse]
Signed-off-by: Paul Walmsley <pjw@kernel.org>
Kernel test robot has reported a regression in the patch "slab: refill
sheaves from all nodes". When taken in isolation like this, there is
indeed a tradeoff - we prefer to use remote objects prior to allocating
new local slabs. It is replicating a behavior that existed before
sheaves for replenishing cpu (partial) slabs - now called
get_from_any_partial() to allocate a single object.
So the possibility of allocating remote objects is intended even if
remote accesses are then slower. But the profiles in the report also
suggested a contention on the list_lock spinlock. And that's something
we can try to avoid without much tradeoff - if someone else has the
spin_lock, it's more likely they are allocating from the node than
freeing to it, so we can skip it even if it means allocating a new local
slab - contributing to that lock's contention isn't worth it. It should
not result in partial slabs accumulating on the remote node.
Thus add an allow_spin parameter to __refill_objects_node() and
get_partial_node_bulk() to make the attempts from __refill_objects_any()
use only a trylock.
Reported-by: kernel test robot <oliver.sang@intel.com>
Link: https://lore.kernel.org/oe-lkp/202601132136.77efd6d7-lkp@intel.com
Link: https://patch.msgid.link/20260129-b4-refill_any_trylock-v1-1-de7420b25840@suse.cz
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
This adds:
- A socinfo entry for the MT8371 Genio 520 SoC
- Support for the Dynamic Voltage and Frequency Scaling
Resource Controller (DVFSRC) version 4, found in the
new MediaTek Kompanio Ultra (MT8196) SoC
- Initial support for the CMDQ mailbox found in the MT8196.
- A memory leak fix in the MediaTek SVS driver's debug ops.
-----BEGIN PGP SIGNATURE-----
iLoEABYKAGIWIQQn3Xxr56ypAcSHzXSaNgTPrZeEeAUCaXNXhhsUgAAAAAAEAA5t
YW51MiwyLjUrMS4xMSwyLDIoHGFuZ2Vsb2dpb2FjY2hpbm8uZGVscmVnbm9AY29s
bGFib3JhLmNvbQAKCRCaNgTPrZeEeF2oAQDcnPsR1I6J92rmK75Dufy4ayA79Ko0
j2cYEOzAoO+mzgD/SDq0VGMpPhyatUv5Lj8md69gARhNlylJVDIpC0jBQAQ=
=v4Rp
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAml7I8IACgkQmmx57+YA
GNmr5g//fM9YAmf/oA9AT6Ta4bQWzzzp9dSIZ18I8v7G4RSpCK9uVK0mVhH4JceU
QXt63dTL+wAZOHjCIXgpQzcZ6tXdCaD4XZux87Ab+EfrxVPBQbOkwMAA4rkj8Vbd
LLJ36G2qOyBvpw8+whCmsQCcUn9UqWuDWkgnIRRt04tn/QOCi5XQcKIdvyzQdglB
TlbGQKxW4XQXzKr1pDrgSF9mie3jtw9Z7povlTiUIhnfqCIkoS5X2upcyWAtGpjr
ON2t5lIH3TvlrhanqREfHltRBB1GF4AvAadaj0q9dA9DoZLwTlr7SfoiSHiR8kf+
QongU9Y4ObeLriAsyDT5/SNZ11lSIL+f2jpJMBTzdQ9+NpalE9mgWHA7O76iR4vf
E3SdwDRSykSXiNg+wzi6GeccHYE1yrpRu05g/MSCuO3N7AMqIYHmvfbWg2sM0ruS
8J0OYdEbr+JmVScYQ7GPHOdcwgdv4mAn5ENjhwmNjWTYMdjPNi1PnQwrXUzo3jYV
s7s8+P8yg4aABdPuDH+vDDaVj5T/fHy+OedMkRtGBfHbInu2WG/g4x6kDutiCR1L
b7Zu72psQEDAtfRrRnQP0/P+8/M7HLV0lbMVWf0H78lM3dQKOHTi6uuyT55IWCTv
WIwITw4Lo9BzapGjBU7QU2YcEW+t6mJ126wo1UbcM7V4JPcID40=
=i7PU
-----END PGP SIGNATURE-----
Merge tag 'mtk-soc-for-v6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/mediatek/linux into soc/drivers
MediaTek soc driver updates
This adds:
- A socinfo entry for the MT8371 Genio 520 SoC
- Support for the Dynamic Voltage and Frequency Scaling
Resource Controller (DVFSRC) version 4, found in the
new MediaTek Kompanio Ultra (MT8196) SoC
- Initial support for the CMDQ mailbox found in the MT8196.
- A memory leak fix in the MediaTek SVS driver's debug ops.
* tag 'mtk-soc-for-v6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/mediatek/linux:
soc: mediatek: mtk-cmdq: Add mminfra_offset adjustment for DRAM addresses
soc: mediatek: mtk-cmdq: Extend cmdq_pkt_write API for SoCs without subsys ID
soc: mediatek: mtk-cmdq: Add pa_base parsing for hardware without subsys ID support
soc: mediatek: mtk-cmdq: Add cmdq_get_mbox_priv() in cmdq_pkt_create()
mailbox: mtk-cmdq: Add driver data to support for MT8196
mailbox: mtk-cmdq: Add mminfra_offset configuration for DRAM transaction
mailbox: mtk-cmdq: Add GCE hardware virtualization configuration
mailbox: mtk-cmdq: Add cmdq private data to cmdq_pkt for generating instruction
soc: mediatek: mtk-dvfsrc: Rework bandwidth calculations
soc: mediatek: mtk-dvfsrc: Get and Enable DVFSRC clock
soc: mediatek: mtk-dvfsrc: Add support for DVFSRCv4 and MT8196
soc: mediatek: mtk-dvfsrc: Write bandwidth to EMI DDR if present
soc: mediatek: mtk-dvfsrc: Add a new callback for calc_dram_bw
soc: mediatek: mtk-dvfsrc: Add and propagate DVFSRC bandwidth type
soc: mediatek: mtk-dvfsrc: Change error check for DVFSRCv4 START cmd
dt-bindings: soc: mediatek: dvfsrc: Document clock
soc: mediatek: mtk-socinfo: Add entry for MT8371AV/AZA Genio 520
soc: mediatek: svs: Fix memory leak in svs_enable_debug_write()
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Amir Goldstein <amir73il@gmail.com> says:
Following the syzbot ovl bug report and a fix by Qing Wang,
I decided to follow up with a small vfs cleanup of some
open coded version of checking "." and ".." name in readdir.
The fix patch is applied at the start of this cleanup series to allow
for easy backporting, but it is not an urgent fix so I don't think
there is a need to fast track it.
* patches from https://patch.msgid.link/20260128132406.23768-1-amir73il@gmail.com:
ovl: use name_is_dot* helpers in readdir code
fs: add helpers name_is_dot{,dot,_dotdot}
ovl: Fix uninit-value in ovl_fill_real
Link: https://patch.msgid.link/20260128132406.23768-1-amir73il@gmail.com
Signed-off-by: Christian Brauner <brauner@kernel.org>
Use the helpers in place of all the different open coded variants.
This makes the code more readable and robust.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Link: https://patch.msgid.link/20260128132406.23768-4-amir73il@gmail.com
Reviewed-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
- Add a poweroff function to the RTKit library which will be required
for the first USB4/Thunderbolt series I hope to submit next cycle.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQS3vz815OHsEaWy0u9EEX0kKnUe6QUCaXaPiQAKCRBEEX0kKnUe
6WxsAP4yy1tXANQgHcOP34gsbvp4LIpQw+OORSMxJ1rH9SerdgD+PlvJKmBXK185
gzxJInYEF+E5vL7DJ0eZWDHDNHrP+wU=
=iswd
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAml7I5EACgkQmmx57+YA
GNkigg/8CaV1afJ7IM9zAh/dvEvMwohOkNC6w+kC8A/LthG5WbS+CnDwq3EuRgSJ
/qvfueuc/+6WvQWzX+Uwmt1g6TR8ejZR/k5a9JjWvdXPLZ3Fle5RgEf6B9rrlp5w
4JhuZy+dIH5J8GEpylbHR/YDsA/dJ/iJ6qGnAEyGwVgZEGFeesUhGn80i0swCHjT
6Uw3g/mSKAnV9Mhr6yjV2TWrMl0iascGd6uRB/EqsYKYyqWgvmAUjnYfxKL1IlpB
ICzrVPLRUaGdvp+S0gjRIaowljme23YcKt1oTDLxiCnIeuD+DnUohC5bklWV5aVN
g1P0tx4nlmgXiLz2kYUgNaZbhazcck/vJYtoWUgXbsN9m2o+xb+AniznbFS8YEvc
SNM0fHsVvVlEC7TIO5YNOs+3khCVNufOpE01aVQP0qSGY9XXSP7yBRuuRYPFq0KC
VhZKIw6a+U5YG/s5qOei38JHKn0s1qI8GjIhcpcf89tBsjcP7d/NkApC1xKLlNIs
NFDothI4tCZEorWAedsWugpqamwlgbF9+GXtR6r5MnI+Vct3A/WM+bvXjmaIB9db
AwQnUEScQN1x0iogEfPccmT92sUImmfuDKRxiv0uFeByw3kq8ERA2ew2MBIg5lKa
2EKURjsd+zI8nbUtgiAR4OsMVeghCa6DQYPYTXSSn4ezjecn7SQ=
=PFGj
-----END PGP SIGNATURE-----
Merge tag 'apple-soc-drivers-6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/sven/linux into soc/drivers
Apple SoC driver updates for 6.20
- Add a poweroff function to the RTKit library which will be required
for the first USB4/Thunderbolt series I hope to submit next cycle.
* tag 'apple-soc-drivers-6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/sven/linux:
soc: apple: rtkit: Add function to poweroff
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Rename the helper is_dot_dotdot() into the name_ namespace
and add complementary helpers to check for dot and dotdot
names individually.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Link: https://patch.msgid.link/20260128132406.23768-3-amir73il@gmail.com
Reviewed-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Syzbot reported a KMSAN uninit-value issue in ovl_fill_real.
This iusse's call chain is:
__do_sys_getdents64()
-> iterate_dir()
...
-> ext4_readdir()
-> fscrypt_fname_alloc_buffer() // alloc
-> fscrypt_fname_disk_to_usr // write without tail '\0'
-> dir_emit()
-> ovl_fill_real() // read by strcmp()
The string is used to store the decrypted directory entry name for an
encrypted inode. As shown in the call chain, fscrypt_fname_disk_to_usr()
write it without null-terminate. However, ovl_fill_real() uses strcmp() to
compare the name against "..", which assumes a null-terminated string and
may trigger a KMSAN uninit-value warning when the buffer tail contains
uninit data.
Reported-by: syzbot+d130f98b2c265fae5297@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=d130f98b2c265fae5297
Fixes: 4edb83bb10 ("ovl: constant d_ino for non-merge dirs")
Signed-off-by: Qing Wang <wangqing7171@gmail.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Link: https://patch.msgid.link/20260128132406.23768-2-amir73il@gmail.com
Acked-by: Miklos Szeredi <mszeredi@redhat.com>
Reviewed-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
This update includes:
- drop the use of of_platform_default_populate() from the
machine specific code, as it is already handled by the DT core
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQTsZ8eserC1pmhwqDmejrg/N2X7/QUCaXSQZwAKCRCejrg/N2X7
/TjtAPsHUI7hPm8xBuoK0edLsbGCUIFIsIqQOJF+cmSMEVPR0QD9E5Y20x+C4yfL
vouBeu20N2MJj8iCU8RR/c9fqWyqZwQ=
=DFnK
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAml7IqgACgkQmmx57+YA
GNkvxQ/+Jadabu0F1An2VSkozPXOQTcK4GFHibHWbvc5Isu2lp8UQYz5SSgswZhb
JI1uU7+dAgnK/rEZcNBqahkn/u1oU1wQz7E7hEbvcF/+NwcHm3bkEHVKqi3oHlwC
x/8f6+Xrja6pbFHVrlMHIwK+dS6BBwsGfVTuYkjNGouDcvOd59BAFxQ6OF71W0S6
47f74CMWAixR3LH/pL2lHIVVgKjlWpQWYzBa2/2WeCN2ooAaP8ygpi9sLLTbwCAq
RNBlaOlo/wrJP4vk77iVVVXQyVI/+PDzHKaLnYIl4fRrxjUPh1aH33WJhN//0Ja2
x7w6llZBj/UembUZyxLLCbAdHszdS4OZ++3rsmLhTDHCm+P1Pz3HmRS5jPMa/K5C
GScJmk8bwHVhY3JTJpYLHZXrTKzZrZ6StP/pBlvKvepgzhSn2YTgTTDLT/54YpvL
RmilvlJHiiaRHD+vlulV2uYaOHh9y45n4xIrvsfBf6FXr5jGIa2kwaFBVVnx+3r0
JHtZBiiYDYqCwnSh8bUS0k5Ey5lwy+Xi/PwmyGYUwV/f/RxC4Gtb0bKAA2VKzx5i
SRMB9unnlOIPFwto1SuMjuB06Yh0kgNYzX30QL/7Y/h+tBIs8OeSYAKzazeZgYTE
fWI2x2tdrsFlcewzrRowUaz8KGcvAeMj7Or17yPcurMjN+6sKBo=
=LuV6
-----END PGP SIGNATURE-----
Merge tag 'at91-soc-6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/at91/linux into soc/arm
Microchip AT91 SoC updates for v6.20
This update includes:
- drop the use of of_platform_default_populate() from the
machine specific code, as it is already handled by the DT core
* tag 'at91-soc-6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/at91/linux:
ARM: at91: remove unnecessary of_platform_default_populate calls
ARM: at91: Move PM init functions to .init_late hook
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
The changes to the logic in the Qualcomm SMEM driver for separating
"failed to probe" from "not yet probed", did not change the
qcom_smem_is_available() function, with the result that clients sees
SMEM as always available. Clients might then proceed to interact with
SMEM in codepaths that aren't suited to cope with -EPROBE_DEFER.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEBd4DzF816k8JZtUlCx85Pw2ZrcUFAml6KNAACgkQCx85Pw2Z
rcVnUhAA4LPSzewPiIg+rwCG7r/G+BFYW14Y0w5O2GdSLGGaAPEl9I88X12CIc7Q
8Ckrgk8WxAwTMWipnCg6IyFQz7GtOZzwIKzpIXIpynAVV14XortHtn7nDHTO7Lj9
eBnYJxcgG8TbLBbfPNscpq0fgPeRH2NcpY2jK06cnmfHLb0xsxpvgwq+/xND2evg
l/OY0TOdyyWdpmVpRQrFLnBHWhwsPExT4nu32OGGxNTQiT9UwTZicBKBLjrVAvmf
0Dw5UXSRyTLJpfYa0CY+nW1oKqWa4ZfMsXo2GlCwjRKylYDMfpSr9fe/x8crGj/C
G1pWdH6sESHvdtFi5Vpv1xsftkw+AF4zoj0cugxaiLj/vQ2f6PXQaL5d9KRuoNJO
lbf+S8U+7RS9uCbQ/rVt6wNBIzpRrmYq/FFmELnhWPfaxoB++ZE/RCwVsUpddrDH
wqPFAKt765XYLHmcwq5o6ndimjnYDA81M6GeWcB2pIXrAFucWIMZR6MjpbhDmTwJ
E0PAG/ucZA6xTf3EnRqiiznOWajDqT1A6tty3pIQ3Pv0j+SQ7qzCKAHHpOzBswPl
c1XX3gAjQQQIMWwLHQiXctN/XZIVNdQiQ2A/Z21pue0BY8a0zTBfzA3XxN2AeGZG
IpCnvpepHPoRtcDcVlJT0gDdLat9VVjHQjBPMrHsB1903I2v9H0=
=cI29
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAml7IhYACgkQmmx57+YA
GNlMMhAAgrf94WF147q0rm6FufhrVpYYDMLqWPO8ObWYrcOQEfhHYgKU/XBz93Yb
8FRyj7lJs8F1lBIPVWEwHm8bRFoEM9dz7KnyhHnfOXptsQjlcudJbXXk0eDJ7Xqv
F+Vf6YExSYlD2pkgSxdXiTfk/Sa5tipW6REDzdtJt1QG3chA6ZgLW96q4l7LQx+x
qebESgivDyn4Nk9UXOtoP2w7tnznoJd32OI5K/MO9liFjRSdWnqTiro2ADPT32cZ
ZeqOHCCE1AB0GY3jx6/9OBeW7kJlZWOqCBsIh8rkTmoB0aJaOYzAAPI5ixtETLZq
fGpm1kDndEzXWa9xvLqboD+oWYxNB5s3J1rNV697ZP6LNLSQz3eYzPaoRUBm7tVo
0Mwv5Hlk8H1orXpXJLYnbwXjcVQ1bh55gkCWwq4gLwqo4dCRy7oOSTr2Kl/g1pBb
BU0YnOM+TZoKbfqKb+CwvHN7g4sfRzSftcaTEjkVoiaaDGBrRb+26j9khdxMPkvh
czWx96XogzB4XNhfUFLgevn+Ao7eCXzoibNnVfNkCgf1UYcmY6nlZXnAoL2BnAnN
aS6XpR6gjfceLqk4J8Wz+xU4EMl0kmw1dSpxN1qIJ2ppLExBydZhjeQaIY9qjRew
rjs0GfY8u9TjZjLpF5bWsV+7xVE8L4aWEUSt01rfcH0a0IIYJh4=
=Ynp6
-----END PGP SIGNATURE-----
Merge tag 'qcom-drivers-fixes-for-6.19' of https://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux into arm/fixes
Qualcomm driver fix for v6.19
The changes to the logic in the Qualcomm SMEM driver for separating
"failed to probe" from "not yet probed", did not change the
qcom_smem_is_available() function, with the result that clients sees
SMEM as always available. Clients might then proceed to interact with
SMEM in codepaths that aren't suited to cope with -EPROBE_DEFER.
* tag 'qcom-drivers-fixes-for-6.19' of https://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux:
soc: qcom: smem: fix qcom_smem_is_available and check if __smem is valid
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Add myself as maintainer of HiSilicon SoC HHA driver as I will be
responsible for this driver.
Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
Acked-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Signed-off-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://lore.kernel.org/r/20260128-audition-professed-31d1b66d5b4e@spud
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
- Shawn Guo steps down and Frank Li steps up as i.MX platform
maintainer
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEFmJXigPl4LoGSz08UFdYWoewfM4FAml2w5gACgkQUFdYWoew
fM71qwgApSyKWYpBI+v067vSpOb4uUEBHK2b/pmW5cKmvz0qdDWYDhVUMwCMQxSX
5ZOYsFgNHyjQuOzFXgIhmCITo+b6lKAvaQqQ2H1Rz8XvIoijD5ooQnRo0vERQbzt
O+rE/zYGaTOEnZNIAqLtwepmBHJ8JNkIO9Le/J2YSYS8+jHdzJ1Cm/Ou6FIFhPaX
k9Gr9d5trBo6bNUWqHnzRkC8w/9KdCqMihu0ueSgNPxeRYDZF2KUrXyByuVPTj5T
hkfnbwYmqK/bQdjfKZ7XXKTgosdc6UgWhEunPJEr7smWKbqAYgU09vns3RICENNd
bxVSnxzx8sYAAcoeiNnicldQNZdocw==
=sc5O
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAml7Ic4ACgkQmmx57+YA
GNkUpg/+I1nDUr5MVmNoJJ+Xpewa2PKCnbb4L1YGsm4b86PkGU53Bfe6CaL7+zvc
xixgKp/OR5JpNy1q1YyZQReWWm8rVQoaHnVhGeZFfkxMg6SiphdO5ugpRwHFGn+C
IhsSDN0iEOYCfBlpoIgHYcpQDP7I6jK1wXz37qJIqc07Ef+S+JgrLFuz6p0DLBJc
pnwV5QsaY2mxEbmwWh3MLiwrW7U+mam5NTwADzEQRemgmuIy7teNEx7gP4sLqiht
609WMSVr0SGfjjOHMGmGqUm7k2W1bqrUbIlwQrJkQXmgLPwqMn8zApUeXA2E9lkx
PT08KOkbOCxRgqHHMQmnMVU397ZvGmBRvAzXYgMGWx28UIZ7JLv4VFPonGzIkboL
vjgHN7T1nHvt7+DMDzyF6AuGSVrGOomeDPCNMRgykYf+EjHI7Q6iTWLDDYWFoioj
MK+nm4XQYqT9863pYpxRCAaaqFvc5/769CbpFnFOGe3Nmsl5f3ERip2nR8DOA3j3
LI/nK25D/kTZpGUTPGTOp+qRMBWpibqBEvhkNvvnZy5D2TYtHJ3BMt/0+FR1qsat
va0AjGCzzIdcZXwTjGAULlPUsoAO6o5zTdjWnpZm6X/ALVXNi1LMDqoXaX6qyHgO
MxCQILU3BvQE3cV1x2MaVt73E1txd+deu6J9e4ji/pep9r4ZbK4=
=LOM4
-----END PGP SIGNATURE-----
Merge tag 'imx-maintainers-6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/shawnguo/linux into arm/fixes
i.MX MAINTAINERS changes for 6.20:
- Shawn Guo steps down and Frank Li steps up as i.MX platform
maintainer
* tag 'imx-maintainers-6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/shawnguo/linux:
MAINTAINERS: Replace Shawn with Frank as i.MX platform maintainer
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
The current implementation uses a linear list to find queued packets by
ID when processing verdicts from userspace. With large queue depths and
out-of-order verdicting, this O(n) lookup becomes a significant
bottleneck, causing userspace verdict processing to dominate CPU time.
Replace the linear search with a hash table for O(1) average-case
packet lookup by ID. A global rhashtable spanning all network
namespaces attributes hash bucket memory to kernel but is subject to
fixed upper bound.
Signed-off-by: Scott Mitchell <scott.k.mitch1@gmail.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
Use the is_leap_year() helper from rtc.h instead of
writing it by hand
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
Similar to IPIP, introduce specific selftest for IP6IP6 flowtable SW
acceleration in nft_flowtable.sh
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
Introduce sw acceleration for tx path of IP6IP6 tunnels relying on the
netfilter flowtable infrastructure.
IP6IP6 tx sw acceleration can be tested running the following scenario
where the traffic is forwarded between two NICs (eth0 and eth1) and an
IP6IP6 tunnel is used to access a remote site (using eth1 as the underlay
device):
ETH0 -- TUN0 <==> ETH1 -- [IP network] -- TUN1 (2001:db8:3::2)
$ip addr show
6: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:00:22:33:11:55 brd ff:ff:ff:ff:ff:ff
inet6 2001:db8:1::2/64 scope global nodad
valid_lft forever preferred_lft forever
7: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:11:22:33:11:55 brd ff:ff:ff:ff:ff:ff
inet6 2001:db8:2::1/64 scope global nodad
valid_lft forever preferred_lft forever
8: tun0@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/tunnel6 2001:db8:2::1 peer 2001:db8:2::2 permaddr ce9c:2940:7dcc::
inet6 2002:db8:1::1/64 scope global nodad
valid_lft forever preferred_lft forever
$ip -6 route show
2001:db8:1::/64 dev eth0 proto kernel metric 256 pref medium
2001:db8:2::/64 dev eth1 proto kernel metric 256 pref medium
2002:db8:1::/64 dev tun0 proto kernel metric 256 pref medium
default via 2002:db8:1::2 dev tun0 metric 1024 pref medium
$nft list ruleset
table inet filter {
flowtable ft {
hook ingress priority filter
devices = { eth0, eth1 }
}
chain forward {
type filter hook forward priority filter; policy accept;
meta l4proto { tcp, udp } flow add @ft
}
}
Reproducing the scenario described above using veths I got the following
results:
- TCP stream received from the IPIP tunnel:
- net-next: (baseline) ~93Gbps
- net-next + IP6IP6 flowtbale support: ~98Gbps
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
Introduce sw acceleration for rx path of IP6IP6 tunnels relying on the
netfilter flowtable infrastructure. Subsequent patches will add sw
acceleration for IP6IP6 tunnels tx path.
IP6IP6 rx sw acceleration can be tested running the following scenario
where the traffic is forwarded between two NICs (eth0 and eth1) and an
IP6IP6 tunnel is used to access a remote site (using eth1 as the underlay
device):
ETH0 -- TUN0 <==> ETH1 -- [IP network] -- TUN1 (2001:db8:3::2)
$ip addr show
6: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:00:22:33:11:55 brd ff:ff:ff:ff:ff:ff
inet6 2001:db8:1::2/64 scope global nodad
valid_lft forever preferred_lft forever
7: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:11:22:33:11:55 brd ff:ff:ff:ff:ff:ff
inet6 2001:db8:2::1/64 scope global nodad
valid_lft forever preferred_lft forever
8: tun0@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/tunnel6 2001:db8:2::1 peer 2001:db8:2::2 permaddr ce9c:2940:7dcc::
inet6 2002:db8:1::1/64 scope global nodad
valid_lft forever preferred_lft forever
$ip -6 route show
2001:db8:1::/64 dev eth0 proto kernel metric 256 pref medium
2001:db8:2::/64 dev eth1 proto kernel metric 256 pref medium
2002:db8:1::/64 dev tun0 proto kernel metric 256 pref medium
default via 2002:db8:1::2 dev tun0 metric 1024 pref medium
$nft list ruleset
table inet filter {
flowtable ft {
hook ingress priority filter
devices = { eth0, eth1 }
}
chain forward {
type filter hook forward priority filter; policy accept;
meta l4proto { tcp, udp } flow add @ft
}
}
Reproducing the scenario described above using veths I got the following
results:
- TCP stream received from the IPIP tunnel:
- net-next: (baseline) ~ 81Gbps
- net-next + IP6IP6 flowtbale support: ~112Gbps
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
Add tunnel hdr_size and tunnel proto fields in nf_flowtable_ctx struct
in order to store IP tunnel header size and protocol used during IPIP
and IP6IP6 tunnel sw offloading decapsulation and avoid recomputing them
during tunnel header pop since this is constant for IPv6.
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
Rely on nf_flowtable_ctx struct pointer in nf_flow_ip4_tunnel_proto and
nf_flow_skb_encap_protocol routine signature. This is a preliminary patch
to introduce IP6IP6 flowtable acceleration since nf_flowtable_ctx will
be used to store IP6IP6 tunnel info.
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
This adds a single change, enabling to compile the
MediaTek HDMIv2 driver as module.
-----BEGIN PGP SIGNATURE-----
iLoEABYKAGIWIQQn3Xxr56ypAcSHzXSaNgTPrZeEeAUCaXNXxBsUgAAAAAAEAA5t
YW51MiwyLjUrMS4xMSwyLDIoHGFuZ2Vsb2dpb2FjY2hpbm8uZGVscmVnbm9AY29s
bGFib3JhLmNvbQAKCRCaNgTPrZeEeHy4AP49d82SuocGcFFLzdBTW57mAtQ5TBAc
90pCAGuE7K8Q+QEA4kwwSpb1Uh3IqiOEDa9kmsZsAwgFSh5aiWzrr7GFJgM=
=SSM5
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAml7GuwACgkQmmx57+YA
GNlbRhAAkttWB37ksyZQ2RtoBRBSDWIhTn0o2GI0ma+RBMW9coPnMqhMSsCMue1q
gAl0dJF0Bp+WF4C04Ajg/TE45Z1So5f/sZrncwLeZJOk/DtLLzLWqDG1vJact+Ba
G4NB7WaYb2BQzPwoK9cmJDf2Hw7N4sugzK3BjP8/kDFsA1hMVsLVfd4NdDiZwRQP
dLDsSa93oCIjr3TSwWzd1dmAOK174aX9nJS+YhHIg3Ig6aQbAd0M6flpV3D5D22x
gvVhkQNXTFu9snzv4GfUNBMeCQmeZlMPBl7lc2y+35gR5+0J/hZbzQIVUtqwHrxS
YvUWo+I6LckA/BRMFWO45RwRct0X+pt3Wtcvb6q1GpPlhuEvvfSvWBeWwGilfg0S
Ts8juDzVBrfN0+XuFzGnuJA3eZz3wiZDnl1H9Vn/ePIqqsd9lBHOJlmDQ7nYAp0M
Xoc8I4ZaYzhwg6rKbC3BtBN44xp+7AHeiAyzaplolLR2n8sFWmDvZWnA2kWSNVw/
kIdfs+Tj1bTZa/nrxe+3+IbS9i3RlJv7vSZ00DOS8mE72h+6tRYHFYyWYkmLg8fm
Ihe5EE92TfOd/LHUidVcIathRAz48gOL3eHDMove/VXG8mtUaAoGJiRIsaMEsFt0
XivJFooXxipLMtDaia4r1xrnhGtPDUj42dVg7V0JpVw9MKFXw90=
=mxBQ
-----END PGP SIGNATURE-----
Merge tag 'mtk-defconfig-for-v6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/mediatek/linux into soc/defconfig
MediaTek defconfig updates
This adds a single change, enabling to compile the
MediaTek HDMIv2 driver as module.
* tag 'mtk-defconfig-for-v6.20' of https://git.kernel.org/pub/scm/linux/kernel/git/mediatek/linux:
arm64: defconfig: Enable Mediatek HDMIv2 driver
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Extend the existing Tegra186 GPIO controller driver with support for the
GPIO controller found on Tegra264.
Use the "wakeup-parent" phandle from the GPIO device tree node to
ensure the GPIO driver associates with the intended PMC device.
Relying only on compatible-based lookup can select an unexpected
PMC node, so fall back to compatible-based lookup when the phandle
is not present.
Signed-off-by: Prathamesh Shete <pshete@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Link: https://patch.msgid.link/20260128085114.1137725-2-pshete@nvidia.com
Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
Extend the existing Tegra186 GPIO controller device tree bindings with
support for the GPIO controller found on Tegra264. The number of pins
is slightly different, but the programming model remains the same.
Add a new header, include/dt-bindings/gpio/nvidia,tegra264-gpio.h,
that defines port IDs as well as the TEGRA264_MAIN_GPIO() helper,
both of which are used in conjunction to create a unique specifier
for each pin.
On Tegra, GPIO wake events are latched and routed via the PMC.
Document the standard DT property, wakeup-parent, which is a phandle to
the PMC interrupt controller that provides the parent wake interrupt
domain for the GPIO controller. If the property is absent the driver
falls back to a compatible-based lookup.
Signed-off-by: Prathamesh Shete <pshete@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://patch.msgid.link/20260128085114.1137725-1-pshete@nvidia.com
[Bartosz: fixed whitespace errors]
Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
A number of stat items related to cpu slabs became unused, remove them.
Two of those were ALLOC_FASTPATH and FREE_FASTPATH. But instead of
removing those, use them instead of ALLOC_PCS and FREE_PCS, since
sheaves are the new (and only) fastpaths, Remove the recently added
_PCS variants instead.
Change where FREE_SLOWPATH is counted so that it only counts freeing of
objects by slab users that (for whatever reason) do not go to a percpu
sheaf, and not all (including internal) callers of __slab_free(). Thus
sheaf flushing (already counted by SHEAF_FLUSH) does not affect
FREE_SLOWPATH anymore. This matches how ALLOC_SLOWPATH doesn't count
sheaf refills (counted by SHEAF_REFILL).
Reviewed-by: Hao Li <hao.li@linux.dev>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Enable the clock driver support for SpacemiT K3 SoC, configure it as
built-in by default, but users should be able to change it as kernel
module if needed.
Signed-off-by: Yixun Lan <dlan@kernel.org>
Signed-off-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://lore.kernel.org/r/20260128-sitter-crazily-184c1a7606db@spud
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
The cpu slabs and their deactivations were removed, so remove the unused
stat items. Weirdly enough the values were also used to control
__add_partial() adding to head or tail of the list, so replace that with
a new enum add_mode, which is cleaner.
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Currently slabs are only frozen after consistency checks failed. This
can happen only in caches with debugging enabled, and those use
free_to_partial_list() for freeing. The non-debug operation of
__slab_free() can thus stop considering the frozen field, and we can
remove the FREE_FROZEN stat.
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
The changes related to sheaves made the description of locking and other
details outdated. Update it to reflect current state.
Also add a new copyright line due to major changes.
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
__refill_objects() currently only attempts to get partial slabs from the
local node and then allocates new slab(s). Expand it to trying also
other nodes while observing the remote node defrag ratio, similarly to
get_any_partial().
This will prevent allocating new slabs on a node while other nodes have
many free slabs. It does mean sheaves will contain non-local objects in
that case. Allocations that care about specific node will still be
served appropriately, but might get a slowpath allocation.
Like get_any_partial() we do observe cpuset_zone_allowed(), although we
might be refilling a sheaf that will be then used from a different
allocation context.
We can also use the resulting refill_objects() in
__kmem_cache_alloc_bulk() for non-debug caches. This means
kmem_cache_alloc_bulk() will get better performance when sheaves are
exhausted. kmem_cache_alloc_bulk() cannot indicate a preferred node so
it's compatible with sheaves refill in preferring the local node.
Its users also have gfp flags that allow spinning, so document that
as a requirement.
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
The macros slub_get_cpu_ptr()/slub_put_cpu_ptr() are now unused, remove
them. USE_LOCKLESS_FAST_PATH() has lost its true meaning with the code
being removed. The only remaining usage is in fact testing whether we
can assert irqs disabled, because spin_lock_irqsave() only does that on
!RT. Test for CONFIG_PREEMPT_RT instead.
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
The cpu slab is not used anymore for allocation or freeing, the
remaining code is for flushing, but it's effectively dead. Remove the
whole struct kmem_cache_cpu, the flushing code and other orphaned
functions.
The remaining used field of kmem_cache_cpu is the stat array with
CONFIG_SLUB_STATS. Put it instead in a new struct kmem_cache_stats.
In struct kmem_cache, the field is cpu_stats and placed near the
end of the struct.
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
The kmalloc_nolock() implementation has several complications and
restrictions due to SLUB's cpu slab locking, lockless fastpath and
PREEMPT_RT differences. With cpu slab usage removed, we can simplify
things:
- relax the PREEMPT_RT context checks as they were before commit
99a3e3a1cf ("slab: fix kmalloc_nolock() context check for
PREEMPT_RT") and also reference the explanation comment in the page
allocator
- the local_lock_cpu_slab() macros became unused, remove them
- we no longer need to set up lockdep classes on PREEMPT_RT
- we no longer need to annotate ___slab_alloc as NOKPROBE_SYMBOL
since there's no lockless cpu freelist manipulation anymore
- __slab_alloc_node() can be called from kmalloc_nolock_noprof()
unconditionally. It can also no longer return EBUSY. But trylock
failures can still happen so retry with the larger bucket if the
allocation fails for any reason.
Note that we still need __CMPXCHG_DOUBLE, because while it was removed
we don't use cmpxchg16b on cpu freelist anymore, we still use it on
slab freelist, and the alternative is slab_lock() which can be
interrupted by a nmi. Clarify the comment to mention it specifically.
Acked-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
There are no more cpu slabs so we don't need their deferred
deactivation. The function is now only used from places where we
allocate a new slab but then can't spin on node list_lock to put it on
the partial list. Instead of the deferred action we can free it directly
via __free_slab(), we just need to tell it to use _nolock() freeing of
the underlying pages and take care of the accounting.
Since free_frozen_pages_nolock() variant does not yet exist for code
outside of the page allocator, create it as a trivial wrapper for
__free_frozen_pages(..., FPI_TRYLOCK).
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
We have removed cpu slab usage from allocation paths. Now remove
do_slab_free() which was freeing objects to the cpu slab when
the object belonged to it. Instead call __slab_free() directly,
which was previously the fallback.
This simplifies kfree_nolock() - when freeing to percpu sheaf
fails, we can call defer_free() directly.
Also remove functions that became unused.
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
We have removed the partial slab usage from allocation paths. Now remove
the whole config option and associated code.
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
We now rely on sheaves as the percpu caching layer and can refill them
directly from partial or newly allocated slabs. Start removing the cpu
(partial) slabs code, first from allocation paths.
This means that any allocation not satisfied from percpu sheaves will
end up in ___slab_alloc(), where we remove the usage of cpu (partial)
slabs, so it will only perform get_partial() or new_slab(). In the
latter case we reuse alloc_from_new_slab() (when we don't use
the debug/tiny alloc_single_from_new_slab() variant).
In get_partial_node() we used to return a slab for freezing as the cpu
slab and to refill the partial slab. Now we only want to return a single
object and leave the slab on the list (unless it became full). We can't
simply reuse alloc_single_from_partial() as that assumes freeing uses
free_to_partial_list(). Instead we need to use __slab_update_freelist()
to work properly against a racing __slab_free().
To reflect the new purpose of get_partial() functions, rename them to
get_from_partial(), get_from_partial_node(), and get_from_any_partial().
The rest of the changes is removing functions that no longer have any
callers.
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
At this point we have sheaves enabled for all caches, but their refill
is done via __kmem_cache_alloc_bulk() which relies on cpu (partial)
slabs - now a redundant caching layer that we are about to remove.
The refill will thus be done from slabs on the node partial list.
Introduce new functions that can do that in an optimized way as it's
easier than modifying the __kmem_cache_alloc_bulk() call chain.
Introduce struct partial_bulk_context, a variant of struct
partial_context that can return a list of slabs from the partial list
with the sum of free objects in them within the requested min and max.
Introduce get_partial_node_bulk() that removes the slabs from freelist
and returns them in the list. There is a racy read of slab->counters
so make sure the non-atomic write in __update_freelist_slow() is not
tearing.
Introduce get_freelist_nofreeze() which grabs the freelist without
freezing the slab.
Introduce alloc_from_new_slab() which can allocate multiple objects from
a newly allocated slab where we don't need to synchronize with freeing.
In some aspects it's similar to alloc_single_from_new_slab() but assumes
the cache is a non-debug one so it can avoid some actions. It supports
the allow_spin parameter, which we always set true here, but the
followup change will reuse the function in a context where it may be
false.
Introduce __refill_objects() that uses the functions above to fill an
array of objects. It has to handle the possibility that the slabs will
contain more objects that were requested, due to concurrent freeing of
objects to those slabs. When no more slabs on partial lists are
available, it will allocate new slabs. It is intended to be only used
in context where spinning is allowed, so add a WARN_ON_ONCE check there.
Finally, switch refill_sheaf() to use __refill_objects(). Sheaves are
only refilled from contexts that allow spinning, or even blocking.
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
