syzbot is reporting
unregister_netdevice: waiting for sit0 to become free. Usage count = 2
problem. A debug printk() patch found that a refcount is obtained at
xdp_convert_md_to_buff() from bpf_prog_test_run_xdp().
According to commit ec94670fcb ("bpf: Support specifying ingress via
xdp_md context in BPF_PROG_TEST_RUN"), the refcount obtained by
xdp_convert_md_to_buff() will be released by xdp_convert_buff_to_md().
Therefore, we can consider that the error handling path introduced by
commit 1c19499825 ("bpf: introduce frags support to
bpf_prog_test_run_xdp()") forgot to call xdp_convert_buff_to_md().
Reported-by: syzbot+881d65229ca4f9ae8c84@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=881d65229ca4f9ae8c84
Fixes: 1c19499825 ("bpf: introduce frags support to bpf_prog_test_run_xdp()")
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Link: https://lore.kernel.org/r/af090e53-9d9b-4412-8acb-957733b3975c@I-love.SAKURA.ne.jp
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add 'stop' subcommand to kublk utility that uses the new
UBLK_CMD_TRY_STOP_DEV command when --safe option is specified.
This allows stopping a device only if it has no active openers,
returning -EBUSY otherwise.
Also add test_generic_16.sh to test the new functionality.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add a best-effort stop command, UBLK_CMD_TRY_STOP_DEV, which only stops a
ublk device when it has no active openers.
Unlike UBLK_CMD_STOP_DEV, this command does not disrupt existing users.
New opens are blocked only after disk_openers has reached zero; if the
device is busy, the command returns -EBUSY and leaves it running.
The ub->block_open flag is used only to close a race with an in-progress
open and does not otherwise change open behavior.
Advertise support via the UBLK_F_SAFE_STOP_DEV feature flag.
Signed-off-by: Yoav Cohen <yoav@nvidia.com>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This function always returns 0, so there is no need to return a value.
Signed-off-by: Yoav Cohen <yoav@nvidia.com>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Expose individual VRAM temperature attributes.
Update Xe hwmon documentation for this entry.
v2:
- Avoid using default switch case for VRAM individual temperatures.
- Append labels with VRAM channel number.
- Update kernel version in Xe hwmon documentation.
v3:
- Add missing brackets in Xe hwmon documentation from VRAM channel sysfs.
- Reorder BMG_VRAM_TEMPERATURE_N macro in xe_pcode_regs.h.
- Add api to check if VRAM is available on the channel.
v4:
- Improve VRAM label handling to eliminate temp variable by
introducing a dedicated array vram_label in xe_hwmon_thermal_info.
- Remove a magic number.
- Change the label from vram_X to vram_ch_X.
v5:
- Address review comments from Raag.
- Change vram to VRAM in commit title and subject.
- Refactor BMG_VRAM_TEMPERATURE_N macro.
- Refactor is_vram_ch_available().
- Rephrase a comment.
- Check individual VRAM temperature limits in addition to VRAM
availability in xe_hwmon_temp_is_visible. (Raag)
- Move VRAM label change out of this patch.
v6:
- Use in_range() for VRAM_N index check instead of if check. (Raag)
- Minor aesthetic changes.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-5-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Expose GPU PCIe average temperature and its limits via hwmon sysfs entry
temp5_xxx.
Update Xe hwmon sysfs documentation for this.
v2: Update kernel version in Xe hwmon documentation. (Raag)
v3:
- Address review comments from Raag.
- Remove redundant debug log.
- Update kernel version in Xe hwmon documentation. (Raag)
v4:
- Address review comments from Raag.
- Group new temperature attributes with existing temperature attributes
as per channel index in Xe hwmon documentation.
- Use TEMP_MASK instead of TEMP_MASK_MAILBOX.
- Add PCIE_SENSOR_MASK which uses REG_FIELD_GET as replacement of
PCIE_SENSOR_SHIFT.
v5:
- Address review comments from Raag.
- Use REG_FIELD_GET to get PCIe temperature.
- Move PCIE_SENSOR_GROUP_ID and PCIE_SENSOR_MASK to xe_pcode_api.h
- Cosmetic change.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-4-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Expose GPU memory controller average temperature and its limits under
temp4_xxx.
Update Xe hwmon documentation for this.
v2:
- Rephrase commit message. (Badal)
- Update kernel version in Xe hwmon documentation. (Raag)
v3:
- Update kernel version in Xe hwmon documentation.
- Address review comments from Raag.
- Remove obvious comments.
- Remove redundant debug logs.
- Remove unnecessary checks.
- Avoid magic numbers.
- Add new comments.
- Use temperature sensors count to make memory controller visible.
- Use temperature limits of package for memory controller.
v4:
- Address review comments from Raag.
- Group new temperature attributes with existing temperature attributes
as per channel index in Xe hwmon documentation.
- Use DIV_ROUND_UP to calculate dwords needed for temperature limits.
- Minor aesthetic refinements.
- Remove unused TEMP_MASK_MAILBOX.
v5:
- Use REG_FIELD_GET to get count from READ_THERMAL_DATA output. (Raag)
- Change count print from decimal to hexadecimal.
- Cosmetic changes.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-3-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Read temperature limits using pcode mailbox and expose shutdown
temperature limit as tempX_emergency, critical temperature limit as
tempX_crit and GPU max temperature limit as temp2_max.
Update Xe hwmon documentation with above entries.
v2:
- Resolve a documentation warning.
- Address below review comments from Raag.
- Update date and kernel version in Xe hwmon documentation.
- Remove explicit disable of has_mbx_thermal_info for unsupported
platforms.
- Remove unnecessary default case in switches.
- Remove obvious comments.
- Use TEMP_LIMIT_MAX to compute number of dwords needed in
xe_hwmon_thermal_info.
- Remove THERMAL_LIMITS_DWORDS macro.
- Use has_mbx_thermal_info for checking thermal mailbox support.
v3:
- Address below minor comments. (Raag)
- Group new temperature attributes with existing temperature attributes
as per channel index in Xe hwmon documentation.
- Rename enums of xe_temp_limit to improve clarity.
- Use DIV_ROUND_UP to calculate dwords needed for temperature limits.
- Use return instead of breaks in xe_hwmon_temp_read.
- Minor aesthetic refinements.
v4:
- Remove a redundant break. (Raag)
- Update drm_dbg to drm_warn to inform user of unavailability for
thermal mailbox on expected platforms.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-2-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Jakub Kicinski says:
====================
tools: ynl: cli: improve the help and doc
I had some time on the plane to LPC, so here are improvements
to the --help and --list-attrs handling of YNL CLI which seem
in order given growing use of YNL as a real CLI tool.
====================
Link: https://patch.msgid.link/20260110233142.3921386-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
As pointed out during review of the --list-attrs support the GET
ops very often return the same attrs from do and dump. Make the
output more readable by combining the reply information, from:
Do request attributes:
- ifindex: u32
netdev ifindex
Do reply attributes:
- ifindex: u32
netdev ifindex
[ .. other attrs .. ]
Dump reply attributes:
- ifindex: u32
netdev ifindex
[ .. other attrs .. ]
To, after:
Do request attributes:
- ifindex: u32
netdev ifindex
Do and Dump reply attributes:
- ifindex: u32
netdev ifindex
[ .. other attrs .. ]
Tested-by: Gal Pressman <gal@nvidia.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20260110233142.3921386-8-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Event and notify handling is quite different from do / dump
handling. Forcing it into print_mode_attrs() doesn't really
buy us anything as events and notifications do not have requests.
Call print_attr_list() directly. Apart form subjective code
clarity this also removes the word "reply" from the output:
Before:
Event reply attributes:
Now:
Event attributes:
Tested-by: Gal Pressman <gal@nvidia.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20260110233142.3921386-7-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We'll soon add more code to the --doc handling. Factor it out
to avoid making main() too long.
Tested-by: Gal Pressman <gal@nvidia.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20260110233142.3921386-6-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
--list-attrs also provides information about the operation itself.
So --doc seems more appropriate. Add an alias.
Tested-by: Gal Pressman <gal@nvidia.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20260110233142.3921386-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Improve the clarity of --help. Reorder, provide some grouping and
add help messages to most of the options.
No functional changes intended.
Tested-by: Gal Pressman <gal@nvidia.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20260110233142.3921386-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We already use textwrap when printing "doc" section about an attribute,
but only to indent the text. Switch to using fill() to split and indent
all the lines. While at it indent the text by 2 more spaces, so that it
doesn't align with the name of the attribute.
Before (I'm drawing a "box" at ~60 cols here, in an attempt for clarity):
| - irq-suspend-timeout: uint |
| The timeout, in nanoseconds, of how long to suspend irq|
|processing, if event polling finds events |
After:
| - irq-suspend-timeout: uint |
| The timeout, in nanoseconds, of how long to suspend |
| irq processing, if event polling finds events |
Tested-by: Gal Pressman <gal@nvidia.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20260110233142.3921386-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
It's a little hard to make sense of the output of --list-attrs,
it looks like a wall of text. Sprinkle a little bit of formatting -
make op and attr names bold, and Enum: / Flags: keywords italics.
Tested-by: Gal Pressman <gal@nvidia.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://patch.msgid.link/20260110233142.3921386-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Instead of having a pre-filled array xen_mmu_ops for Xen PV paravirt
functions, drop the array and assign each element individually.
This is in preparation of reducing the paravirt include hell by
splitting paravirt.h into multiple more fine grained header files,
which will in turn require to split up the pv_ops vector as well.
Dropping the pre-filled array makes life easier for objtool to
detect missing initializers in multiple pv_ops_ arrays.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Link: https://patch.msgid.link/20260105110520.21356-18-jgross@suse.com
Commit 4d4021b0bb ("ASoC: tegra: Fix uninitialized flat cache warning
in tegra210_ahub") attempted to fix the uninitialized flat cache warning
that is observed for the Tegra210 AHUB driver. However, the change broke
various audio tests because an -EBUSY error is returned when accessing
registers from cache before they are read from hardware. Revert this
change for now, until a proper fix is available.
Fixes: 4d4021b0bb ("ASoC: tegra: Fix uninitialized flat cache warning in tegra210_ahub")
Signed-off-by: sheetal <sheetal@nvidia.com>
Acked-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20251217132524.2844499-1-sheetal@nvidia.com
Signed-off-by: Mark Brown <broonie@kernel.org>
Add a filter to skip the RT172 VB configuration if a SmartMic Function
is not found in the SDCA descriptors.
If the ACPI information is incorrect this can only be quirked further
with DMI information.
Signed-off-by: Vijendar Mukunda <Vijendar.Mukunda@amd.com>
Link: https://patch.msgid.link/20260110064505.1485927-1-Vijendar.Mukunda@amd.com
Signed-off-by: Mark Brown <broonie@kernel.org>
Remove the dangling Kconfig references to CLK_FIXED_FCH since they
are not used anywhere else in the kernel source tree.
Fixes: 281ddf62f5 ("ASoC: amd: Kconfig: Select fch clock support with machine driver")
Fixes: d4c750f2c7 ("ASoC: amd: acp: Add generic machine driver support for ACP cards")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://patch.msgid.link/20251228190542.2482910-1-rdunlap@infradead.org
Signed-off-by: Mark Brown <broonie@kernel.org>
The a2l_style is only relevant to the command line version, so rename
to make this clearer.
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Howard Chu <howardchu95@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephen Brennan <stephen.s.brennan@oracle.com>
Cc: Tony Jones <tonyj@suse.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Move the aes_encrypt_zvkned() and aes_decrypt_zvkned() assembly
functions into lib/crypto/, wire them up to the AES library API, and
remove the "aes-riscv64-zvkned" crypto_cipher algorithm.
To make this possible, change the prototypes of these functions to
take (rndkeys, key_len) instead of a pointer to crypto_aes_ctx, and
change the RISC-V AES-XTS code to implement tweak encryption using the
AES library instead of directly calling aes_encrypt_zvkned().
The result is that both the AES library and crypto_cipher APIs use
RISC-V's AES instructions, whereas previously only crypto_cipher did
(and it wasn't enabled by default, which this commit fixes as well).
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-15-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Move the POWER8 AES assembly code into lib/crypto/, wire the key
expansion and single-block en/decryption functions up to the AES library
API, and remove the superseded "p8_aes" crypto_cipher algorithm.
The result is that both the AES library and crypto_cipher APIs are now
optimized for POWER8, whereas previously only crypto_cipher was (and
optimizations weren't enabled by default, which this commit fixes too).
Note that many of the functions in the POWER8 assembly code are still
used by the AES mode implementations in arch/powerpc/crypto/. For now,
just export these functions. These exports will go away once the AES
modes are migrated to the library as well. (Trying to split up the
assembly file seemed like much more trouble than it would be worth.)
Another challenge with this code is that the POWER8 assembly code uses a
custom format for the expanded AES key. Since that code is imported
from OpenSSL and is also targeted to POWER8 (rather than POWER9 which
has better data movement and byteswap instructions), that is not easily
changed. For now I've just kept the custom format. To maintain full
correctness, this requires executing some slow fallback code in the case
where the usability of VSX changes between key expansion and use. This
should be tolerable, as this case shouldn't happen in practice.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-14-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Move the PowerPC SPE AES assembly code into lib/crypto/, wire the key
expansion and single-block en/decryption functions up to the AES library
API, and remove the superseded "aes-ppc-spe" crypto_cipher algorithm.
The result is that both the AES library and crypto_cipher APIs are now
optimized with SPE, whereas previously only crypto_cipher was (and
optimizations weren't enabled by default, which this commit fixes too).
Note that many of the functions in the PowerPC SPE assembly code are
still used by the AES mode implementations in arch/powerpc/crypto/. For
now, just export these functions. These exports will go away once the
AES modes are migrated to the library as well. (Trying to split up the
assembly files seemed like much more trouble than it would be worth.)
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-13-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Move the ARM64 optimized AES key expansion and single-block AES
en/decryption code into lib/crypto/, wire it up to the AES library API,
and remove the superseded crypto_cipher algorithms.
The result is that both the AES library and crypto_cipher APIs are now
optimized for ARM64, whereas previously only crypto_cipher was (and the
optimizations weren't enabled by default, which this fixes as well).
Note: to see the diff from arch/arm64/crypto/aes-ce-glue.c to
lib/crypto/arm64/aes.h, view this commit with 'git show -M10'.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-12-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Move the ARM optimized single-block AES en/decryption code into
lib/crypto/, wire it up to the AES library API, and remove the
superseded "aes-arm" crypto_cipher algorithm.
The result is that both the AES library and crypto_cipher APIs are now
optimized for ARM, whereas previously only crypto_cipher was (and the
optimizations weren't enabled by default, which this fixes as well).
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-11-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Now that the AES library's performance has been improved, replace
aes_generic.c with a new file aes.c which wraps the AES library.
In preparation for making the AES library actually utilize the kernel's
existing architecture-optimized AES code including AES instructions, set
the driver name to "aes-lib" instead of "aes-generic". This mirrors
what's been done for the hash algorithms. Update testmgr.c accordingly.
Since this removes the crypto_aes_set_key() helper function, add
temporary replacements for it to arch/arm/crypto/aes-cipher-glue.c and
arch/arm64/crypto/aes-cipher-glue.c. This is temporary, as that code
will be migrated into lib/crypto/ in later commits.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-10-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Remove aes-fixed-time, i.e. CONFIG_CRYPTO_AES_TI. This was a wrapper
around the 256-byte-table-based AES implementation in lib/crypto/aes.c,
with extra code to enable and disable IRQs for constant-time hardening.
While nice in theory, in practice this had the following issues:
- For bulk en/decryption it was 2-4 times slower than aes-generic. This
resulted in aes-generic still being needed, creating fragmentation.
- Having both aes-generic and aes-fixed-time punted an AES
implementation decision to distros and users who are generally
unprepared to handle it. In practice, whether aes-fixed-time gets
used tends to be incidental and not match an explicit distro or user
intent. (While aes-fixed-time has a higher priority than aes-generic,
whether it actually gets enabled, loaded, and used depends on the
kconfig and whether a modprobe of "aes" happens to be done. It also
has a lower priority than aes-arm and aes-arm64.)
- My changes to the generic AES code (in other commits) significantly
close the gap with aes-fixed-time anyway. The table size is reduced
from 8192 bytes to 1024 bytes, and prefetching is added.
- While AES code *should* be constant-time, the real solutions for that
are AES instructions (which most CPUs have now) or bit-slicing. arm
and arm64 already have bit-sliced AES code for many modes; generic
bit-sliced code could be written but would be very slow for single
blocks. Overall, I suggest that trying to write constant-time
table-based AES code is a bit futile anyway, and in the rare cases
where a proper AES implementation is still unavailable it's reasonable
to compromise with an implementation that simply prefetches the table.
Thus, this commit removes aes-fixed-time and CONFIG_CRYPTO_AES_TI. The
replacement is just the existing CONFIG_CRYPTO_AES, which for now maps
to the existing aes-generic code, but I'll soon be changing to use the
improved AES library code instead.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-9-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Instead of crypto_ft_tab[0] from aes_generic.c, use aes_enc_tab from
lib/crypto/aes.c. These contain the same data, so the result is the
same. This will allow aes_generic.c to eventually be removed.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-8-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
The call to sha256() occurs in code that is built when either
CRYPTO_AES_ARM64_CE_BLK or CRYPTO_AES_ARM64_NEON_BLK. The option
CRYPTO_AES_ARM64 is unrelated, notwithstanding its documentation. I'll
be removing CRYPTO_AES_ARM64 soon anyway, but before doing that, fix
where CRYPTO_LIB_SHA256 is selected from.
Fixes: 01834444d9 ("crypto: arm64/aes - use SHA-256 library instead of crypto_shash")
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-7-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Instead of crypto_ft_tab and crypto_it_tab from aes_generic.c, use
aes_enc_tab and aes_dec_tab from lib/crypto/aes.c. These contain the
same data in the first 1024 bytes (which is the part that this code
uses), so the result is the same. This will allow aes_generic.c to
eventually be removed.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-6-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Instead of crypto_ft_tab and crypto_it_tab from aes_generic.c, use
aes_enc_tab and aes_dec_tab from lib/crypto/aes.c. These contain the
same data in the first 1024 bytes (which is the part that this code
uses), so the result is the same. This will allow aes_generic.c to
eventually be removed.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-5-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
aes-neonbs-glue.c calls __aes_arm_encrypt() and __aes_arm_decrypt() to
en/decrypt single blocks for CBC encryption, XTS tweak encryption, and
XTS ciphertext stealing. In preparation for making the AES library use
this same ARM-optimized single-block AES en/decryption code and making
it an internal implementation detail of the AES library, replace the
calls to these functions with calls to the AES library.
Note that this reduces the size of the aesbs_cbc_ctx and aesbs_xts_ctx
structs, since unnecessary decryption round keys are no longer included.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-4-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
The kernel's AES library currently has the following issues:
- It doesn't take advantage of the architecture-optimized AES code,
including the implementations using AES instructions.
- It's much slower than even the other software AES implementations: 2-4
times slower than "aes-generic", "aes-arm", and "aes-arm64".
- It requires that both the encryption and decryption round keys be
computed and cached. This is wasteful for users that need only the
forward (encryption) direction of the cipher: the key struct is 484
bytes when only 244 are actually needed. This missed optimization is
very common, as many AES modes (e.g. GCM, CFB, CTR, CMAC, and even the
tweak key in XTS) use the cipher only in the forward (encryption)
direction even when doing decryption.
- It doesn't provide the flexibility to customize the prepared key
format. The API is defined to do key expansion, and several callers
in drivers/crypto/ use it specifically to expand the key. This is an
issue when integrating the existing powerpc, s390, and sparc code,
which is necessary to provide full parity with the traditional API.
To resolve these issues, I'm proposing the following changes:
1. New structs 'aes_key' and 'aes_enckey' are introduced, with
corresponding functions aes_preparekey() and aes_prepareenckey().
Generally these structs will include the encryption+decryption round
keys and the encryption round keys, respectively. However, the exact
format will be under control of the architecture-specific AES code.
(The verb "prepare" is chosen over "expand" since key expansion isn't
necessarily done. It's also consistent with hmac*_preparekey().)
2. aes_encrypt() and aes_decrypt() will be changed to operate on the new
structs instead of struct crypto_aes_ctx.
3. aes_encrypt() and aes_decrypt() will use architecture-optimized code
when available, or else fall back to a new generic AES implementation
that unifies the existing two fragmented generic AES implementations.
The new generic AES implementation uses tables for both SubBytes and
MixColumns, making it almost as fast as "aes-generic". However,
instead of aes-generic's huge 8192-byte tables per direction, it uses
only 1024 bytes for encryption and 1280 bytes for decryption (similar
to "aes-arm"). The cost is just some extra rotations.
The new generic AES implementation also includes table prefetching,
making it have some "constant-time hardening". That's an improvement
from aes-generic which has no constant-time hardening.
It does slightly regress in constant-time hardening vs. the old
lib/crypto/aes.c which had smaller tables, and from aes-fixed-time
which disabled IRQs on top of that. But I think this is tolerable.
The real solutions for constant-time AES are AES instructions or
bit-slicing. The table-based code remains a best-effort fallback for
the increasingly-rare case where a real solution is unavailable.
4. crypto_aes_ctx and aes_expandkey() will remain for now, but only for
callers that are using them specifically for the AES key expansion
(as opposed to en/decrypting data with the AES library).
This commit begins the migration process by introducing the new structs
and functions, backed by the new generic AES implementation.
To allow callers to be incrementally converted, aes_encrypt() and
aes_decrypt() are temporarily changed into macros that use a _Generic
expression to call either the old functions (which take crypto_aes_ctx)
or the new functions (which take the new types). Once all callers have
been updated, these macros will go away, the old functions will be
removed, and the "_new" suffix will be dropped from the new functions.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-3-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Rename struct aes_key in aesp8-ppc.h and aes-gcm-p10-glue.c to
p8_aes_key and p10_aes_key, respectively. This frees up the name to use
in the library API in <crypto/aes.h>.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20260112192035.10427-2-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
We already added lockdep_assert_cpuset_lock_held(), use this new function
to keep consistency.
Signed-off-by: Zhao Mengmeng <zhaomengmeng@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
Add an implementation of addr2line that uses libdw.
Other addr2line implementations are slow, particularly in the case of
forking addr2line.
Add an implementation that caches the libdw information in the dso and
uses it to find the file and line number information.
Inline information is supported but because cu_walk_functions_at visits
the leaf function last add a inline_list__append_tail to reverse the
lists order.
Committer testing:
# perf probe -x ~/bin/perf libdw__addr2line
Added new event:
probe_perf:libdw_addr2line (on libdw__addr2line in /home/acme/bin/perf)
You can now use it in all perf tools, such as:
perf record -e probe_perf:libdw_addr2line -aR sleep 1
#
# perf stat -e probe_perf:libdw_addr2line perf report -f --dso perf --stdio -s srcfile,srcline
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 4K of event 'cpu/cycles/Pu'
# Event count (approx.): 5535180842
#
# Overhead Source File Source:Line
# ........ ............ ...............
#
99.04% inlineloop.c inlineloop.c:21
0.46% inlineloop.c inlineloop.c:20
#
# (Tip: For tracepoint events, try: perf report -s trace_fields)
#
Performance counter stats for 'perf report -f --dso perf --stdio -s srcfile,srcline':
44 probe_perf:libdw_addr2line
0.037260744 seconds time elapsed
0.025299000 seconds user
0.011918000 seconds sys
#
Adding probes to the other addr2line implementations (llvm__addr2line,
libbfd__addr2line and cmd__addr2line) I noticed some fallbacks to the
llvm one:
Performance counter stats for 'perf report -f --dso perf --stdio -s srcfile,srcline':
44 probe_perf:libdw_addr2line
23 probe_perf:llvm_addr2line
0 probe_perf:libbfd_addr2line
0 probe_perf:cmd_addr2line
Something to investigate further, but at least we don't fallback to the
cmd based one :-)
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Howard Chu <howardchu95@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephen Brennan <stephen.s.brennan@oracle.com>
Cc: Tony Jones <tonyj@suse.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The purpose of this workload is to gather samples in an inlined
function. This can be used to test whether inlined addr2line works
correctly.
Committer testing:
$ perf record perf test -w inlineloop 1
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.161 MB perf.data (4005 samples) ]
$ perf report --stdio --dso perf -s srcfile,srcline
#
# Total Lost Samples: 0
#
# Samples: 4K of event 'cpu/cycles/Pu'
# Event count (approx.): 5535180842
#
# Overhead Source File Source:Line
# ........ ............ ...............
#
99.04% inlineloop.c inlineloop.c:21
0.46% inlineloop.c inlineloop.c:20
#
$
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Howard Chu <howardchu95@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephen Brennan <stephen.s.brennan@oracle.com>
Cc: Tony Jones <tonyj@suse.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The addition of addr_location__exit() causes use-after put on the maps
and map references in the unwind info. Add the gets and then add the
map_symbol__exit() calls.
Fixes: 0dd5041c9a ("perf addr_location: Add init/exit/copy functions")
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Ian Rogers <irogers@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Howard Chu <howardchu95@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephen Brennan <stephen.s.brennan@oracle.com>
Cc: Tony Jones <tonyj@suse.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since ML-DSA is FIPS-approved, add the boot-time self-test which is
apparently required.
Just add a test vector manually for now, borrowed from
lib/crypto/tests/mldsa-testvecs.h (where in turn it's borrowed from
leancrypto). The SHA-* FIPS test vectors are generated by
scripts/crypto/gen-fips-testvecs.py instead, but the common Python
libraries don't support ML-DSA yet.
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: David Howells <dhowells@redhat.com>
Link: https://lore.kernel.org/r/20260107044215.109930-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Since the architecture-specific implementations of NH initialize memory
in assembly code, they aren't compatible with KMSAN as-is.
Fixes: 382de740759a ("lib/crypto: nh: Add NH library")
Link: https://lore.kernel.org/r/20260105053652.1708299-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
CONFIG_CRYPTO_NHPOLY1305_NEON, CONFIG_CRYPTO_NHPOLY1305_SSE2, and
CONFIG_CRYPTO_NHPOLY1305_AVX2 no longer exist. The equivalent
optimizations are now just enabled automatically when Adiantum support
is enabled. Update the fscrypt documentation accordingly.
Link: https://lore.kernel.org/r/20251211011846.8179-13-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Remove nhpoly1305 support from crypto_shash. It no longer has any user
now that crypto/adiantum.c no longer uses it.
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Link: https://lore.kernel.org/r/20251211011846.8179-11-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
This feature isn't useful in practice. Simplify and streamline the code
in the synchronous case, i.e. the case that actually matters, instead.
For example, by no longer having to support resuming the calculation
after an asynchronous return of the xchacha cipher, we can just keep
more of the state on the stack instead of in the request context.
Link: https://lore.kernel.org/r/20251211011846.8179-10-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Call the newer, easier-to-read functions memcpy_to_sglist() and
memcpy_from_sglist() directly instead of calling
scatterwalk_map_and_copy(). No change in behavior.
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Link: https://lore.kernel.org/r/20251211011846.8179-9-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
