These structs are never modified.
To prevent malicious or accidental modifications due to bugs,
mark them as const.
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add the __maybe_unused attribute to the function definitions and remove
the now-unnecessary forward declarations.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fold both assignments into a single max() call to simplify the code.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Return type of crypto_tfm_ctx() is void *. Remove explicit type cast.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace zynqmp with xilinx to have more generic name.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Don't enqueue requests which are supposed to fallback to s/w crypto.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Encrypt requests can be handled by driver when authsize is less than 16.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Convert FW error code to linux error code and remove dmesg error print for
authentication tag mismatch failure.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
zynqmp-aes-gcm updated to self discover, corresponding dt binding
can be mark deprecated.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
"alg" in zynqmp_aead_drv_ctx is single field union variable.
Remove unnecessary alg union from structure.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
For better maintainability move crypto related API's to
new zynqmp-crypto.c file.
Signed-off-by: Harsh Jain <h.jain@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the for loops with calls to unregister_aeads(),
unregister_ahashes(), and unregister_skciphers(), respectively. Return
'ret' immediately and remove the goto statements to simplify the error
handling code. No functional changes.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the for loop with a call to crypto_unregister_lskciphers().
Return 'ret' immediately and remove the goto statement to simplify the
error handling code. No functional changes.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the for loop with a call to crypto_unregister_skciphers().
Return 'ret' immediately and remove the goto statement to simplify the
error handling code. No functional changes.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the for loop with a call to crypto_unregister_shashes(). Return
'ret' immediately and remove the goto statement to simplify the error
handling code. No functional changes.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the for loop with a call to crypto_unregister_ahashes(). Return
'ret' immediately and remove the goto statement to simplify the error
handling code. No functional changes.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace the for loop with a call to crypto_unregister_acomps(). Return
'ret' immediately and remove the goto statement to simplify the error
handling code. No functional changes.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In esw_acl_ingress_lgcy_setup(), if esw_acl_table_create() fails,
the function returns directly without releasing the previously
created counter, leading to a memory leak.
Fix this by jumping to the out label instead of returning directly,
which aligns with the error handling logic of other paths in this
function.
Compile tested only. Issue found using a prototype static analysis tool
and code review.
Fixes: 07bab95026 ("net/mlx5: E-Switch, Refactor eswitch ingress acl codes")
Signed-off-by: Zilin Guan <zilin@seu.edu.cn>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Link: https://patch.msgid.link/20260120134640.2717808-1-zilin@seu.edu.cn
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
After commit bdce162f2e ("riscv: Use 64-bit variable for output in
__get_user_asm"), there is a warning when building for 32-bit RISC-V:
In file included from include/linux/uaccess.h:13,
from include/linux/sched/task.h:13,
from include/linux/sched/signal.h:9,
from include/linux/rcuwait.h:6,
from include/linux/mm.h:36,
from include/linux/migrate.h:5,
from mm/migrate.c:16:
mm/migrate.c: In function 'do_pages_move':
arch/riscv/include/asm/uaccess.h:115:15: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
115 | (x) = (__typeof__(x))__tmp; \
| ^
arch/riscv/include/asm/uaccess.h:198:17: note: in expansion of macro '__get_user_asm'
198 | __get_user_asm("lb", (x), __gu_ptr, label); \
| ^~~~~~~~~~~~~~
arch/riscv/include/asm/uaccess.h:218:9: note: in expansion of macro '__get_user_nocheck'
218 | __get_user_nocheck(x, ptr, __gu_failed); \
| ^~~~~~~~~~~~~~~~~~
arch/riscv/include/asm/uaccess.h:255:9: note: in expansion of macro '__get_user_error'
255 | __get_user_error(__gu_val, __gu_ptr, __gu_err); \
| ^~~~~~~~~~~~~~~~
arch/riscv/include/asm/uaccess.h:285:17: note: in expansion of macro '__get_user'
285 | __get_user((x), __p) : \
| ^~~~~~~~~~
mm/migrate.c:2358:29: note: in expansion of macro 'get_user'
2358 | if (get_user(p, pages + i))
| ^~~~~~~~
Add an intermediate cast to 'unsigned long', which is guaranteed to be the same
width as a pointer, before the cast to the type of the output variable to clear
up the warning.
Fixes: bdce162f2e ("riscv: Use 64-bit variable for output in __get_user_asm")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202601210526.OT45dlOZ-lkp@intel.com/
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Link: https://patch.msgid.link/20260121-riscv-fix-int-to-pointer-cast-v1-1-b83eebe57c76@kernel.org
Signed-off-by: Paul Walmsley <pjw@kernel.org>
The actual GMCH_CRTL lives in the host bridge aka. device 0,
but device 2 has a read-only mirror on i85x/i865+. Document
that fact.
Also remove the ancient tales about where the defines are used.
Those haven't been true in a long time.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-20-ville.syrjala@linux.intel.com
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
The PCI config space register defines in i915_drm.h are
a bit of a mess; Whitespace is all over the place, register
masks and values are defined in inconsistent ways.
Clean it up a bit.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-19-ville.syrjala@linux.intel.com
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Use the proper IS0_R name for the VGA input status register 0, instead
of using the VGA_MIS_W alias which is meant for write accesses to the
same address. Yes, VGA registers are weird.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-16-ville.syrjala@linux.intel.com
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Add a proper name for the "Input status register 0" IO address.
Currently we have some code that does reads using the aliasing
VGA_MSR_W define, making it unclear what register we're
actually reading.
v2: Remove stray '?'
Cc: Helge Deller <deller@gmx.de>
Cc: linux-fbdev@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251209075549.14051-1-ville.syrjala@linux.intel.com
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Acked-by: Helge Deller <deller@gmx.de>
On pre-g4x VGA registers are accessible via MMIO. Make use of
it so that we can avoid dealing with the VGA arbiter.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-14-ville.syrjala@linux.intel.com
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
VGA register are rather special since they either get accessed
via the global IO addresses, or possibly through MMIO on
pre-g4x platforms. Wrap all VGA register accesses in
intel_vga_{read,write}() to make it obvious where they get
accessed.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-13-ville.syrjala@linux.intel.com
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
intel_de_read8() is only needed for VGA register MMIO access
by the CRT code on gen2/3. Remove the redundant wakelock stuff,
and add a platform check to make sure this won't get used on
any platform where MMIO VGA register accesses don't work.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-11-ville.syrjala@linux.intel.com
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
We want our VGA register accesses to land on the correct GPU.
Check that the VGA routing is appropriately configured.
For the iGPU this just means the IO decode enable on the GPU, but
for dGPUs we also need the entire chain of bridges to forward the
VGA accesses.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-10-ville.syrjala@linux.intel.com
Acked-by: Jani Nikula <jani.nikula@intel.com>
intel_gmch_vga_set_state() is a complete lie on ILK+ because
the GMCH_CTRL register is locked and can't actually be written.
But we still need to remove the iGPU from the VGA arbitration
on iGPU+dGPU systems, or else Xorg performance will tank due
to the constant VGA arbiter accesses.
For VGA memory decode we can't turn off the PCI_COMMAND
memory deocde as that would disable even normal MMIO.
Instead we can disable just the VGA memory decode via
the VGA MSR register. And we can do that just once
when disabling the VGA plane. That way we don't have
to touch VGA registers anywhere else.
We can also inform the arbiter that we're no longer decoding
VGA memory. This will stop the arbiter from disabling all
memory decode for the iGPU via PCI_COMMAND (and thus breaking
everything) whenever some other GPU wants to own the VGA memory
accesses.
For IO we can disable all IO decode via the PCI_COMMAND
register, except around the few VGA register accesses that
we need to do in intel_vga_disable(). Unfortunately we can't
disable IO decode permanently as it makes some laptops (eg.
Dell Latitude E5400) hang during reboot/shutdown. One option
would be to re-enable IO decode from the poweroff hooks, but
that won't help the sysrq emergency reboot/shutdown since it
won't call said hooks. So let's try to keep IO decode in its
original setting unless we really need to disable it to
exclude the GPU from VGA arbitration.
I suppose we could keep frobbing GMCH_CTRL on pre-ILK, but
it seems better to not do it since it has other side effects
such as changing the class code of the PCI device.
For discrete GPUs we'll rely on the bridge control instead.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-9-ville.syrjala@linux.intel.com
Acked-by: Jani Nikula <jani.nikula@intel.com>
Avoid using the VGA arbiter during intel_vga_get() for iGPUs because
that will clobber the VGA routing for whatever external GPU is the
current VGA device. That will cause all reads from VGA memory to
come back as 0xff/white, and thus we get a white rectangle on screen
when the external GPU switches from vgacon to fbcon.
The iGPU has the highest VGA decode priority so it will steal all
VGA register accesses whenever its IO decoding is enabled. We'll only
keep the IO decode enabled for a short time so hopefully we don't
end up eating too many unrelated VGA register accesses.
For discrete GPUs we need all the bridges to have their VGA forwarding
bits correctly configured so we can't really avoid the VGA arbiter
there. Although we only do this stuff on dGPUs when the VGA plane was
actaully enabled, so the dGPU should be the current VGA device
and thus have VGA routed to it already anyway.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-8-ville.syrjala@linux.intel.com
Acked-by: Jani Nikula <jani.nikula@intel.com>
Turns out at least some systems (eg. HSW Lenovo ThinkCentre E73)
configure the VGA registers even when booting in UEFI mode. So
in order to avoid any issues with the MSR register we should
clean up the VGA registers anyway.
For now this mostly avoids the potential for unclaimed register
accesses due to the power well vs. MDA/CGA selection. But this
will become more important soon as we'll start to rely on the
MSR register to control VGA memory decode as well.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-7-ville.syrjala@linux.intel.com
Acked-by: Jani Nikula <jani.nikula@intel.com>
On some systems the BIOS will disable the VGA decode logic in the
iGPU (via GMCH_CTRL) when an external GPU is used as the primary
VGA device. In that case the iGPU will never claim any VGA register
accesses, and any access we do will in fact end up on the external
GPU. Don't go poking around in the other GPUs' registers.
Note that (at least on the g4x board where I tested this) the BIOS
forgets to set the VGACNTR VGA_DISP_DISABLE bit, and the reset
value for said bit is 0. That apparently prevents the pipes from
running, so we must still remember to set the bit, despite the VGA
plane was never actually enabled. On more modern platforms (hsw+
maybe?) the reset value for VGACNTR was changed to have
VGA_DISP_DISABLE already set.
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patch.msgid.link/20251208182637.334-6-ville.syrjala@linux.intel.com
Acked-by: Jani Nikula <jani.nikula@intel.com>
Enable flexible thread-to-queue mapping in batch I/O mode to support
arbitrary combinations of threads and queues, improving resource
utilization and scalability.
Key improvements:
- Support N:M thread-to-queue mapping (previously limited to 1:1)
- Dynamic buffer allocation based on actual queue assignment per thread
- Thread-safe queue preparation with spinlock protection
- Intelligent buffer index calculation for multi-queue scenarios
- Enhanced validation for thread/queue combination constraints
Implementation details:
- Add q_thread_map matrix to track queue-to-thread assignments
- Dynamic allocation of commit and fetch buffers per thread
- Round-robin queue assignment algorithm for load balancing
- Per-queue spinlock to prevent race conditions during prep
- Updated buffer index calculation using queue position within thread
This enables efficient configurations like:
- Any other N:M combinations for optimal resource matching
Testing:
- Added test_batch_02.sh: 4 threads vs 1 queue
- Added test_batch_03.sh: 1 thread vs 4 queues
- Validates correctness across different mapping scenarios
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add --batch/-b for enabling F_BATCH_IO.
Add batch_01 for covering its basic function.
Add stress_08 and stress_09 for covering stress test.
Add recovery test for F_BATCH_IO in generic_04 and generic_05.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
More tests need to be covered in existing generic tests, and default
45sec isn't enough, and timeout is often triggered, increase timeout
by adding setting file.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add support for UBLK_U_IO_FETCH_IO_CMDS to enable efficient batch
fetching of I/O commands using multishot io_uring operations.
Key improvements:
- Implement multishot UBLK_U_IO_FETCH_IO_CMDS for continuous command fetching
- Add fetch buffer management with page-aligned, mlocked buffers
- Process fetched I/O command tags from kernel-provided buffers
- Integrate fetch operations with existing batch I/O infrastructure
- Significantly reduce uring_cmd issuing overhead through batching
The implementation uses two fetch buffers per thread with automatic
requeuing to maintain continuous I/O command flow. Each fetch operation
retrieves multiple command tags in a single syscall, dramatically
improving performance compared to individual command fetching.
Technical details:
- Fetch buffers are page-aligned and mlocked for optimal performance
- Uses IORING_URING_CMD_MULTISHOT for continuous operation
- Automatic buffer management and requeuing on completion
- Enhanced CQE handling for fetch command completions
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Implement UBLK_U_IO_COMMIT_IO_CMDS to enable efficient batched
completion of I/O operations in the batch I/O framework.
This completes the batch I/O infrastructure by adding the commit
phase that notifies the kernel about completed I/O operations:
Key features:
- Batch multiple I/O completions into single UBLK_U_IO_COMMIT_IO_CMDS
- Dynamic commit buffer allocation and management per thread
- Automatic commit buffer preparation before processing events
- Commit buffer submission after processing completed I/Os
- Integration with existing completion workflows
Implementation details:
- ublk_batch_prep_commit() allocates and initializes commit buffers
- ublk_batch_complete_io() adds completed I/Os to current batch
- ublk_batch_commit_io_cmds() submits batched completions to kernel
- Modified ublk_process_io() to handle batch commit lifecycle
- Enhanced ublk_complete_io() to route to batch or legacy completion
The commit buffer stores completion information (tag, result, buffer
details) for multiple I/Os, then submits them all at once, significantly
reducing syscall overhead compared to individual I/O completions.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Implement support for UBLK_U_IO_PREP_IO_CMDS in the batch I/O framework:
- Add batch command initialization and setup functions
- Implement prep command queueing with proper buffer management
- Add command completion handling for prep and commit commands
- Integrate batch I/O setup into thread initialization
- Update CQE handling to support batch commands
The implementation uses the previously established buffer management
infrastructure to queue UBLK_U_IO_PREP_IO_CMDS commands. Commands are
prepared in the first thread context and use commit buffers for
efficient command batching.
Key changes:
- ublk_batch_queue_prep_io_cmds() prepares I/O command batches
- ublk_batch_compl_cmd() handles batch command completions
- Modified thread setup to use batch operations when enabled
- Enhanced buffer index calculation for batch mode
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add the foundational infrastructure for UBLK_F_BATCH_IO buffer
management including:
- Allocator utility functions for small sized per-thread allocation
- Batch buffer allocation and deallocation functions
- Buffer index management for commit buffers
- Thread state management for batch I/O mode
- Buffer size calculation based on device features
This prepares the groundwork for handling batch I/O commands by
establishing the buffer management layer needed for UBLK_U_IO_PREP_IO_CMDS
and UBLK_U_IO_COMMIT_IO_CMDS operations.
The allocator uses CPU sets for efficient per-thread buffer tracking,
and commit buffers are pre-allocated with 2 buffers per thread to handle
overlapping command operations.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
