This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci
This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.
Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.
Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
- swiotlb: added new configuration option for the default pool size
(Jagadeesh Pagadala) and reduced overhead for high watermark tracking
(chenhuguanshen)
- minor code cleanups and improvements (Vova Sharaienko, Honglei Huang
and Marek Szyprowski)
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQSrngzkoBtlA8uaaJ+Jp1EFxbsSRAUCaoxGQgAKCRCJp1EFxbsS
RFPEAP0eo9usjFcvh0YKTPh6/mXgqxRuTNQZ7i+2lRGEczKcJQEA7mwkgwpiOaKn
f++mMVOmsPvl2Y7r/5XqBWywwhyygA0=
=X8pY
-----END PGP SIGNATURE-----
mergetag object 04a19b35dc
type commit
tag dma-mapping-7.3-2026-08-24-2
tagger Marek Szyprowski <m.szyprowski@samsung.com> 1787582472 +0200
second dma-mapping update for Linux 7.3:
- important dma-mapping update for confidential-computing, which adds
proper tracking of the shared DMA state through direct, pool and swiotlb
paths (Aneesh Kumar K.V)
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQSrngzkoBtlA8uaaJ+Jp1EFxbsSRAUCaoxYzgAKCRCJp1EFxbsS
RM9bAP4mJuHHzj2DqsKV7QX19uhyzmsHIg+ecjBNRaOdUAgelQD9FsaG/fwrZnRT
y89H0QUErqLsdmkDqV0zsXfaGzWY4gk=
=dKjS
-----END PGP SIGNATURE-----
Merge tags 'dma-mapping-7.3-2026-08-24' and 'dma-mapping-7.3-2026-08-24-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping updates from Marek Szyprowski:
- swiotlb:
- new configuration option for the default pool size
(Jagadeesh Pagadala)
- reduce overhead for high watermark tracking (chenhuguanshen)
- minor code cleanups and improvements (Vova Sharaienko, Honglei Huang
and Marek Szyprowski)
- add proper tracking of the shared DMA state through direct, pool and
swiotlb paths (Aneesh Kumar K.V)
This is important for confidential-computing
* tag 'dma-mapping-7.3-2026-08-24' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
dma/swiotlb: decouple high watermark tracking from CONFIG_DEBUG_FS
MAINTAINERS: update tree for DMA MAPPING HELPERS
dma/swiotlb: introduce Kconfig option for compile-time default pool size
dma-direct: Improve readability of the dma_direct_map_sg() for P2PDMA case
iommu/dma: simplify dma_iova_destroy() and drop the free_iova helper
dma-coherent: use KiB in DMA allocation logs
dma-coherent: fix spacing coding style issue
* tag 'dma-mapping-7.3-2026-08-24-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux: (23 commits)
swiotlb: remove unused SWIOTLB_FORCE flag
dma: swiotlb: handle set_memory_decrypted() failures
dma: swiotlb: free dynamic pools from process context
dma-direct: rename ret to cpu_addr in alloc helpers
dma-direct: select DMA address encoding from __DMA_ATTR_ALLOC_CC_SHARED
dma-direct: set decrypted flag for remapped DMA allocations
dma-direct: make dma_direct_map_phys() honor DMA_ATTR_CC_SHARED
dma-direct: Move dma_direct_map_phys() to dma/direct.c
dma-direct: pass attrs to dma_capable() for DMA_ATTR_CC_SHARED checks
dma-mapping: make dma_pgprot() honor __DMA_ATTR_ALLOC_CC_SHARED
dma: swiotlb: track pool encryption state and honor DMA_ATTR_CC_SHARED
dma: swiotlb: pass mapping attributes by reference
dma-pool: track decrypted atomic pools and select them via attrs
dma-direct: use __DMA_ATTR_ALLOC_CC_SHARED in alloc/free paths
dma-mapping: Add internal shared allocation attribute
coco: arm64: s390: powerpc: Mark secure guests with CC_ATTR_GUEST_MEM_ENCRYPT
dma-direct: swiotlb: handle swiotlb alloc/free outside __dma_direct_alloc_pages
s390: Expose protected virtualization through cc_platform_has()
swiotlb: Preserve allocation virtual address for dynamic pools
dma: free atomic pool pages by physical address
...
Small update this cycle:
- Formal API for "no iommu" mode in VFIO. iommufd now works in this
environment and provides page pinning and phyiscal address services to
userspace. This avoids nasty fragile tricks with mprotect and pgmap
- Fix sykzaller crash racing change_process with map_pages
- Various skyzkaller/AI fixes for the selftests
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRRRCHOFoQz/8F5bUaFwuHvBreFYQUCaoS93gAKCRCFwuHvBreF
YeG7AP0QF6RQbPmH0y53sxtXhupaOsKZVdtbxM6FA4N3NPlUrQD7BD5zS3AN8T7X
pCuiY7uwyiHtmfBXaXDtOyPQrRXlrw8=
=fgTq
-----END PGP SIGNATURE-----
Merge tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd
Pull iommufd updates from Jason Gunthorpe:
"One small feature this cycle, the noiommu mode is useful in
single-purpose VMs running something like DPDK. It avoids the double
translation overhead and it seems to be commonly used with some hacks.
Summary:
- Formal API for "no iommu" mode in VFIO. iommufd now works in this
environment and provides page pinning and phyiscal address services
to userspace. This avoids nasty fragile tricks with mprotect and
pgmap
- Fix sykzaller crash racing change_process with map_pages
- Various skyzkaller/AI fixes for the selftests"
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd:
iommufd: Fix UAF in selftest IOPF reporting
iommu/iommufd: Fix NULL pointer deref in iommufd_ioas_change_process when racing with iopt_map_file_pages
Documentation: Update VFIO NOIOMMU mode
vfio: Enable cdev noiommu mode under iommufd
iommufd: Add an ioctl to query PA from IOVA for noiommu mode
iommufd: Allow binding to a noiommu device
iommufd: Move igroup allocation to a function
iommufd: Support a HWPT without an iommu driver for noiommu
iommufd: Simplify iommufd_device_remove_vdev()
iommufd: Fix grammar and spelling in comments
iommu: Fix dev_iommu memory leak when device_add fails in iommu_mock_device_add
iommufd/selftest: Fix dmabuf leak in iommufd_test_dmabuf_get()
iommufd/selftest: Avoid selftest dirty bitmap size wrap
- ARM SMMUv2:
- Device-tree binding updates for Qualcomm Eliza, Maili, Shikra and
IPQ9650 SoCs.
- Add support for Qualcomm SM8450.
- Numerous fixes for lifetime and ordering issues found by Sashiko in
the Qualcomm driver.
- ARM SMMUv3:
- Fix interrupt type in device-tree binding example for NVIDIA CMDQV.
- Numerous fixes for issues identified by Sashiko in the NVIDIA CMDQV
driver.
- Work around TLB erratum T264-SMMU-3 on Tegra264 by repeating the
invalidation sequence.
- Add support for HAFT (hardware access flag in table entries) when
using SVA.
- Probe for 52-bit addressing with a page size smaller than 64k ('DS')
but don't do anything with it for now.
- Minor driver improvements (remove sort_nonatomic(), use
readl_relaxed_poll_timeout_atomic(), fix IOPF teardown ordering).
- Intel VT-d:
- Consolidation of complex enablement logic into a clean, priority-based
state machine.
- Support for the DMA_REMAP_OPT_OUT flag from the VT-d v5.2
specification.
- An update to cache_tag_flush_devtlb_psi() to use full-range constants
instead of modifying shared variables for CACHE_TAG_NESTING_DEVTLB.
- A fix for the UCTP context-table slot when copying root entries.
- Fixes for several pre-existing issues reported by Sashiko.
- General code cleanup and refinement.
- AMD IOMMU:
- Add SNP page-mode-0 support, enabling passthrough, v2 DMA page
tables and host SVA on supporting systems.
- Fix invalid PPR handling, COMPLETE_PPR responses and guest-mode
reporting.
- Improve Southbridge IOAPIC validation and remove the dependency on
hard-coded device IDs.
- Fix PCI-device lifetime, debugfs and diagnostic issues.
- IOMMU core and IOMMUFD:
- Restore serialization of the shared MSI-page list.
- Fix SVA-handle publication and several IOMMUFD reference and error
path leaks.
- Return the expected zero result for invalid generic page-table
translations.
- Allocate per-CPU IOVA magazines lazily to reduce memory use on
large systems.
- PCI ATS:
- Make VF support checks account for the associated PF and validate
that VF and PF Smallest Translation Unit settings agree.
- Platform drivers:
- Fix Qualcomm runtime-PM, probe unwind, fault reporting and page
table initialization races.
- Rework Rockchip state handling and fix clock, probe and stale-fault
handling.
- Fix smaller issues in the MSM and MediaTek drivers.
- Device-tree bindings:
- Add new Qualcomm SMMU compatibles, convert the OMAP IOMMU binding
to YAML, and fix the Tegra264 CMDQV interrupt example.
- Various smaller cleanups, documentation fixes and a Rust IOMMU
safety/readability improvement.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEr9jSbILcajRFYWYyK/BELZcBGuMFAmqDJowACgkQK/BELZcB
GuOPNw//aE9PbA4JWkYou6W2kVQnCowxvX87yE0ZDA6WAr9dOGJh15ZgIqdBMZEo
L28ZtaaH8BXM/DmMuQwVMinuvQ4jq5PcFbgu+erfrN/1PpT9fSE8eLPKX63b3fng
Qgv2gf2lAW6aTmMr8N8XKKlZUOLKfFPjOKzeqz77gTu2Z/FspEvM14QgoXmHum68
xtaEtrV11Do69CgGUCtXMunjpYQ3PHwQafjcvlsYaczWZ/NjTU/snjAWaqgpVBHh
1A8tdwbR5CEsLEHbFLeeQMuZJlEsTedv9lE95QZSlHsoqEkrrgkC930w+CaK+a3C
rd5chPEBwltz6TmzIoAF8PubzANiUNCZXnFL7U3GTkOLqwfr3l0T8lO0NWDhVqU/
L3jSGbFc/QOqVH4nygdh28rh7DHfhkpa1NNHtbFgNdl2W5UDseCnok/2MAvn8+wV
gflGhtsLLxbNis9FMS+w7QTOgHkJDq2eWCJp+uaDtZtY2IsVEy9Qi5dxS1VQ9i9T
jzVz6gsDdP7uAJBXByZcns7acjC6q2X2iLFhcU+mAk2H68rFZj411WEz/eRw8xvi
RGA8nY9DCVuiIo2NR6dE0r1DhTzI+eJCS/F3xeKyjxDNM6P4EMi53PvKAaIL/wXt
JF7FsfFlmlSgGeDCoin6CRHBvb41drpLwczs6+qURmL1X9iNi9s=
=V5FT
-----END PGP SIGNATURE-----
Merge tag 'iommu-updates-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux
Pull iommu updates from Joerg Roedel:
"ARM SMMUv2:
- Device-tree binding updates for Qualcomm Eliza, Maili, Shikra and
IPQ9650 SoCs
- Add support for Qualcomm SM8450
- Numerous fixes for lifetime and ordering issues found by Sashiko in
the Qualcomm driver
ARM SMMUv3:
- Fix interrupt type in device-tree binding example for NVIDIA CMDQV
- Numerous fixes for issues identified by Sashiko in the NVIDIA CMDQV
driver
- Work around TLB erratum T264-SMMU-3 on Tegra264 by repeating the
invalidation sequence
- Add support for HAFT (hardware access flag in table entries) when
using SVA
- Probe for 52-bit addressing with a page size smaller than 64k
('DS') but don't do anything with it for now
- Minor driver improvements (remove sort_nonatomic(), use
readl_relaxed_poll_timeout_atomic(), fix IOPF teardown ordering)
Intel VT-d:
- Consolidation of complex enablement logic into a clean,
priority-based state machine
- Support for the DMA_REMAP_OPT_OUT flag from the VT-d v5.2
specification
- An update to cache_tag_flush_devtlb_psi() to use full-range
constants instead of modifying shared variables for
CACHE_TAG_NESTING_DEVTLB
- A fix for the UCTP context-table slot when copying root entries
- Fixes for several pre-existing issues reported by Sashiko
- General code cleanup and refinement
AMD IOMMU:
- Add SNP page-mode-0 support, enabling passthrough, v2 DMA page
tables and host SVA on supporting systems
- Fix invalid PPR handling, COMPLETE_PPR responses and guest-mode
reporting
- Improve Southbridge IOAPIC validation and remove the dependency on
hard-coded device IDs
- Fix PCI-device lifetime, debugfs and diagnostic issues
IOMMU core and IOMMUFD:
- Restore serialization of the shared MSI-page list
- Fix SVA-handle publication and several IOMMUFD reference and error
path leaks
- Return the expected zero result for invalid generic page-table
translations
- Allocate per-CPU IOVA magazines lazily to reduce memory use on
large systems
PCI ATS:
- Make VF support checks account for the associated PF and validate
that VF and PF Smallest Translation Unit settings agree
Platform drivers:
- Fix Qualcomm runtime-PM, probe unwind, fault reporting and page
table initialization races
- Rework Rockchip state handling and fix clock, probe and stale-fault
handling
- Fix smaller issues in the MSM and MediaTek drivers
Device-tree bindings:
- Add new Qualcomm SMMU compatibles, convert the OMAP IOMMU binding
to YAML, and fix the Tegra264 CMDQV interrupt example
Various smaller cleanups, documentation fixes and a Rust IOMMU
safety/readability improvement"
* tag 'iommu-updates-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux: (93 commits)
iommu/amd: Add SNP page mode 0 support
iommu/amd: Fix GN bit setting in COMPLETE_PPR_REQUEST command
iommu/amd: Rate limit INVALID_PPR_REQUEST error logging
iommu/amd: Fix missing CMD_COMPLETE_PPR response for invalid PPR requests
iommu/amd: Introduce PPR_TAG_LAST_PAGE() macro
iommu/amd: Fix incorrect device ID in invalid PASID error message
iommu/vt-d: Flush context cache with correct SID when tearing down aliases
iommu/vt-d: Tear down scalable-mode context on probe failure
iommu/vt-d: Fix iopf_refcount leak on RID domain replacement
iommu/vt-d: Clear Present bit before tearing down copied context entry
iommu/vt-d: Fix copied_tables bitmap leak on error in copy_translation_tables
iommu/vt-d: Cache max domain ID to avoid redundant calculation
iommu/vt-d: Support the new DMA_REMAP_OPT_OUT flag bit
iommu/vt-d: Remove dmar_disabled
iommu/vt-d: Remove the 'force_on' variable
iommu/vt-d: Call dmar_can_force_on() for tboot opt-in
iommu/vt-d: Use dmar_can_force_on() for platform opt-in
iommu/vt-d: Consolidate dmar policy management and force_on logic
iommu/vt-d: Remove dead code when CONFIG_INTEL_IOMMU is not set
iommu/vt-d: Force requesting ACS when tboot is enabled
...
- container_of:
- Apply typeof_member(), remove the local __mptr variable to eliminate
variable shadowing warnings on nested container_of() calls, and remove
unnecessary parentheses.
- core:
- Add driver name to probe debug print for initcall_debug.
- Avoid repeatedly printing the same 'Fixed dependency cycle' log.
- Unwind device_add() on attribute creation failure in
attribute_container_add_class_device().
- Remove statistics group if encryption group creation fails in
transport_add_class_device().
- debugfs:
- Fix lockdown check for mmap_prepare().
- Warn if file creation failed due to uninitialized debugfs.
- device property:
- Implement fw_devlink support for software nodes by adding
software_node_add_links(), which creates fwnode links from DEV_PROP_REF
properties to enable automatic probe ordering. Add kunit-managed fwnode
helpers and test coverage.
- Fix infinite loop in fwnode_for_each_child_node() when the secondary fwnode
has more than one child. Add test cases.
- Fix out-of-bounds access in software_node_get_reference_args() when called
with index -1 (UINT_MAX).
- Refactor to use RAII approach with __free().
- Add Bartosz Golaszewski as software node reviewer.
- firmware loader:
- Fix race where a sysfs fallback request can complete before being queued as
pending, leading to a use-after-free on the next fallback request.
- Reject 0-size built-in firmware and fail the build on empty firmware files
in CONFIG_EXTRA_FIRMWARE.
- kobject:
- Provide __KOBJ_ATTR() and __KOBJ_ATTR_RO/WO() initialization macros and
allow the constification of kobject attributes, enabling them to reside in
read-only memory.
- platform:
- Provide platform_device_set_of_node(), platform_device_set_fwnode(), and
platform_device_set_of_node_from_dev() helpers that encapsulate firmware
node reference counting for dynamically allocated platform devices. Convert
all in-tree users that manually assigned dev.of_node or dev.fwnode, fixing a
pre-existing refcount bug in powermac. Switch to counting references of all
firmware node types, not only OF nodes.
- Unify the release path for dynamically allocated platform devices by
removing platform_device_release_full(). Amend the fwnode setter API
contract to warn if a primary software node is overwritten. Add KUnit tests
for correct software node removal on device unregistration.
- Rust:
- Auxiliary:
- Add registration_data_with() closure-based API for invariant ForLt types.
- Debugfs:
- Migrate BinaryWriter and BinaryReaderMut trait requirements from
kernel::transmute traits to zerocopy traits.
- Device:
- Add BoundInternal device context and InternalBoundContext trait for bus
abstractions that need internal access to a bound device.
- Make the lifetime on Core and CoreInternal invariant to prevent coercion
to shorter lifetimes.
- Devres:
- Fix race between concurrent revokers where the losing revoker could return
before the winning revoker finished dropping the inner data, causing
use-after-free.
- Ensure revocation is complete before the device finishes unbinding by
making the synchronization bidirectional.
- Add DevresLt<F: ForLt>, a wrapper around Devres that shortens 'static back
to the caller's borrow scope. Implement ForLt and CovariantForLt for Bar,
IoMem, and ExclusiveIoMem.
- Driver:
- Switch from index-based to pointer-based device ID info lookup, storing
static references in driver_data. Centralize device ID handling in
device_id.rs, removing the open-coded ACPI/OF matching logic and duplicate
ID table from driver.rs.
- I/O:
- Make I/O regions typed (with a dynamically-sized Region type for the
existing untyped case), create view types representing subregions of a
mapped I/O region, and add io_project!() for safely creating subviews.
- Split Io into a base trait (IoBase) and an extension trait (Io) with a
blanket implementation, preventing implementers from overriding provided
methods that unsafe code relies on.
- Add a SysMem backend for shared system memory with volatile access, and
make Coherent implement Io via an I/O view type. Add IoSysMap as sum type
of Mmio and SysMem. Add copying methods (memcpy_{from,to}io()) and
read_val()/write_val() for typed access.
- Replace dma_read!()/dma_write!() with io_read!()/io_write!() for
primitives and copying methods for aggregates; drop the old macros.
Convert nova-core to use I/O projection.
- Fix internal shortcut rule dispatch in the register!() macro, remove
unused rule arguments, and use path fragments for alias destinations.
- IRQ:
- Make irq::Registration compatible with lifetime-bound drivers by
removing the 'static bound on Handler/ThreadedHandler and
replacing Devres<RegistrationInner> with direct
request_irq()/free_irq() calls. Handlers can now directly own
lifetime-bound device resources.
- PCI:
- Convert IrqVectorRegistration to a lifetime-annotated owning type,
giving drivers explicit control over the allocation lifetime.
IrqVector embeds a resolved IrqRequest, making the conversion
infallible. Remove the redundant
request_irq()/request_threaded_irq() wrappers from pci::Device.
- Add pci_irq_type() C helper and expose it via irq_type() on
IrqVectorRegistration and IrqVector, returning PCI_IRQ_MSIX, PCI_IRQ_MSI,
or PCI_IRQ_INTX.
- Mark pci::Device refcount methods inline.
- Serdev:
- Add Rust abstractions for the serial device bus, including serdev::Driver
trait, serdev::Device wrapping struct serdev_device, and serdev::Adapter
implementing RegistrationOps. Includes a sample driver. Markus Probst
takes over as serdev maintainer for both C and Rust code.
- Misc:
- Split ForLt into a base trait (providing the Of<'a> GAT) and an unsafe
CovariantForLt subtrait guaranteeing covariance, enabling invariant types
(e.g. those containing Mutex<&'bound T>) to participate in the ForLt
abstraction.
- Fix Coherent read past EOF returning -ERANGE instead of zero.
- Fix firmware example UB by avoiding null-pointer ARef.
- misc:
- Avoid iattr allocation in kernfs listxattr by using kernfs_iattrs_noalloc().
- Unregister SoC bus on early device registration failure.
- Remove unused DMA_FENCE_TRACE Kconfig symbol.
- Fix /sys/module path in comment.
- Refactor ISA bus init to remove nested blocks.
- Remove redundant nodemask clears in numa_init().
- Add kernel-doc for fwnode_operations and sys_soc.h, mark internal property
data as private for kernel-doc, and add property.h/fwnode.h to driver-api
infrastructure docs.
- Add MAINTAINERS entry for sys_soc.h.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQS2q/xV6QjXAdC7k+1FlHeO1qrKLgUCaoSPugAKCRBFlHeO1qrK
LlBrAP958EaYZVyfAsqq0EyHv86ahXKWoDDvkwbRUZA5CfV9tAD/bMWVtPLRYaaq
eNEZT7p5w3Haezh61H3c5zpuwFNMTA0=
=VRIa
-----END PGP SIGNATURE-----
Merge tag 'driver-core-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core
Pull driver core updates from Danilo Krummrich:
"container_of:
- Apply typeof_member(), remove the local __mptr variable to
eliminate variable shadowing warnings on nested container_of()
calls, and remove unnecessary parentheses
core:
- Add driver name to probe debug print for initcall_debug
- Avoid repeatedly printing the same 'Fixed dependency cycle' log
- Unwind device_add() on attribute creation failure in
attribute_container_add_class_device()
- Remove statistics group if encryption group creation fails in
transport_add_class_device()
debugfs:
- Fix lockdown check for mmap_prepare()
- Warn if file creation failed due to uninitialized debugfs
device property:
- Implement fw_devlink support for software nodes by adding
software_node_add_links(), which creates fwnode links from
DEV_PROP_REF properties to enable automatic probe ordering. Add
kunit-managed fwnode helpers and test coverage
- Fix infinite loop in fwnode_for_each_child_node() when the
secondary fwnode has more than one child. Add test cases
- Fix out-of-bounds access in software_node_get_reference_args() when
called with index -1 (UINT_MAX)
- Refactor to use RAII approach with __free()
- Add Bartosz Golaszewski as software node reviewer
firmware loader:
- Fix race where a sysfs fallback request can complete before being
queued as pending, leading to a use-after-free on the next fallback
request
- Reject 0-size built-in firmware and fail the build on empty
firmware files in CONFIG_EXTRA_FIRMWARE
kobject:
- Provide __KOBJ_ATTR() and __KOBJ_ATTR_RO/WO() initialization macros
and allow the constification of kobject attributes, enabling them
to reside in read-only memory
platform:
- Provide platform_device_set_of_node(), platform_device_set_fwnode(),
and platform_device_set_of_node_from_dev() helpers that encapsulate
firmware node reference counting for dynamically allocated platform
devices
Convert all in-tree users that manually assigned dev.of_node or
dev.fwnode, fixing a pre-existing refcount bug in powermac. Switch
to counting references of all firmware node types, not only OF
nodes
- Unify the release path for dynamically allocated platform devices
by removing platform_device_release_full(). Amend the fwnode setter
API contract to warn if a primary software node is overwritten. Add
KUnit tests for correct software node removal on device
unregistration
Rust:
- Auxiliary:
- Add registration_data_with() closure-based API for invariant
ForLt types
- Debugfs:
- Migrate BinaryWriter and BinaryReaderMut trait requirements
from kernel::transmute traits to zerocopy traits
- Device:
- Add BoundInternal device context and InternalBoundContext trait
for bus abstractions that need internal access to a bound
device.
- Make the lifetime on Core and CoreInternal invariant to prevent
coercion to shorter lifetimes
- Devres:
- Fix race between concurrent revokers where the losing revoker
could return before the winning revoker finished dropping the
inner data, causing use-after-free.
- Ensure revocation is complete before the device finishes
unbinding by making the synchronization bidirectional.
- Add DevresLt<F: ForLt>, a wrapper around Devres that shortens
'static back to the caller's borrow scope. Implement ForLt and
CovariantForLt for Bar, IoMem, and ExclusiveIoMem
- Driver:
- Switch from index-based to pointer-based device ID info lookup,
storing static references in driver_data. Centralize device ID
handling in device_id.rs, removing the open-coded ACPI/OF
matching logic and duplicate ID table from driver.rs
- I/O:
- Make I/O regions typed (with a dynamically-sized Region type
for the existing untyped case), create view types representing
subregions of a mapped I/O region, and add io_project!() for
safely creating subviews.
- Split Io into a base trait (IoBase) and an extension trait (Io)
with a blanket implementation, preventing implementers from
overriding provided methods that unsafe code relies on.
- Add a SysMem backend for shared system memory with volatile
access, and make Coherent implement Io via an I/O view type.
Add IoSysMap as sum type of Mmio and SysMem. Add copying
methods (memcpy_{from,to}io()) and read_val()/write_val() for
typed access.
- Replace dma_read!()/dma_write!() with io_read!()/io_write!()
for primitives and copying methods for aggregates; drop the old
macros. Convert nova-core to use I/O projection.
- Fix internal shortcut rule dispatch in the register!() macro,
remove unused rule arguments, and use path fragments for alias
destinations
- IRQ:
- Make irq::Registration compatible with lifetime-bound drivers
by removing the 'static bound on Handler/ThreadedHandler and
replacing Devres<RegistrationInner> with direct
request_irq()/free_irq() calls. Handlers can now directly own
lifetime-bound device resources
- PCI:
- Convert IrqVectorRegistration to a lifetime-annotated owning
type, giving drivers explicit control over the allocation
lifetime. IrqVector embeds a resolved IrqRequest, making the
conversion infallible. Remove the redundant
request_irq()/request_threaded_irq() wrappers from pci::Device.
- Add pci_irq_type() C helper and expose it via irq_type() on
IrqVectorRegistration and IrqVector, returning PCI_IRQ_MSIX,
PCI_IRQ_MSI, or PCI_IRQ_INTX.
- Mark pci::Device refcount methods inline
- Serdev:
- Add Rust abstractions for the serial device bus, including
serdev::Driver trait, serdev::Device wrapping struct
serdev_device, and serdev::Adapter implementing
RegistrationOps. Includes a sample driver. Markus Probst takes
over as serdev maintainer for both C and Rust code
- Misc:
- Split ForLt into a base trait (providing the Of<'a> GAT) and an
unsafe CovariantForLt subtrait guaranteeing covariance,
enabling invariant types (e.g. those containing Mutex<&'bound T>)
to participate in the ForLt abstraction.
- Fix Coherent read past EOF returning -ERANGE instead of zero.
- Fix firmware example UB by avoiding null-pointer ARef
misc:
- Avoid iattr allocation in kernfs listxattr by using
kernfs_iattrs_noalloc().
- Unregister SoC bus on early device registration failure.
- Remove unused DMA_FENCE_TRACE Kconfig symbol.
- Fix /sys/module path in comment.
- Refactor ISA bus init to remove nested blocks.
- Remove redundant nodemask clears in numa_init().
- Add kernel-doc for fwnode_operations and sys_soc.h, mark
internal property data as private for kernel-doc, and add
property.h/fwnode.h to driver-api infrastructure docs.
- Add MAINTAINERS entry for sys_soc.h"
* tag 'driver-core-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core: (129 commits)
rust: pci: expose the allocated interrupt type
PCI: Add pci_irq_type() to query the allocated interrupt type
rust: pci: remove request_irq() and request_threaded_irq() from Device
rust: pci: resolve IRQ in index() and embed IrqRequest in IrqVector
rust: pci: convert IrqVectorRegistration to a lifetime-managed owning type
kernfs: avoid iattr allocation in listxattr
rust: serdev: use ThisModule::as_ptr() instead of field access
ACPI/IORT: use platform_device_set_fwnode()
ACPI/APMT: use platform_device_set_fwnode()
firmware_loader: do not queue completed sysfs fallback requests
rust: pci: Mark Device refcount methods inline
rust: irq: make Registration compatible with lifetime-bound drivers
rust: net/phy: remove expansion from doc
rust: dma: return zero for Coherent reads past EOF
rust: io: register: use path fragment for alias destination
rust: io: register: remove unused rule arguments
rust: io: register: dispatch shortcut rules internally
MAINTAINERS: add sys_soc.h to DRIVER CORE
rust: debugfs: remove unsafe blocks from traits impl for Vec
rust: debugfs: migrate debugfs traits requirements to zerocopy
...
ACPI:
- Combine reads of AMU counters into a single FFH feedback counter op.
Confidential computing:
- Fix smp_processor_id() in preemptible context when retrieving an
attestation token inside a realm.
- Convert pKVM over to a "CC platform".
- Clean-up our SWIOTLB configuration in preparation for reworking the
handling of encrypted/decryped DMA buffers in the dma-mapping tree.
CPU errata handling:
- Work around broken device memory ordering on NVIDIA Olympus cores.
- Fix broken 'nospectre_bhb' command-line option.
- Select the idle loop backend instruction on the command-line.
CPU features:
- Replace our BBML2-noabort feature with the new architectural BBML3
feature.
- Disable in-kernel BTI for recent versions of Clang due to issues with
livepatch that are still being investigated.
- Clean-up documentation describing which ID register fields are exposed
to userspace.
Interrupts:
- Preliminary work towards supporting FEAT_NMI, which cleans up our
IRQ entry code and fixes some latent issues with pseudo-NMI.
- Support for an SDEI backend to trigger an NMI backtrace.
Memory management:
- Treat all devices as coherent when CLIDR_EL1.LoC == 0.
- Fix no-map handling of sub-page-sized regions.
- Second attempt at unmapping the linear aliases of the kernel data and
bss sections.
- Fix EFI runtime calls when software-PAN is enabled.
Miscellaneous:
- Add Mark Rutland as a reviewer!
- Tidy-up our futex cmpxchg logic when using the new LSUI instructions.
- Drop the requirement on DYNAMIC_FTRACE_WITH_CALL_OPS when selecting
HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS.
- Fix a false-positive KCSCAN splat in the delay loop.
- Use a portable typedef for 128-bit scalar types in our UAPI headers.
- Non-critical fixes for Sashiko reports all over.
MPAM:
- Hook MPAM memory bandwidth counters into resctrl's counter assignment
interface.
- Fix a quirk in the MPAM bandwidth counting on Nvidia T241 so that it
also applies to 63 bit counters.
Perf:
- Workarounds for hardware issues in the CMN-S3 PMU (Graviton 5) and CPU
PMU (NVIDIA Olympus again!)
- Add support for the DDR PMU on Marvell CN20K SoCs.
- Add support for Picoheart implementations of the DCW PCIe PMU.
- Add support for Channel/Rank/Bank filtering in the CXL PMU driver.
- Add support for 64-bit counters in the CSPMU device.
- Add support for revision 2 of the CMN S3 PMU.
Ptrace:
- Fix a decade-old bug in our handling of seccomp and tracing on syscall
entry.
- Fix regset handling for inactive SVE and SSVE registers.
Selftests
- Add some tests for the decade-old bug that we just tried to fix in our
syscall entry path.
- Fix SVE test crash on SME-only CPUs.
-----BEGIN PGP SIGNATURE-----
iQFEBAABCgAuFiEEPxTL6PPUbjXGY88ct6xw3ITBYzQFAmp+9HkQHHdpbGxAa2Vy
bmVsLm9yZwAKCRC3rHDchMFjNEjXB/4y8KueLwgMOcpjzJom6ujmE2Ab7IOI+I5v
rYQkGDdHLcQd0WW9gtFXoxlq/EUcWuVCGAdOF3PxkUmVhHJC45ZNSmBGS9/LbU3f
AqeFskSE46D2gOdMXSscwGAQ+OiQ6WlSSgqy2dU1ZfTXy7ioxydWVUtvmqMmB85O
ctN9Ocr3zAZuptuyLJurCJNnyCuzd55SIuy+WKU6NIpWseiIqiLEb/6HFbWcw6iV
vVZjD/Je5my8Gfjuu12z7kJ6o5jwGNpI8U7Z0b68ESUG3b5fh9E2zVLIq0fvn83B
rjPqg/Wj2TVGag/+yZgafLwQ/7111hhkdf1bPBitClx4GxOoZEjG
=Iypw
-----END PGP SIGNATURE-----
Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Will Deacon:
"There's a reasonable amount of stuff here, including a bunch of
updates to the perf PMU drivers and some MPAM updates to expose the
memory bandwidth counters via resctrl.
On the architecture side, some highlights include support for BBML3
and steps towards support for an architectural NMI solution, all
wrapped up in a web of fixes for latent issues identified by Sashiko.
ACPI:
- Combine reads of AMU counters into a single FFH feedback counter op
Confidential computing:
- Fix smp_processor_id() in preemptible context when retrieving an
attestation token inside a realm
- Convert pKVM over to a "CC platform"
- Clean-up our SWIOTLB configuration in preparation for reworking the
handling of encrypted/decryped DMA buffers in the dma-mapping tree
CPU errata handling:
- Work around broken device memory ordering on NVIDIA Olympus cores
- Fix broken 'nospectre_bhb' command-line option
- Select the idle loop backend instruction on the command-line
CPU features:
- Replace our BBML2-noabort feature with the new architectural BBML3
feature
- Disable in-kernel BTI for recent versions of Clang due to issues
with livepatch that are still being investigated
- Clean-up documentation describing which ID register fields are
exposed to userspace
Interrupts:
- Preliminary work towards supporting FEAT_NMI, which cleans up our
IRQ entry code and fixes some latent issues with pseudo-NMI
- Support for an SDEI backend to trigger an NMI backtrace
Memory management:
- Treat all devices as coherent when CLIDR_EL1.LoC == 0
- Fix no-map handling of sub-page-sized regions
- Second attempt at unmapping the linear aliases of the kernel data
and bss sections
- Fix EFI runtime calls when software-PAN is enabled
Miscellaneous:
- Add Mark Rutland as a reviewer!
- Tidy-up our futex cmpxchg logic when using the new LSUI
instructions
- Drop the requirement on DYNAMIC_FTRACE_WITH_CALL_OPS when
selecting HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
- Fix a false-positive KCSCAN splat in the delay loop
- Use a portable typedef for 128-bit scalar types in our UAPI headers
- Non-critical fixes for Sashiko reports all over
MPAM:
- Hook MPAM memory bandwidth counters into resctrl's counter
assignment interface
- Fix a quirk in the MPAM bandwidth counting on Nvidia T241 so that
it also applies to 63 bit counters
Perf:
- Workarounds for hardware issues in the CMN-S3 PMU (Graviton 5) and
CPU PMU (NVIDIA Olympus again!)
- Add support for the DDR PMU on Marvell CN20K SoCs
- Add support for Picoheart implementations of the DCW PCIe PMU
- Add support for Channel/Rank/Bank filtering in the CXL PMU driver
- Add support for 64-bit counters in the CSPMU device
- Add support for revision 2 of the CMN S3 PMU
Ptrace:
- Fix a decade-old bug in our handling of seccomp and tracing on
syscall entry
- Fix regset handling for inactive SVE and SSVE registers
Selftests
- Add some tests for the decade-old bug that we just tried to fix in
our syscall entry path
- Fix SVE test crash on SME-only CPUs"
* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (95 commits)
arm64/efi: Avoid voluntary preemption with efi_mm installed
arm64: bti: Disable in-kernel BTI with recent versions of Clang
arm64: entry: Avoid unnecessary local_irq_disable() on kernel exit
irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs
arm64: Disable KCSAN instrumentation in delay.o
arm_mpam: Disable driver unbind to avoid UAF
arm_mpam: Fix a NULL pointer dereference on unbinding after an error interrupt
perf: arm_pmuv3: Zero initialize hw_id branch stack field
arm64: mm: Unmap kernel data/bss entirely from the linear map
iommu/arm-smmu-v3-sva: Use system_supports_bbml3() to detect CPU feature
perf/arm-cmn: Support CMN S3 r2
perf/arm-cmn: Plumb in new filter types
perf/arm-cmn: Refactor event filter data
perf/arm-cmn: Refactor event filter programming
perf/arm-cmn: Rename filter variables for clarity
arm64: mm: fix accidental linear mapping of no-map reserved memory
tools: Ensure tools copy of linux/filter.h exports the UAPI
kselftest/arm64: Fix abi test compilation errors
arch: arm64: add early_param idle=<wfi|yield|nop>
arm64: entry: mask DAIF before returning from C EL1 handlers
...
IOMMUFD selftest TRIGGER_IOPF borrows an attach handle from
group->pasid_array without synchronizing against PASID detach,
then a concurrent iommu_report_device_fault() can dereference
that borrowed handle's domain pointer after the detach erases
the handle and frees the backing struct iommufd_attach_handle.
TRIGGER_IOPF then dereferences the freed handle, causing a UAF.
Fix by adding a iopf_rwsem in mock_dev to follow the expected design
of a real driver. Hold its read side across the whole
iommu_report_device_fault() call, and its write side around every
path that attaches, detaches, or replaces a device domain.
This can block new reports and drains in-flight reports before an old
attach handle or the IOPF fault parameter can be removed.
Also take the write side while registering a mock device, since
it can invoke the mock driver's default-domain attach callback.
Closes: https://lore.kernel.org/all/D5E3AA41600B2056+f4e15662-bd2b-43ea-91cb-518de429e72c@smail.nju.edu.cn/
Fixes: ddee199710 ("iommufd/selftest: Add IOPF support for mock device")
Cc: stable@vger.kernel.org
Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
Assisted-by: Codex:gpt-5.6-terra
Signed-off-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Link: https://patch.msgid.link/38C8DF0A118B7176+20260811095551.2756745-1-peiyang_he@smail.nju.edu.cn
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Newer AMD IOMMUs supports DTE[Mode]=0 for SNP-enabled system. This is
detected using new feature bit (EFR2[SNP_Page_Mode_0_Sup]). If this
feature is enabled, then IOMMU supports:
- Passthrough mode (i.e. iommu=pt)
- amd_iommu=pgtbl_v2: Forcing Linux DMA-API to use IOMMU v2 page table
- Setting up device for SVA mode in the host.
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Tested-by: Amandeep Kaur Longia <AmandeepKaur.Longia@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The GN bit in the COMPLETE_PPR_REQUEST command indicates whether the
device is operating under a guest (v2) page table. Currently,
dev_data->pri_tlp is incorrectly used to derive this bit. However,
pri_tlp indicates whether the device uses PRI TLP which is unrelated
to page table mode.
Fix this by refactoring amd_iommu_complete_ppr() into a static internal
helper __amd_iommu_complete_ppr() that accepts an explicit 'gn' argument.
The amd_iommu_complete_ppr() wrapper then derives the GN bit correctly
from the device's active protection domain via pdom_is_v2_pgtbl_mode().
In the INVALID_PPR_REQUEST error handling path, the device's protection
domain may not be accessible, so derive the GN bit directly from the
EVENT_FLAG_PPR_GN flag in the event log entry instead.
While at it, change the 'gn' parameter type in build_complete_ppr()
from u8 to bool to better reflect its semantics.
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The amd_iommu_report_ppr_err() function logs an error message for every
INVALID_PPR_REQUEST event. Under certain fault conditions, a misbehaving
or malicious device can flood the IOMMU event log with PPR faults, causing
the kernel log to be overwhelmed with repeated error messages.
Switch from dev_err() to dev_err_ratelimited() to suppress duplicate
messages when INVALID_PPR_REQUEST events occur at a high rate.
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The AMD IOMMU spec, requires the host to respond with a CMD_COMPLETE_PPR
command when an EVENT_TYPE_INV_PPR_REQ event is received with the RX bit
cleared. This response was missing in the current implementation, leaving
invalid PPR requests unacknowledged.
Introduce amd_iommu_report_ppr_err() to handle EVENT_TYPE_INV_PPR_REQ
events. The new function logs the invalid PPR request and when the RX
bit is cleared, sends CMD_COMPLETE_PPR response.
Reported-by: Gaultier Delbarre <Gaultier.Delbarre@amd.com>
Co-developed-by: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The PPR tag field (PPRtag) encodes two distinct fields: the 9-bit tag
value (bits 8-0) and the last-page indicator L bit (bit 9).
Fix PPR_TAG() to mask only the 9-bit tag field and introduce
PPR_TAG_LAST_PAGE to explicitly extract the L bit. This way it becomes
easy to read.
Cc: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The IO page fault notifier handler logs pdev->dev.id when reporting an
invalid PASID, but pdev->dev.id is the kernel-internal device ID and
not the IOMMU device ID (BDF). Use dev_data->devid instead, which
reflects actual devid.
Fixes: 978d626b8f ("iommu/amd: Add IO page fault notifier handler")
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
domain_context_clear_one() and device_pasid_table_teardown() are both
invoked once per DMA alias of a device. Each function locates the context
entry using the bus/devfn pair provided by the pci_for_each_dma_alias()
callback, then calls intel_context_flush_no_pasid(), which constructs a
device-selective context-cache invalidation from info->bus and
info->devfn (that is, always the requester ID of the device itself).
As a result, for every alias other than the device’s own RID, the context
entry that was just cleared in memory is never invalidated in the context
cache. Hardware may continue using that stale cached entry. In the
scalable-mode teardown path, intel_pasid_free_table() can then free the
PASID directory still referenced by that stale entry, allowing the IOMMU
to walk freed memory.
Fix this by passing the source ID of the entry being torn down to
intel_context_flush_no_pasid(), instead of deriving it from @info.
Fixes: f90584f4be ("iommu/vt-d: Add helper to flush caches for context change")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
intel_pasid_setup_sm_context() walks a PCI device’s DMA aliases via
pci_for_each_dma_alias() and programs a scalable-mode context entry for
each RID. For a device with a dma_alias_mask, the callback is invoked
once for the device’s own RID and once for each alias bit, all with the
same pci_dev, so device_pasid_table_setup() runs for multiple RIDs.
pci_for_each_dma_alias() stops at the first callback error. Therefore, a
failure partway through the walk can leave context entries for already
processed RIDs present and still pointing to the device’s PASID table.
On this error path, intel_iommu_probe_device() currently jumps directly
to intel_pasid_free_table(), which frees the PASID table without
first tearing down those context entries. The IOMMU may then walk a
present context entry whose PASID table pointer references freed
memory.
intel_iommu_release_device() already performs teardown before freeing the
table. Apply the same ordering on the probe failure path.
device_pasid_table_teardown() safely handles RIDs that were never
programmed: iommu_context_addr() returns NULL when no context table has
been allocated, and clearing the Present bit of an already non-present
entry is a no-op. So unwind is safe for both the alias that failed and
any aliases not yet reached.
Fixes: 301f1a8048 ("iommu/vt-d: Setup scalable mode context entry in probe path")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
intel_iommu_attach_device() enables IOPF for the new domain but never
disables it for the old one. device_block_translation(), called at the
start of the function, tears down translation but does not touch any IOPF
state; blocking_domain_attach_dev() has to call iopf_for_domain_remove()
explicitly before invoking it for exactly this reason.
identity_domain_attach_dev() has the same problem. Its comment claims
that no PRI handling is needed because the device has been put in the
blocking state, but the blocking state and the IOPF reference count are
independent of each other.
As a result, replacing a domain that has an iopf_handler with another
domain at RID level leaks a reference in info->iopf_refcount. The count
never drops back to zero, so iopf_queue_remove_device() is never called
and iommu_disable_pci_pri() triggers its WARN_ON(info->iopf_refcount)
when the device is released.
The PASID paths already handle this correctly by way of
iopf_for_domain_replace(); convert the two RID paths to do the same.
Using the replace helper rather than a bare remove keeps the enable
before the disable, so the reference count does not transiently reach
zero and evict the device from the IOPF queue.
Fixes: 17fce9d233 ("iommu/vt-d: Put iopf enablement in domain attach path")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
copied_context_tear_down() zeroes the 128-bit context entry with
context_clear_entry() while the Present bit is still set, and only then
issues the context-cache and IOTLB invalidations. This leaves a window
in which hardware can fetch a torn entry, with some fields already zeroed
while Present is still set, leading to unpredictable behaviour or
spurious faults. While x86 provides strong write ordering, the compiler
may reorder the writes to the two 64-bit halves of the entry, and the
hardware fetch is not guaranteed to be atomic with respect to multiple
CPU writes.
There is no cacheline flush before the invalidation either, so on an
IOMMU without coherent access to the context table the zeroed entry may
not be visible to hardware at the point the invalidation is submitted.
Apply the same ownership handshake described in the VT-d spec, Section
6.5.3.3 ("Guidance to Software for Invalidations"): clear only the Present
bit, flush it out to the IOMMU, perform the invalidations, and only then
zero the remainder of the entry.
Fixes: c7191984e5 ("iommu/vt-d: Factor out helpers from domain_context_mapping_one()")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The iommu->copied_tables bitmap was introduced by the IOMMU live
update series to track which context entries have been copied from
the previous kernel. The allocation via bitmap_zalloc() was added
inside copy_translation_tables(), but the error paths were not
updated to free it:
1. When old_rt_phys is 0 (invalid root table address)
2. When memremap(old_rt_phys) fails
3. When kcalloc for ctxt_tbls fails (goto out_unmap, which only
unmaps old_rt without releasing the bitmap)
The bitmap is only cleaned up by free_dmar_iommu(), which is
called from the free_iommu error label in init_dmars(). However,
when copy_translation_tables() fails, init_dmars() does not jump
to free_iommu -- it logs the error, falls through, and continues
with the next IOMMU. As a result, copied_tables is leaked.
Fix this by converting the two early returns to goto a new
err_free_bitmap label, and by making out_unmap fall through to
it so that the bitmap is always freed on any error path. The
success path performs memunmap(old_rt) inline and returns 0
directly, since copied_tables must remain allocated for
subsequent use.
Signed-off-by: ZhaoJinming <zhaojinming@uniontech.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The cap_ndoms() helper calculates the maximum available domain ID from
the value of capability register, which can be inefficient if called
repeatedly. Cache the maximum supported domain ID in max_domain_id field
during initialization to avoid redundant calls to cap_ndoms() throughout
the IOMMU driver.
No functionality change.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Some BIOS already provides config options to expose/hide VT-d units
as a whole to/from system software. A new demand is to allow exposing
VT-d units but requesting system software to disable DMA remapping
while sustaining interrupt remapping. This can be communicated now by
setting the new DMA_REMAP_OPT_OUT flag bit in the DMAR table, as
introduced in VT-d spec v5.2 (section 8.1, DMA Remapping Reporting
Structure).
Introduce a new off policy (DMAR_FW_OFF) for DMA_REMAP_OPT_OUT. As
the strongest off policy, it cannot be overridden by user opts or
any force_on types. If tboot is enabled in the meantime, kernel will
panic. It is user responsibility to configure BIOS properly.
One cleanup is left for future - the DMAR flag is parsed multiple
times, in detect_intel_iommu(), dmar_platform_optin() (which can be
called at run-time), etc. Caching it is a cleaner way.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
It's replaced by dmar_policy_off() now, covering both "iommu=off"
and "intel_iommu=off". Also remove unnecessary checks on no_iommu,
leaving only one exception in intel_iommu_init() which skips debugfs
init for "iommu=off" but not "intel_iommu=off". Keep it to avoid
surprise for now.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The force_on variable is now redundant - same information captured
by "dmar_policy == DMAR_FORCE_ON". Replace all force_on checks
with dmar_policy_force_on().
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
So the policy of requesting ACS in detect_intel_iommu() is consistent
with that in tboot_force_iommu().
Though tboot is the strongest override so far, dmar_can_force_on() may
return false due to future extensions. In this case panic the kernel,
as is already done when failing to initialize DMA remapping for tboot.
No functional impact at this point.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
So the policy of requesting ACS in detect_intel_iommu() is consistent
with that in platform_optin_force_iommu().
While at it, remove no_platform_optin which is unnecessary now.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Currently the dmar on/off is carried by multiple variables (no_iommu,
dmar_disabled, no_platform_optin, etc.) with error-prone force_on logic
scattered in multiple places.
Unify/centralize the policy/priority management for various force_on
scenarios.
No functional impact except one case - "intel_iommu=off" sets
no_platform_optin which is checked in platform_optin_force_iommu()
but not in detect_intel_iommu(), leading to ACS unnecessarily requested
when iommu could not be forced on later. Now with the unified logic
this becomes more consistent.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Those are leftovers and unreachable now: the entire intel directory
is built only when CONFIG_INTEL_IOMMU is set.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Currently the conditions of requesting ACS in detect_intel_iommu()
don't include tboot, leading to a possible misconfiguration with ACS
disabled (e.g. due to user opts) while iommu is later forced on by
tboot_force_iommu().
Fix it by checking tboot in detect_intel_iommu().
Fixes: 5d990b6275 ("PCI: add pci_request_acs")
Cc: stable@vger.kernel.org
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
If user explicitly requests to disable iommu (via "iommu=off" or
"intel_iommu=off"), there is no reason to force enabling it due
to platform opt-in (for external-facing devices). User should be
aware of any security implication of doing so.
"intel_iommu=off" implements this policy by setting no_platform_optin
to skip platform opt-in in platform_optin_force_iommu().
However, "iommu=off" (no_iommu=1) doesn't set no_platform_optin
hence is broken in this aspect:
- detect_intel_iommu() doesn't request ACS if no_iommu=1
- platform_optin_force_iommu() forces iommu on if external-facing
devices exist and no_platform_optin is not set
This leads to a bad configuration with ACS disabled while DMA
remapping is enabled.
Instead of setting no_platform_optin (will soon be removed) for
no_iommu=1, directly check no_iommu in platform_optin_force_iommu().
Fixes: 89a6079df7 ("iommu/vt-d: Force IOMMU on for platform opt in hint")
Cc: stable@vger.kernel.org
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Simplify 'dmar_perf_latency_write()' by using the convenient
'kstrtoint_from_user()'.
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
In cache_tag_flush_range(), the CACHE_TAG_NESTING_DEVTLB case modifies the
shared local variables 'addr' and 'mask' before falling through to
CACHE_TAG_DEVTLB. This causes all subsequent CACHE_TAG_DEVTLB entries in
the same loop iteration to incorrectly use the full-range flush parameters
(addr=0, mask=MAX_AGAW_PFN_WIDTH) instead of the precisely calculated PSI
range. This is not the intended behavior, as regular DEVTLB entries should
always perform targeted range-based invalidation.
Fix this by having CACHE_TAG_NESTING_DEVTLB directly call
cache_tag_flush_devtlb_psi() with the full-range constants and break,
instead of modifying shared variables and falling through. This ensures
CACHE_TAG_DEVTLB always uses the original calculated addr and mask for
precise range flush.
This change slightly affects trace_cache_tag_flush_range() behavior.
Previously, after addr/mask were overwritten, the tracepoint could record
a full-range flush even when the caller requested a narrower range.
The tracepoint should reflect caller intent. Although this helper may
widen the actual hardware invalidation range for implementation reasons,
that does not change what the caller requested, so logging the requested
range is the correct behavior.
If the actual invalidation range sent to hardware is needed, it is already
visible via the qi_submit trace event, which records the invalidation
descriptors emitted by the driver.
Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Replace bitwise OR (|) with logical OR (||) in the privilege mode
validation check. While both operators produce the same result for
boolean values (0 or 1), using logical OR is semantically correct and
makes the intent clearer.
No functional change, but improves code readability.
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
When translation is already enabled at boot (e.g. kdump), the vt-d driver
copies context tables from the previous kernel's root table. In scalable
mode, buses that only populate the upper root half (UCTP, devfn >= 0x80)
should be written to ctxt_tbls[tbl_idx + 1] through copy_context_table().
However, the current copy path always uses tbl[tbl_idx + 0] in this situa-
tion. Since idx wraps to 0 at devfn 0x80 due to a zeroed LCTP, new_ce for
LCTP will be NULL and keep pos equals to 0. Thus, UCTP entries will be co-
pied into tbl[tbl_idx + 0] instead of tbl[tbl_idx + 1], and written after-
wards to root_entry[bus].lo instead of .hi in copy_translation_tables().
In short, devices on bus 0x80 with devfn >= 0x80 fail DMA with fault 0x39,
which will break drivers running in kernels with translation pre-enabled.
This fixes NO_PASID DMAR faults for UCTP-only buses such as:
DMAR: [DMA Read NO_PASID] Request device [80:14.0] fault addr 0xe81759000
[fault reason 0x39] SM: Present bit in Root Entry is clear
For instance, this fault yielded to locking issues between systemd and
xHCI, blocking a system's reboot after a vmcore was captured with kdump:
systemd-udevd[246]: usb3: Worker [255] processing SEQNUM=2193 is taking a long time
dracut-initqueue[277]: Timed out while waiting for udev queue to empty.
systemd-udevd[246]: usb3: Worker [255] processing SEQNUM=2193 killed
systemd-udevd[246]: usb3: Worker [255] terminated by signal 9 (KILL).
...
kdump[569]: saving vmcore complete
...
systemd-shutdown[1]: Rebooting.
INFO: task kworker/0:1:11 blocked for more than 122 seconds.
Not tainted 7.0.0-clean #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/0:1 state:D stack:0 pid:11 tgid:11 ppid:2 task_flags:0x4208160 flags:0x00080000
Workqueue: usb_hub_wq hub_event
Call Trace:
<TASK>
__schedule+0x299/0x5c0
schedule+0x27/0x80
schedule_timeout+0xbd/0x100
__wait_for_common+0x97/0x1b0
? __pfx_schedule_timeout+0x10/0x10
xhci_alloc_dev+0x9e/0x2b0
usb_alloc_dev+0x7a/0x3b0
hub_port_connect+0x285/0x960
hub_port_connect_change+0x94/0x290
port_event+0x4bb/0x840
hub_event+0x141/0x460
process_one_work+0x196/0x390
worker_thread+0x1af/0x320
? __pfx_worker_thread+0x10/0x10
kthread+0xe3/0x120
? __pfx_kthread+0x10/0x10
ret_from_fork+0x199/0x260
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>
INFO: task systemd-shutdow:1 blocked for more than 122 seconds.
Not tainted 7.0.0-clean #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:systemd-shutdow state:D stack:0 pid:1 tgid:1 ppid:0 task_flags:0x400100 flags:0x00080000
Call Trace:
<TASK>
__schedule+0x299/0x5c0
schedule+0x27/0x80
schedule_preempt_disabled+0x15/0x30
__mutex_lock.constprop.0+0x547/0xac0
device_shutdown+0xac/0x1b0
kernel_restart+0x3a/0x70
__do_sys_reboot+0x147/0x240
do_syscall_64+0x11b/0x6a0
? handle_mm_fault+0x110/0x350
? do_user_addr_fault+0x206/0x680
? irqentry_exit+0x7a/0x4d0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7fe2958da917
RSP: 002b:00007ffc5c458618 EFLAGS: 00000206 ORIG_RAX: 00000000000000a9
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe2958da917
RDX: 0000000001234567 RSI: 0000000028121969 RDI: 00000000fee1dead
RBP: 00007ffc5c458790 R08: 0000000000000069 R09: 00000000ffffffff
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 00007ffc5c4588b8 R15: 0000000000000000
</TASK>
INFO: task systemd-shutdow:1 is blocked on a mutex likely owned by task kworker/0:1:11.
Fixes: 091d42e43d ("iommu/vt-d: Copy translation tables from old kernel")
Signed-off-by: Desnes Nunes <desnesn@redhat.com>
Tested-by: Tao Liu <ltao@redhat.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Unlike a group's default domain, which is always freshly allocated
and privately owned (iommu_group_alloc_default_domain()), VFIO type1's
legacy container merges any newly attached group into an existing
domain whenever their iommu_ops and cache-coherency enforcement match.
iommu_dma_get_msi_page() only asserts the caller's own group mutex is
held (iommu_group_mutex_assert()). On an IOMMU that publishes
IOMMU_RESV_SW_MSI, e.g. ARM SMMU, a VM with two such devices assigned
through the legacy container can have their guest drivers probe and
allocate MSIs in parallel; each host-side VFIO_DEVICE_SET_IRQS lands
on a different device fd and group mutex, but both devices' domains
are the same merged domain, so both can enter
iommu_dma_get_msi_page() concurrently and corrupt msi_page_list.
commit 288683c92b ("iommu: Make iommu_dma_prepare_msi() into a
generic operation") dropped the prior msi_prepare_lock on the
reasoning that "each iommu_domain is unique to a group," which holds
for default domains but not this VFIO type1 case. Restore the static
lock, since it's only guarding a corner case and will likely never
be contended.
iommufd avoids the equivalent problem by having its own callers
(iommufd_sw_map_msi()) take a ctx-wide sw_msi_lock before ever
reaching the shared list. VFIO type1 can't mirror that since it
dispatches to iommu_dma_sw_msi() which is outside VFIO's jurisdiction.
Fixes: 288683c92b ("iommu: Make iommu_dma_prepare_msi() into a generic operation")
Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Nutty Liu <nutty.liu@hotmail.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
DOMAIN_NS(iova_to_phys) returns a phys_addr_t and follows the IOMMU
iova_to_phys() convention of returning 0 when there is no valid
translation.
make_range() returns a negative errno if the input IOVA cannot be
represented by this page table format. Returning that errno directly
casts it to phys_addr_t. Callers treat zero as no translation and use
non-zero values as physical addresses, so this can make them consume a
bogus physical address instead of seeing a failed translation.
Match the page-table walk failure path and return 0 for range validation
failures too.
Fixes: 9d4c274cd7 ("iommupt: Add iova_to_phys op")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
iommufd_take_all_iova_rwsem() takes an object reference and the
iova_rwsem write lock before storing the IOAS in the temporary ioas_list
xarray.
If xa_store() fails, the current IOAS has not been inserted into
ioas_list yet. iommufd_release_all_iova_rwsem() only unwinds IOAS
objects already present in that xarray, so it cannot release the current
IOAS.
Release the current IOAS rwsem and object reference before unwinding the
previously stored entries.
Fixes: 051ae5aa73 ("iommufd: Lock all IOAS objects")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
iommufd_access_notify_unmap() skips internal accesses because they do
not have an external unmap callback to invoke.
However, the current test calls iommufd_lock_obj() before checking
whether the access is internal. If iommufd_lock_obj() succeeds, the loop
then sees the internal access and continues, bypassing the matching
iommufd_put_object() used by the normal unmap path. This leaks the
object reference taken by iommufd_lock_obj().
Check for internal accesses first so skipped entries are never locked.
Fixes: 27b77ea5fe ("iommufd/access: Bypass access->ops->unmap for internal use")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
iommu_attach_device_pasid() installs the new SVA attach handle in the
group PASID lookup before iommu_sva_bind_device() returns. A concurrent
bind can therefore find and reuse the same handle after iommu_sva_lock is
dropped.
handle->dev was initialized after dropping iommu_sva_lock. This leaves a
window where a racing bind can return a handle whose dev pointer is still
NULL. A subsequent iommu_sva_unbind_device() can then dereference it via
handle->dev->iommu_group.
Initialize handle->dev before releasing iommu_sva_lock so any visible SVA
handle is fully initialized.
Fixes: be51b1d6bb ("iommu/sva: Refactoring iommu_sva_bind/unbind_device()")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmp49yoeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiG1WIH/1SD1rNIb4Prf0C7
Rk9Hz0IZ3uYNe1Xi+Ozv3zQUuLB6d7ewl9e4zyHHYT4eeIKvBZv9jtIF3QH4ty5X
JB9Gy62WICtqIEkzzt2bkcV1byFmKz2+BLv04fgcyTXEGOEByCDiZ3/N3kSJOqvg
YxsGOmKkQSTavFLSe96Gj2pgchJqZ6j3qucDO6GrPOo8depP2kfzxfaJbXix0HEZ
ln08X9hgZDxy4I98PmSaaJsK2R/i6T6jQdA3i8DnSbbyNhLn+3FK6gkP08ZEpaor
82WwJf+MVCLHXLYDy0aYOUDHpkNkkJWOj45A2Hv9JkOupGcUpyPGr82l4yh/MrxC
+kM28xs=
=rIbd
-----END PGP SIGNATURE-----
Merge tag 'v7.2-rc7' into driver-core-next
We need the driver-core fixes in here as well to build on top of.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Commit 94104e3cfa ("arm64: cpufeature: Rename BBML2_NOABORT as BBML3")
renamed the cpu_supports_bbml2_noabort() helper to cpu_supports_bbml3(),
as the Linux-defined "noabort" semantics have now been incorporated into
the architecture under the BBML3 feature.
Update the caller in the SMMUv3 SVA driver to use the new function.
There is a slightly oddity in that the SMMUv3 architecture already
defined BBML2 in such a way that aborts were prohibited, so we use the
BBML3 feature on the CPU to enable BBML2 in the SMMU.
Fixes: 94104e3cfa ("arm64: cpufeature: Rename BBML2_NOABORT as BBML3")
Signed-off-by: Will Deacon <will@kernel.org>
The number of master->num_streams per master device is typically very
small in practice. Sorting this array takes a very small amount of
time, so there is no practical risk of triggering a soft lockup that
would necessitate calling cond_resched() during the sort.
Replace sort_nonatomic() with the standard sort(). Since this is the
only remaining in-tree caller of sort_nonatomic(), this change paves
the way to eventually remove the unused sort_nonatomic() API from the
core library.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Will Deacon <will@kernel.org>
An SMMU implementing SMMU_IDR5.DS extends the range invalidation commands:
the SCALE field grows a 6th bit, raising its maximum value from 31 to 39,
and TTL == 0b01 becomes a valid level hint for a 16KB translation granule.
Add a new ARM_SMMU_FEAT_DS feature detecting the DS bit, and widen the
CMDQ_TLBI_0_SCALE field to its architectural 6 bits. Mask the scale value
explicitly in arm_smmu_cmdq_batch_add_range(), so the range invalidation
path emits the same commands as before, keeping the pre-existing 5-bit
truncation of a scale above 31.
Also list DS as a valid IDR5 field in the iommu_hw_info_arm_smmuv3 kdoc:
iommufd has always reported the raw IDR5 register, so a VMM may conclude
from that bit alone that it can expose DS to its guest.
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Assisted-by: Claude:claude-fable-5
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
The arm_smmu_write_reg_sync() helper is currently implemented using
readl_relaxed_poll_timeout() (that relies on usleep_range() internally)
which becomes a critical issue when used in the gerror irq handler.
If the SMMU hits a gerror and enters Service Failure Mode
(GERROR_SFM_ERR), the gerror handler calls arm_smmu_device_disable() in
hard-irq context. This becomes a problem as arm_smmu_device_disable()
inevitably calls arm_smmu_write_reg_sync() which might attempt to sleep
inside a hard-irq context.
Fix this by converting the arm_smmu_write_reg_sync to use the
readl_relaxed_poll_timeout_atomic() polling helper.
(Discovered while running Sashiko locally on another patch series).
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 48ec83bcbc ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Signed-off-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
Since table access flags cannot be software-managed, if process
pagetables are using HAFT then SVA must require the SMMU to support and
enable it too, otherwise page aging is liable to get out of whack.
For unbinding, we can't disable HAFT atomically with HA as might be
desired, but luckily we can get away with just not disabling HA either.
Cc: stable@vger.kernel.org
Fixes: 62df5870eb ("arm64: Enable ARCH_HAS_NONLEAF_PMD_YOUNG")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
Change swiotlb_tbl_map_single() to take the DMA mapping attributes by
reference and update the direct callers accordingly.
This is a preparatory change for a follow-up patch which updates the
attributes based on the selected swiotlb pool. Keeping the signature change
separate makes the follow-up patch easier to review.
No functional change in this patch.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-13-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Teach the atomic DMA pool code to distinguish between encrypted and
unencrypted pools, and make pool allocation select the matching pool based
on DMA attributes.
Introduce a dma_gen_pool wrapper that records whether a pool is
unencrypted, initialize that state when the atomic pools are created, and
use it when expanding and resizing the pools. Update dma_alloc_from_pool()
to take attrs and skip pools whose encrypted state does not match
__DMA_ATTR_ALLOC_CC_SHARED. Update dma_free_from_pool() accordingly.
Also pass __DMA_ATTR_ALLOC_CC_SHARED from the swiotlb atomic allocation
path so decrypted swiotlb allocations are taken from the correct atomic
pool.
Tested-by: Jiri Pirko <jiri@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-12-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
The non-blocking, non-coherent allocation path uses dma_alloc_from_pool(),
which returns the allocated page and fills cpu_addr only on success.
Do not rely on cpu_addr to detect allocation failure in this path. Check
the returned page directly before using it for the IOMMU mapping.
Fixes: 9420139f51 ("dma-pool: fix coherent pool allocations for IOMMU mappings")
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-4-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Several bug fixes found by tools and fuzzing:
- Incorrect domain passed during replace to ack faults
- Block the access API from using dmabuf
- Missing unlock on error unwind
- Race seeing a partially setup vdevice in the xarray
- Do not allow vdevices to have multiple stream ids in SMMUv3
- Possible UAF if racing faults with domain changes
-----BEGIN PGP SIGNATURE-----
iHUEABYIAB0WIQRRRCHOFoQz/8F5bUaFwuHvBreFYQUCamu3EAAKCRCFwuHvBreF
YfibAP9Pv3SmmvPvbJjnUo7Vfu8Ra8uLOkK7CGFJFnqvYFnZ5gD/Xd0leOM0nk7B
/2Zvv1byPIONmNUjJP5GHRt/FiK9mAw=
=74z6
-----END PGP SIGNATURE-----
Merge tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd
Pull iommufd fixes from Jason Gunthorpe:
"Several bug fixes found by tools and fuzzing:
- Incorrect domain passed during replace to ack faults
- Block the access API from using dmabuf
- Missing unlock on error unwind
- Race seeing a partially setup vdevice in the xarray
- Do not allow vdevices to have multiple stream ids in SMMUv3
- Possible UAF if racing faults with domain changes"
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd:
iommu/iommufd: Fix IOPF group ownership UAF
iommu/arm-smmu-v3-iommufd: Require exactly one Stream ID for a vDEVICE
iommufd/viommu: Publish a vDEVICE only after vdevice_init() succeeds
iommufd/viommu: Release the igroup lock on the vdevice_size error path
iommufd: Reject DMABUF pages from the access pin path
iommufd: Fix wrong hwpt passed to iommufd_auto_response_faults on replace
A large amount of memory may be allocated for these magazines on
machines with a lot of IOMMU groups and CPU cores. Not all may be used
as some devices may be unused or be bound to drivers that do not use the
DMA-API. Furthermore, some drivers may not use all levels or CPUs.
Move the initialization of the loaded and prev magazines for each CPU on
the first attempt to try to insert a freed IOVA to them.
Signed-off-by: Logan Odell <loganodell@google.com>
Signed-off-by: Michal Clapinski <mclapinski@google.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
