Commit graph

1,481,581 commits

Author SHA1 Message Date
Matthew Auld
7cf85c1908 drm/xe/mmio_gem: reject VM_EXEC and drop VM_DONTCOPY
Hardware MMIO registers should never be executable; reject VM_EXEC
at mmap time and clear VM_MAYEXEC to prevent later mprotect attempts.

Also drop VM_DONTCOPY so that child processes across fork() can
inherit the mapping and lazily fault in the PFNs, matching standard
DRM GEM semantics and making the existing drm_gem_vm_open() callback
functional.

This aligns with existing PCI_BARRIER, which will use this in the next
patch. We don't want any noticeable behaviour change there, since this
will be user visible. This will also be the first user.

Assisted-by: LLM
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-17-matthew.auld@intel.com
2026-09-09 10:57:10 +01:00
Ilia Levi
fb2ee38bab drm/xe/mmio_gem: fix destroy flow
xe_mmio_gem_destroy() currently frees the GEM object directly, bypassing
reference counting.  Since existing VMAs hold a reference and the fault
handler accesses the object through vma->vm_private_data, this is
use-after-free.  Additionally, nothing prevents the fault handler from
installing PTEs to the real MMIO after destroy.

Fix this with proper synchronization and refcounting. Also, do not set
vm_pgoff to zero. Many DRM drivers do this because helpers like
dma_mmap_pages() interpret vm_pgoff as an intra-buffer page offset;
leaving the DRM fake offset there would break these helpers.
Those drivers can get away with zeroing it because they map eagerly -
all PTEs are established before mmap returns, so vm_pgoff is never
consulted again. Our driver does not use such helpers and the newly
introduced call to drm_vma_node_unmap() relies on vm_pgoff being untouched.

v2: (Matt Auld)
- use dma_resv lock to serialize fault handler with destroy
- SIGBUS on access after destroy

Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-16-matthew.auld@intel.com
2026-09-09 10:57:10 +01:00
Ilia Levi
8bf6213f98 drm/xe/mmio_gem: cache the dummy page per object
Currently, when the fault handler provides a dummy page, it
allocates a new one on every invocation and ties its lifetime to
the drm_device via drmm_add_action_or_reset(). Concurrent faults
after hot-unplug therefore accumulate pages that persist until
device teardown.

Cache a single dummy page in the xe_mmio_gem object and use dma_resv
lock to protect its allocation. Free it with the object.

v2: use dma_resv lock to protect the allocation (Matt Auld)

Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-15-matthew.auld@intel.com
2026-09-09 10:57:10 +01:00
Shuicheng Lin
32f0cb2505 drm/xe/mmio_gem: Revoke drm_vma_node on xe_mmio_gem destroy
xe_mmio_gem_create() calls drm_vma_node_allow() but nothing ever calls
drm_vma_node_revoke(). The drm_vma_offset_file rb-tree entry allocated
by drm_vma_node_allow() is not freed by drm_gem_object_release(), so
it is leaked on every create/destroy cycle.

Add a struct drm_file * parameter to xe_mmio_gem_destroy() and call
drm_vma_node_revoke() from there, mirroring the drm_vma_node_allow()
call in xe_mmio_gem_create().

Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Suggested-by: Ilia Levi <ilia.levi@intel.com>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Reviewed-by: Ilia Levi <ilia.levi@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-14-matthew.auld@intel.com
2026-09-09 10:57:10 +01:00
Ilia Levi
6666ca9192 drm/xe/mmio_gem: simplify fault handler loop
Make the iteration over the addresses in the VMA more explicit.
No functional change, as the VMA matches the GEM object exactly.

Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-13-matthew.auld@intel.com
2026-09-09 10:57:10 +01:00
Ilia Levi
1e8e28e35d drm/xe/mmio_gem: use write-back mapping for dummy page
Currently vmf_insert_pfn() maps the dummy page as UC, inheriting the
VMA's page protection which was set for the real MMIO region. This
conflicts with the direct map's WB mapping of the same page, creating a
cache type alias which is architecturally undefined on some platforms.

Use vmf_insert_pfn_prot() with a WB pgprot instead. Also simplify to
fault in the requested page instead of the whole VMA.

Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260525125801.975038-6-ilia.levi%40intel.com
Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-12-matthew.auld@intel.com
2026-09-09 10:57:10 +01:00
Ilia Levi
f3391a0b12 drm/xe/mmio_gem: forbid VMA split
The fault handler assumes it always operates on a VMA spanning the entire
GEM object. This does not hold when the VMA has been split, e.g. by a
partial munmap or mprotect. In that case the handler may map wrong
physical pages or cause SIGBUS.

Handle this by forbidding VMA split, as partial unmaps are not deemed
useful for MMIO GEMs.

Suggested-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-11-matthew.auld@intel.com
2026-09-09 10:57:10 +01:00
Varun Gupta
683eb75c15 drm/xe: Add multi_queue_active_lrca debugfs
Add a per-GT debugfs file, multi_queue_active_lrca, that prints, for
every engine supporting multi-queue, the currently active queue ID
(CSMQDEBUG) and the LRCA of the exec queue occupying that slot within
the running multi-queue group.

RING_CURRENT_LRCA only reports the primary queue's LRCA for the group
and does not update to reflect the active queue in multi-queue mode,
which makes it hard to tell which queue is actually running when
debugging multi-queue CSB/context-switch issues. Resolve the active
LRCA by matching the primary LRCA against each queue's group and
picking the queue at the reported active_id position.

v4:
 - Extracted multi_queue_active_lrca into a dedicated
   multi_queue_debugfs_list to prevent debugfs node registration on
   platforms lacking multi-queue support entirely, via
   xe_gt_has_multi_queue() gating. (Tejas)
v3:
 - Use xe_exec_queue_get_lrc() instead of raw pointer dereference to
   safely handle concurrent multi-queue group creation and avoid race
   conditions (Sashiko)
v2:
 - Maintain alphabetical order for includes and
   pf_only_debugfs_list (Tejas)
 - Export and reuse xe_lrc_get_multi_queue_active_queue_id() instead
   of duplicate MMIO read (Tejas)

Bspec: 60321, 73976
Signed-off-by: Varun Gupta <varun.gupta@intel.com>
Link: https://patch.msgid.link/20260908033417.1019602-2-varun.gupta@intel.com
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
2026-09-08 20:12:32 +05:30
Thomas Hellström
434514b6fe drm/xe: Flush LSC untyped L1 dataport cache after rcs/ccs batches
emit_render_cache_flush() sets PIPE_CONTROL0_HDC_PIPELINE_FLUSH to
flush the L2/HDC data cache before fence signalling, but it never
requests a flush of the LSC untyped L1 data cache via the 'Untyped
Data-Port Cache Flush Enable' bit in PIPE_CONTROL DWord0[11].

Per the Bspec, in 3D pipeline mode HDC Pipeline Flush is documented to
also flush/invalidate the untyped L1 cache, but only depending on how
HDC_CHICKEN0[13:11] is programmed. Starting with MTL, this coupling
between HDC Pipeline Flush and the untyped L1 cache flush no longer
holds in practice, regardless of how HDC_CHICKEN0 is programmed, so
relying on it is not safe on newer platforms such as BMG. Mesa's Vulkan
driver (anv) has been assuming the kernel flushes both caches between
submissions, and hit user-visible corruption in apps such as Llama.cpp
because of this gap; it now works around it by flushing both caches
again from userspace at the end of every command buffer.

Correctness between submissions on the same queue is userspace's
responsibility and belongs in Mesa, not the kernel. However, for
security we must ensure stale data can't leak through the untyped L1
dataport cache once memory is reclaimed or evicted, which requires the
KMD to flush it before releasing memory for reuse.

Prior to MTL, HDC_CHICKEN0 could be programmed (as already done for
DG2 via Wa_22010960976/Wa_14013347512) to reliably keep HDC Pipeline
Flush coupled to the untyped L1 cache flush, so those platforms are
unaffected. Mesa's own anv driver found that on MTL the HW
disconnected the two independently of how HDC_CHICKEN0 is programmed,
and could not bring the old behavior back even by writing the register
by hand; see Mesa commit 7c2ff46a4fc3 ("anv: don't prevent L1 untyped
cache flush in 3D mode"). The kernel can't reliably request the flush
from the CS on MTL either, so restrict the new PIPE_CONTROL bit to
GRAPHICS_VERx100 >= 2000 (Xe2 and later), where it can be relied on.

Explicitly set PIPE_CONTROL0_UNTYPED_DATAPORT_CACHE_FLUSH together
with PIPE_CONTROL0_HDC_PIPELINE_FLUSH in emit_render_cache_flush() on
Xe2 and later, so the L1 data cache is known clean before memory is
released for reuse, without depending on undocumented
platform-specific HDC_CHICKEN0 behavior.

Bspec: 56551
Link: 7c2ff46a4f
Fixes: 9f8f93bee3 ("drm/xe: Emit a render cache flush after each rcs/ccs batch")
Reported-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/8909
Cc: José Roberto de Souza <jose.souza@intel.com>
Cc: intel-xe@lists.freedesktop.org
Cc: <stable@vger.kernel.org> # v6.8+
Assisted-by: GitHub_Copilot:claude-sonnet-5
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260903114552.48634-1-thomas.hellstrom@linux.intel.com
2026-09-08 14:32:50 +02:00
Arvind Yadav
3959215038 drm/xe/gt: Report GT reset failure using SIGID
Route the GT reset failure log through the structured SIGID logging helper.

Failure to complete the full graphics reset means GT reset did not finish
as expected. Use the GT component, which maps to XE_SIGID_GT_TDR, and pass
the reset errno value to the helper.

Also update the message to describe the actual failure. The driver does not
clear GRDOM_FULL directly. The reset should complete and clear it within
the timeout.

Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Suggested-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20260903054610.3640788-5-arvind.yadav@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
2026-09-08 12:39:28 +05:30
Arvind Yadav
36ba01e430 drm/xe/svm: Report terminal page-fault failures using SIGID
Use structured SIGID logging for SVM page-fault errors after
retries are exhausted.

Report these failures with the PAGEFAULT component, which maps to
XE_SIGID_MEM_FAULT.

Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Suggested-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20260903054610.3640788-4-arvind.yadav@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
2026-09-08 12:39:28 +05:30
Arvind Yadav
58980bf190 drm/xe/guc: Report reset failure using SIGID
Route the GuC submission engine reset failure through the structured SIGID
logging helper.

Use the GUCSUBMIT component, which maps to XE_SIGID_GT_TDR, and pass -EIO
as the errno value.

Cc: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Suggested-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20260903054610.3640788-3-arvind.yadav@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
2026-09-08 12:39:28 +05:30
Arvind Yadav
26793f00d6 drm/xe/log: Add GUCSUBMIT component
The proper SIGID for GuC submission failures is GT_TDR, but using the
generic GT component may hide the real source of the report.

Add GUCSUBMIT as a DRIVER component and map it to XE_SIGID_GT_TDR.

Cc: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Suggested-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20260903054610.3640788-2-arvind.yadav@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
2026-09-08 12:39:27 +05:30
Honglei Huang
18299151ca drm/gpusvm: keep an IOVA mapped range dma address inline
dma_iova_try_alloc() reserves one contiguous IOVA for the whole range and
links each page at the next offset, so the device addresses run
contiguously from entry 0 and one entry describes them all. A 2 MiB range
of 4 KiB pages then drops the same 8 KiB array as a THP backed one.

Fold only when state_offset covers the full range, which proves no device
page was mapped in between, and only single page entries, so the order
kept is 0 and stays true. Widening it instead would tell a consumer to use
a huge page for npages separate CPU pages, which hangs Vega20 on amdgpu.

The kept entry no longer bounds the segment, so skip the unmap walk when
it has nothing to do, keyed off dpagemap rather than the flags, which are
not published yet on the error unwind. Consumers need the same
distinction, so drm_gpusvm_pages_first_dma() returns it alongside the
array from one read of the flags; xe passes it to xe_res_first_dma().

Suggested-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-7-honghuan@amd.com
2026-09-07 22:44:02 -07:00
Honglei Huang
8d242e452e drm/gpusvm: keep a single DMA mapping inline for THP
drm_gpusvm_get_pages() sizes the dma_addr array for one drm_pagemap_addr
per page, but the mapping loop advances by page order, so a range backed
by one huge page needs a single entry. For a 2 MiB THP that is an 8 KiB
array holding 16 bytes of address.

Union that entry with the array pointer, discriminated by a new
inline_dma_mapping flag. When drm_gpusvm_dma_map_pages() ends up with one
entry it stores it inline and frees the array, after the last error
unwind, which still walks the array form. An unchecked dma_addr read is
now type confusion rather than a compile error, so reads go through the
new drm_gpusvm_pages_first_dma() accessor, including the two
xe_pt_stage_bind() paths.

Only get_pages() and the free path write the union, never the notifier,
and both run under the driver lock that every address reader already
holds. The unlocked short circuit in drm_gpusvm_pages_valid_unlocked()
goes for the same reason: it cannot resolve the union, and every instance
it rejects has to be reset before the allocation loop reuses it.

Suggested-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-6-honghuan@amd.com
2026-09-07 22:44:00 -07:00
Honglei Huang
22a81666c2 drm/gpusvm: make the DMA mapping step in get_pages() optional
Some drivers (e.g. AMDXDNA) only need the CPU pages faulted in and tracked
by the notifier, no need DMA mapping.

Add a drm_gpusvm_ctx::no_dma_map flag. When set, get_pages() does the
shared HMM fault and records notifier_seq, but skips svm_pages->drm
validation, the dma_addr allocation and drm_gpusvm_dma_map_pages().
With no mapping state to check, the fault is redone on every call. The
default (no_dma_map == 0) is unchanged.

Suggested-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-5-honghuan@amd.com
2026-09-07 22:43:59 -07:00
Honglei Huang
cc2a6cbd68 drm/gpusvm: let drm_gpusvm_get_pages() map an array of pages
With the N:1 drm_gpusvm_pages layout, one CPU range mirrored on several
drm_devices, the caller had to invoke get_pages() once per device and
repeat the HMM fault every time.

Make get_pages() take a contiguous array of drm_gpusvm_pages plus a
count: fault once, then DMA map each instance by
drm_gpusvm_dma_map_pages() under a single read_retry gate. xe range and
userptr callers are updated.

Document the N:1 array usage in the Overview, showing how get_pages()
and drm_gpusvm_range_set_unmapped() take the whole array and its count
while the unmap and free paths stay per-instance.

Suggested-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-4-honghuan@amd.com
2026-09-07 22:43:57 -07:00
Honglei Huang
1ae616edb6 drm/gpusvm: extract drm_gpusvm_dma_map_pages() helper
Move the per-device DMA mapping loop of drm_gpusvm_get_pages() into a
helper drm_gpusvm_dma_map_pages(). The mapping logic is only moved, not
changed, so there is no functional change.

The helper maps the already-faulted pfns into one drm_gpusvm_pages
instance under the notifier lock and unwinds its own partial mapping on
error. The HMM fault and the notifier retry loop stay in get_pages()
common code rather than being pushed down to drivers, so no driver has
to reimplement the subtle fault and retry logic.

With the mapping isolated per instance, get_pages() can later fault once
and DMA map an array of drm_gpusvm_pages plus a count, one per owning
drm_device.

Suggested-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-3-honghuan@amd.com
2026-09-07 22:43:56 -07:00
Honglei Huang
ca76137442 drm/gpusvm: move dma_addr allocation before the notifier lock
The dma_addr allocation was in a lazy allocation flow, it needs unlock
and goto map_pages. The allocation only needs npages, so just do it
before taking the lock. Drop the map_pages label and the relock flow, so
the sequence becomes fault, allocate, then lock, validate, map and
unlock. No functional change intended.

Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-2-honghuan@amd.com
2026-09-07 22:43:55 -07:00
Varun Gupta
fcc2431d22 drm/xe: Guard page-fault worker with runtime PM check
During VM teardown, the VM's runtime PM reference is dropped
asynchronously, allowing the device to autosuspend while stale page
faults belonging to the now-dead VM are still queued. When the
page-fault worker later tries to ack one of these, it calls into
guc_ct_send_locked() on an already-suspended device, tripping:

  Assertion `!xe_pm_runtime_suspended(xe)` failed!
  WARNING at xe_device.c:1267 xe_device_assert_mem_access+0x11c/0x140 [xe]

A live VM/exec queue always holds a PM reference while it has
outstanding work, so if the device is suspended at ack time, the
owning context is already gone and the fault is stale.

Take a runtime PM reference across the entire pagefault
queue worker to safely deliver acks for torn-down VMs.

v3:
 - Move PM ref to the generic xe_pagefault_queue_work using
   guard(xe_pm_runtime)(xe) instead of tracking it in the GuC
   backend(Matt Brost).

v2:
 - Hold PM ref across the entire batch (begin/end) instead of per-ack.
   This prevents the device from autosuspending mid-batch, which would
   leave write_only acks written but the end flush skipped, and skip
   counter++, desyncing the cadence check.(Himal)
 - Add a comment explaining stale faults.(Himal)

Fixes: f289f78071 ("drm/xe: Add xe_guc_pagefault layer")
Signed-off-by: Varun Gupta <varun.gupta@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Link: https://patch.msgid.link/20260907050011.497181-2-varun.gupta@intel.com
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
2026-09-07 19:04:16 +05:30
Tejas Upadhyay
6ef3eb10ae drm/xe: Add fault-inject based VRAM page offline injection
Add a fault-inject based debugfs interface for testing VRAM page
offlining. This replaces the previous standalone debugfs approach
with the standard kernel fault-inject infrastructure.

Two debugfs entries are created under the xe debugfs root for
CRI platforms:
- inject_mempage_offline/: Standard fault-inject knobs (probability,
  times, interval, etc.) created by fault_create_debugfs_attr().
  Without CONFIG_FAULT_INJECTION_DEBUG_FS, the stub returns
  ERR_PTR(-ENODEV) and no knobs are created, making the trigger
  effectively a no-op.
- inject_mempage_offline_trigger: Write a PFN value to inject a
  specific page, or write "0" to auto-pick the last unallocated
  VRAM page

The trigger accepts:
- "0"      : auto-pick last unallocated page
- "0xPFN"  : inject fault at a specific PFN address

Usage:
  echo 100 > inject_mempage_offline/probability
  echo 1 > inject_mempage_offline/times
  echo 0 > inject_mempage_offline_trigger

  probability: likelihood of should_fail() returning true (0-100)
  times: number of times injection is allowed (-1 for unlimited)

v7:
- Use PAGE_SIZE
v6(Himal):
- Add warning to rebind driver post test run
v5(Sashiko):
- exclude SRIOV and remove dpa_base addition, already absolute dpa
v4(Himal):
- Use xe_fault_mempage_offline() instead of IS_ENABLED() +
  direct should_fail(). CONFIG_FAULT_INJECTION_DEBUG_FS is now
  an implicit requirement for the trigger to function.
v3(Himal):
- Use FAULT_ACTION
v2(sashiko):
- use cond_resched()
- validate input first and fix addr < 0 case
- validate vr, move block, found var as local to scope_guard

Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-32-tejas.upadhyay@intel.com
2026-09-03 22:30:16 -07:00
Tejas Upadhyay
69183b7062 drm/xe/uapi: Expose ban reason in EXEC_QUEUE_GET_PROPERTY_BAN
Extend DRM_XE_EXEC_QUEUE_GET_PROPERTY_BAN to return a bitmask indicating
the reason for the ban, rather than a simple boolean. This allows
userspace to distinguish between different ban causes:

- DRM_XE_EXEC_QUEUE_BAN_REASON_GPU_HANG (bit 0): exec queue was banned
  due to a GPU hang or job timeout detected by the TDR.
- DRM_XE_EXEC_QUEUE_BAN_REASON_PAGE_OFFLINE (bit 1): exec queue was
  banned because a VRAM page backing its resources was taken offline.

The ban_reason field is added to struct xe_exec_queue and set at the
point where the ban is triggered:
- In guc_exec_queue_timedout_job() for GPU hang.
- In xe_ttm_vram_purge_page() for memory page offline, before calling
  xe_exec_queue_kill() or xe_vm_kill().

The reset_status op is updated to return u64 with the reason bitmask.
When a queue is banned but no explicit reason was recorded (e.g., from a
generic CAT error), it defaults to GPU_HANG for backward compatibility.
A value of 0 means the exec queue is not banned.

v5 (Sashiko/MattB):
- Take the write lock for the traversal to tag ban_reason
v4(Sashiko):
- Add ban reason for non-LR exec queues
- Add TODO for multiqueue
v3(Rodrigo):
- Add doc in xe_drm.h
v2(Sashiko):
- Use atomic_t for ban_reason to fix concurrent updates from TDR and
  page-offline
- Guard GPU_HANG bit with !exec_queue_killed to avoid masking
  page-offline reason
- Clear ban_reason on queue recovery (clear_exec_queue_banned path)
- Use atomic_read in guc_exec_queue_reset_status for lockless read

Assisted-by: Copilot:claude-opus-4.6
Acked-by: José Roberto de Souza <jose.souza@intel.com>
Acked-by: Michal Mrozek <michal.mrozek@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-31-tejas.upadhyay@intel.com
2026-09-03 22:30:13 -07:00
Tejas Upadhyay
fb19798cc3 drm/xe: Expose bad VRAM pages via debugfs
Add a debugfs file "vram_bad_pages" that shows offlined and queued
VRAM pages across all tiles. Each entry displays the page frame number,
GPU page size, and status flag (R=reserved, P=pending, F=failed).

example,
cat /sys/kernel/debug/dri/0/vram_bad_pages

max_pages: 10000
0x0000000000000000 : 0x0000000000001000 : R
0x0000000000001234 : 0x0000000000001000 : P
0x0000000000080000 : 0x0000000000001000 : R   ← tile 1 addr

v3(Michal):
-add kernel doc
v2(Sashikoi/Michal/Himal):
-Remove block iteration, use offline and queue list only
-Move platform check inside api

Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-30-tejas.upadhyay@intel.com
2026-09-03 22:30:10 -07:00
Tejas Upadhyay
6ff99f4335 drm/xe/vram: Check disable_vram_page_offline policy in fault handler
Before reserving a page at a faulting address, check the cached
disable_vram_page_offline policy from xe->ras. If the policy is
enabled (logging only), log the corrupted address and return
-EOPNOTSUPP so that RAS can report to firmware to drop the
address from the SRAM queue without attempting to offline the
page.

Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-29-tejas.upadhyay@intel.com
2026-09-03 22:30:08 -07:00
Tejas Upadhyay
586041a291 drm/xe/ras: Cache disable_vram_page_offline policy at init
The configfs disable_vram_page_offline attribute can only be
set before device bind, so its value is immutable at runtime.
Cache it in struct xe_drm_ras during xe_ras_init() to avoid
repeated configfs lookups on every fault.

Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-28-tejas.upadhyay@intel.com
2026-09-03 22:30:05 -07:00
Tejas Upadhyay
12f8e9ed28 drm/xe/configfs: Add disable_vram_page_offline attribute
Add a new configfs attribute 'disable_vram_page_offline' to control
how bad VRAM pages are handled:
  0, n, N, false - Do not disable (Offlining is active - default)
  1, y, Y, true  - Disable vram page offline (Logging only)

The attribute can only be set before binding to the device and defaults
to false (offlining enabled). This gives administrators control over
whether corrupted VRAM pages detected by hardware (e.g., ECC errors)
are actively offlined or only logged.

v3(Sashiko):
- fix doc
v2(Sashiko):
- keep default value in case configfs not present

Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-27-tejas.upadhyay@intel.com
2026-09-03 22:30:02 -07:00
Tejas Upadhyay
7737f819e9 drm/xe/vram: Add VRAM page offline fault handler
Add the core VRAM page offlining logic to handle HW-reported faulty
physical addresses:

- xe_ttm_vram_purge_page(): Purges the BO containing the faulty
  address. Bans the associated VM (if page table BO) and exec queue
  (if LRC BO). Moves xe_exec_queue_kill() outside xe_bo_lock() to
  avoid AB-BA deadlock with vm->lock. Uses READ_ONCE(bo->q) to
  safely access the exec queue pointer.

- xe_ttm_vram_page_already_processed(): Checks if an address is
  already tracked in offlined_pages or queued_pages lists to avoid
  double-processing.

- xe_ttm_vram_reserve_page_at_addr(): Two-phase reservation that
  first queues the page, purges the BO outside the lock, then
  reserves the buddy block. Handles both allocated (BO present)
  and free page cases. Returns -EIO for critical kernel BOs to
  trigger system reset.

- xe_ttm_vram_addr_to_region(): Maps a DPA to its corresponding
  VRAM region. Checks if the address falls within usable space,
  or infrastructure zones (CCS, GSM, DSM) where it returns NULL
  to flag a reset path. If the target address is outside any
  known region returns ERR_PTR(-EOPNOTSUPP)

- xe_ttm_vram_handle_addr_fault(): Entry point called by RAS.
  Returns -EEXIST if already processed, -EIO for GSM/critical BO,
  -EOPNOTSUPP if out of bounds.

v15:
- Add immediate reservation for free pages
- Use lock before accessing tbo
v14(Sashiko/MattB):
- Retry queued pages during xe_ttm_vram_mgr_del()
- Move all changes to find owner of bo to its API
v13(Himal):
- Remove redundant code
v12(Sashiko):
- Handle multi tile and add assert for 4K align
- Remove unaligned action comment for ENXIO
v11(Himal):
- match everywhere with enum vs bool for status member
- Fix comment and remove unused var
- if purge fail let next alloc confirm failure
- pass absolute address, useful for multi tile

Assisted-by: Copilot:claude-opus-4.6
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-26-tejas.upadhyay@intel.com
2026-09-03 22:29:52 -07:00
Tejas Upadhyay
556774b11e drm/xe/vram: Add page offline data structures and lifecycle
Add xe_ttm_vram_offline_resource to track individual offlined VRAM
pages, and extend xe_ttm_vram_mgr with offlined_pages/queued_pages
lists and their counters.

Initialize the lists in __xe_ttm_vram_mgr_init() and add
xe_ttm_vram_free_bad_pages() to release all tracked pages during
xe_ttm_vram_mgr_fini() teardown.

v3(Sashiko):
- Reorder xe_ttm_vram_buddy_free and list_del_rcu
- Introduce reservation status
v2(Himal):
- Address possible leak in xe_ttm_vram_mgr_fini()
- Remove unused dev and add comment for used_visible_size 0

Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-25-tejas.upadhyay@intel.com
2026-09-03 22:29:49 -07:00
Tejas Upadhyay
afc588a890 drm/xe/vram: Extract buddy allocation and free helpers
Factor out xe_ttm_vram_buddy_alloc() and xe_ttm_vram_buddy_free()
from xe_ttm_vram_mgr_new() and xe_ttm_vram_mgr_del().

These new helpers consolidate block allocation/deallocation with
visible-size tracking. This modularization makes the logic reusable by
the upcoming VRAM page offlining reservation path.

No functional change.

Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-24-tejas.upadhyay@intel.com
2026-09-03 22:29:46 -07:00
Tejas Upadhyay
f34e06068b drm/xe: Guard teardown paths against purged BOs
VRAM page offlining can purge BOs that are still referenced by page
tables, exec queues, and DMA-buf exports. Add xe_bo_is_purged()
guards in the teardown paths to prevent unpinning or mapping an
already-purged BO:

- xe_bo_unpin_map_no_vm(): skip unpin if purged
- xe_dma_buf_map(): return -ENOENT early if purged
- xe_exec_queue_update_run_ticks(): skip LRC timestamp read if purged
- xe_pt_destroy(): skip unpin if purged

v4(Sashiko):
- Move bo purge check in xe_bo_unpin() to handle all instances
v3(Sashiko):
- Remove dma_resv is already held
v2(Himal):
- take dma_resv lock before calling xe_bo_is_purged()

Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-23-tejas.upadhyay@intel.com
2026-09-03 22:29:43 -07:00
Tejas Upadhyay
e86c44c4c9 drm/xe/bo: Make xe_bo_is_user() public
Export xe_bo_is_user() so it can be used by the VRAM page offline
code to distinguish user-created BOs from kernel BOs when deciding
whether a faulty page can be safely purged or requires a full reset.

Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-22-tejas.upadhyay@intel.com
2026-09-03 22:29:40 -07:00
Tejas Upadhyay
529f45af3e drm/xe: Handle NULL resource and allow purging of VRAM pages
Add NULL checks for the buffer object's resource pointer in
xe_bo_is_vram() and xe_bo_is_stolen() to prevent potential NULL
pointer dereferences when checking memory location during page
offlining.

Additionally, remove the `!ttm_bo->ttm` check from
xe_ttm_bo_purge(). This restriction is stripped away to
explicitly allow the driver to purge VRAM pages during the
memory page offline sequence.

v2(Sashiko):
- Split export and this patch also Check NULL res in
  xe_bo_is_vram() and xe_bo_is_stolen()

Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Andi Shyti <andi.shyti@linux.intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-21-tejas.upadhyay@intel.com
2026-09-03 22:29:35 -07:00
Tejas Upadhyay
1faafb979f drm/xe: Export xe_ttm_bo_purge()
Remove the static qualifier from xe_ttm_bo_purge() and add its
prototype to xe_bo.h. This allows the function to be called
from other parts of the driver outside of xe_bo.c, specifically
needed for the memory page offline feature.

Cc: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Arvind Yadav <arvind.yadav@intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-20-tejas.upadhyay@intel.com
2026-09-03 22:29:32 -07:00
Tejas Upadhyay
4356370caf drm/xe: Link LRC BO and its execution queue with safe lifetime rules
Introduce an execution queue back-pointer (`q`) within `struct xe_bo`, primarily
for Logical Ring Context (LRC) Buffer Objects. This back-pointer allows the
driver to identify and execute targeted corrective actions on a specific queue
if its associated LRC BO encounters errors like memory corruption or eviction.

Because this back-pointer takes no reference on its target execution queue, strict
lifetime and serialization rules are implemented to prevent concurrent readers
from encountering use-after-free or dangling pointer bugs:

- Encapsulate tracking logic inside xe_exec_queue_set_lrc_bo_backpointer() and
  xe_exec_queue_clear_lrc_bo_backpointer().

- Explicitly wrap all back-pointer writes and clears under the BO's dma_resv lock
  via xe_bo_lock(). Readers must hold this same lock across both the pointer read
  and its subsequent xe_exec_queue_get_unless_zero() call to guarantee serialization
  against teardown.

- Defer publishing the back-pointer until the very end of xe_exec_queue_create().
  This ensures that early initialization failure paths (which bypass the kref
  mechanism and immediately free the queue structure) never leak a transient pointer
  to concurrent readers.

- Clear the back-pointer at the absolute top of __xe_exec_queue_fini(). This strips
  the pointer before q->ops->fini() destroys the hardware backend, ensuring that
  any reader holding the BO lock either observes a fully functional queue or NULL.

For multi-queue engines, secondary LRC BOs safely point to the primary queue, which
is guaranteed to outlive the teardown pass due to active references held by its
secondaries.

V4 (MattB):
  - Add LRC BO's execution queue safe lifetime rules
V3 (Sashiko):
  - Use 8-byte placeholder structure compatibility for non-LRC BO cases.
  - Wrap assignments and clears securely under dma_resv locks.
V2 (Matt B):
  - Add native support handling multi-queue configuration tracking.

Assisted-by: Copilot:claude-opus-4.6
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-19-tejas.upadhyay@intel.com
2026-09-03 22:29:21 -07:00
Tejas Upadhyay
c1b1906f86 drm/xe: Link VRAM resource allocation with gpu buddy
Setup to link allocation resource structure inside
gpu buddy. This functionality is critical for
supporting the memory page offline feature on CRI,
where identified faulty pages must be traced back to their
originating buffer for safe removal.

v3(MattB): store memory resource(res) instead of tbo
V2(MattB): Clear block->private in xe_ttm_vram_mgr_del as well

Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260903161553.528932-18-tejas.upadhyay@intel.com
2026-09-03 22:29:18 -07:00
Jonathan Cavitt
88064894bc drm/xe/vm: Add srcid to xe_vm_get_property_ioctl fault report
Add the SRCID of the faulting hardware unit to the return of the
xe_vm_get_property_ioctl fault report.

v2:
- Readd pad check, as the pad in the ioctl struct was not changed
  (jcavitt)

v3:
- Squash SRCID with ASID to keep the struct compact (Matthew)

Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: David Airlie <airlied@gmail.com>
Cc: Simona Vetter <simona@ffwll.ch>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260902214358.348399-3-jonathan.cavitt@intel.com
2026-09-03 13:38:44 -07:00
Jonathan Cavitt
e0ba981c31 drm/xe/pagefault: Add SRCID to pagefault struct
Add SRCID information to pagefault struct for the purpose of reporting
the hardware unit that resulted in the pagefault.

v2:
- Squash SRCID with ASID to keep the struct compact (Matthew)

v3:
- Use BUILD_BUG_ON and move ASID definition in one function (Matthew)

v4:
- Various formatting fixes

Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: David Airlie <airlied@gmail.com>
Cc: Simona Vetter <simona@ffwll.ch>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260902214358.348399-2-jonathan.cavitt@intel.com
2026-09-03 13:38:42 -07:00
Matthew Auld
c874bc70c8 drm/xe/vram: add early VRAM health check
During early probe, use the last page as a canary for BAR sizing, CCS
sizing, identity map setup etc. If something is wrong the last page is
where we will likely find it. Hit it with everything we have. For now
this is gated behind a debug config option, so shouldn't trigger on
production.

Main motivation is around CCS sizing where on some BMG cards the CCS
offset is programmed misaligned, for whatever reason, and our handling
of that was busted, as found by Linus, leading to some amount of CCS
storage getting pulled into the allocator as normal VRAM.

Nothing in our CI farm has such a misaligned offset it would seem,
however I did get this to pop on my b570, which does also have the
misaligned CCS offset:

  Tile 0: Running VRAM memtest...
  Tile 0: VRAM bounds overlap CCS region! VRAM sizing is incorrect.

With the fix from Linus, this goes away:

  Tile 0: Running VRAM memtest...
  Tile 0: VRAM memtest completed.

For the VRAM health check itself, this adds:

  - CPU access to the last page (BAR).
  - GPU access to the last page (identity map).
  - CCS overlap check. This one is more involved, but overall idea is
    fill the last page with a known pattern, and also save the CCS state
    for the first 4M of VRAM to some scratch memory. We then zero the CCS
    storage for that same range, all using the proper CCS copy instruction.
    At this point we readback the last page, and check if the pattern we
    wrote changed. Finally we restore the CCS state. This works since CCS
    1:1 maps with VRAM, so the start of the raw CCS, should map to the
    start of VRAM. This successfully catches the issue that Linus found
    and fixed.

Assisted-by: Gemini:gemini-3.1-pro-preview
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Stuart Summers <stuart.summers@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260902124117.918018-12-matthew.auld@intel.com
2026-09-03 11:33:26 +01:00
Matthew Auld
6df8aa0e8a drm/xe: add force option for global invalidation
The flush is gated by the WA, since there would no valid use for it
otherwise, but for some very special usecases allow bypassing the WA.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260902124117.918018-11-matthew.auld@intel.com
2026-09-03 11:33:24 +01:00
Matthew Auld
bab69ad31c drm/xe/vram: revamp CPU VRAM mapping
Previously, we called devm_ioremap_wc() for the entire PCI LMEM BAR (which
can be significantly larger than usable memory, e.g., mapping 16G for a
10G card), and then simply assigned subsets of this global mapping to each
tile.

By moving the devm_ioremap_wc() call into vram_region_init() and mapping
on a per-tile basis, we restrict the virtual address space to exactly
the usable_size of each tile. The other big win is that the core kernel
will place a guard page at the end of each per-tile mapping to help
catch OOB CPU writes (e.g. into the flat CCS storage) by triggering an
immediate page fault instead of silent memory corruption.

As a consequence the global vram->mapping is now NULL. But that was
unused anyway, with CPU access already correctly routed through the per
tile mapping.

v2 (Sashiko)
  - Make sure to update the panic flow to now use the root tile.

Assisted-by: Gemini:gemini-3.1-pro-preview
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260902124117.918018-10-matthew.auld@intel.com
2026-09-03 11:33:23 +01:00
Matthew Auld
d00b7f4f03 drm/xe/vram: report FLAT_CCS base misalignment
So we can easily check if a machine had the CCS bug, when looking back
over bug reports where we have the same machine with newer kernel.

Example print for a machine with the CCS bug:

  FLAT_CCS base:27bbff800, aligned:no

v2 (Matt B):
  - Unconditionally print the base + alignment

Fixes: 3717339274 ("drm/xe/vram: fix ccs offset calculation")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: stable@kernel.org
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260902124117.918018-9-matthew.auld@intel.com
2026-09-03 11:33:21 +01:00
Matthew Auld
188d813994 drm/xe/migrate: support 4K PTEs for identity map
When VRAM limit (usable_size) is no longer 2M aligned (e.g. because flat
CCS storage size rounds it down to 4K), we must map the tail of the
identity mapping using 4K pages.

This removes the 2M alignment assert (which pops) and introduces a 4K
fallback in the identity map programming loop to handle the last 2M
chunk of usable VRAM.

This now also ensures we are only mapping the usable portion of VRAM.

Might be possible to skip backporting this. The assert that pops should
be harmless and is only on debug builds plus only a stray write would
hit CCS, which technically get mapped.

Assisted-by: Gemini:gemini-3.1-pro-preview
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260902124117.918018-8-matthew.auld@intel.com
2026-09-03 11:33:20 +01:00
Matthew Brost
cc46796964 Merge drm/drm-next into drm-xe-next
Prep drm-xe-next for 7.4 PR.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
2026-08-31 15:42:28 -07:00
Lu Yao
2c6fbda5fd
drm/xe/oa: Remove sysfs entry on idr_alloc failure in xe_oa_add_config_ioctl()
If idr_alloc() fails after create_dynamic_oa_sysfs_entry() has
succeeded, the error path frees the OA config without removing the
metrics sysfs group.

Remove the sysfs group before releasing the config, and fix up the
misleading error message copied from the sysfs creation failure path.

Fixes: cdf02fe1a9 ("drm/xe/oa/uapi: Add/remove OA config perf ops")
Signed-off-by: Lu Yao <yaolu@kylinos.cn>
Link: https://patch.msgid.link/20260831014218.28515-1-yaolu@kylinos.cn
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2026-08-31 17:18:02 -04:00
Michał Grzelak
68b9663102 drm/xe/pcode: match argument naming with PCODE_DATAx
Variable naming of val and val1 should match register macro naming.

For xe, register macro naming of PCODE_DATA0 and PCODE_DATA1 is already
synced with register name. Nevertheless there are still functions
declaring val and val1. Rename val into val0 in functions declaring val1
in xe.

v4->v5
- clean changelog's versioning

v3->v4
- split i915 changes from xe (Jani)

v2->v3
- resolve conflict from rebase

v1->v2
- rebase onto drm-tip (Jani)

Bspec: 19978, 19979
Suggested-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Michał Grzelak <michal.grzelak@intel.com>
Link: https://patch.msgid.link/20260828190956.2622350-3-michal.grzelak@intel.com
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2026-08-31 12:06:16 +03:00
Linus Torvalds
cee9395acd Linux 7.3-rc1 v7.3-rc1 2026-08-30 13:34:40 -07:00
Linus Torvalds
78bb208b99 i2c-fixes for v7.3-rc1
Fixes mainly for teardown and resource handling, runtime PM and
 hardware-specific controller issues.
 
 - core: fix debugfs use-after-free when removing the adapter
 - designware: apply interrupt mask quirk for HJMC3001
 - imx-lpi2c: avoid target accesses on master-only controllers
 - mux: release channel node when adapter registration fails
 - qcom-cci: fix autosuspend and runtime PM cleanup on removal
 - qcom-geni: fix timing parameters for 32 MHz clock
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQScDfrjQa34uOld1VLaeAVmJtMtbgUCapRFYgAKCRDaeAVmJtMt
 bpVsAQDz0v5x5AdiZQS224oErSA6vfd7ztavFkh8gg3tR+PgogEAhhV+xChRle4U
 DTwvfC2DYuzxJxYpuQbMGdvu39cuTgA=
 =t/+3
 -----END PGP SIGNATURE-----

Merge tag 'i2c-fixes-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/andi.shyti/linux

Pull i2c fixes from Andi Shyti:
 "Fixes mainly for teardown and resource handling, runtime PM and
  hardware-specific controller issues:

   - fix debugfs use-after-free when removing the adapter

   - designware: apply interrupt mask quirk for HJMC3001

   - imx-lpi2c: avoid target accesses on master-only controllers

   - mux: release channel node when adapter registration fails

   - qcom-cci: fix autosuspend and runtime PM cleanup on removal

   - qcom-geni: fix timing parameters for 32 MHz clock"

* tag 'i2c-fixes-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/andi.shyti/linux:
  i2c: core: fix debugfs UAF on adapter removal
  i2c: imx-lpi2c: avoid accessing target registers on master-only controllers
  i2c: qcom-cci: fix autosuspend cleanup
  i2c: designware: Enable interrupt mask workaround for HJMC3001
  i2c: qcom-geni: update frequency table to fix timing parameters
  i2c: mux: Fix channel node leak on adapter add failure
2026-08-30 10:53:24 -07:00
Linus Torvalds
eea8bdcb59 Coccinelle patches for Linux v7.3
Sang-Heon Jeon <ekffu200098@gmail.com> has cleaned up a number
 of the semantic patches in the scripts/coccinelle directory,
 particularly with respect to functions that no longer exist in
 the kernel.  He and I have also done some reorganizations that
 improve performance.
 
 I eliminatd some false positives.
 
 相浦彰 / AIURA,AKIRA <akira.aiura.bj@hitachi.com> fixed an
 out of date URL.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEnGZC8gbRfLXdcpA0F+92B3f5RZ0FAmqUJzoACgkQF+92B3f5
 RZ1ZKA//XmJPmm4VapiaWb3Kd9R11JXEqUrkDMkexkwpa+844xC6XfUKsAJTsxff
 18UksaPH+GQyLInWtu271uPDL+3X/yXbSU0iQA675XVQsG9Bz9MJ3R7tDcmFiJR9
 Oflm7VwlG0QFVk1HVJRtY5+kfZ5ob54CZ82QltHCRIrPKQJ9JWqS8ffnNggLQu91
 D079YK9gL/SqTXsWSUg+oU/TN5zILAZ0XDAdqN5THuMtq3ADurlKoPFeGHTiXdDs
 rFlmbu3XX40aL06qun6B4WLD/86fpvaVQ9NwqJnTEoTA4GNwENY/wYY9GJdsWa7h
 N7bTB7R8g441orRx0CeGyFBJkNocg4V5ImT42aGtsxfgIxDvFPxcfVg3TnCSbO8/
 SYvvnzaIzdYkx41IIIHPLZHNbW96kbT8rI5s3DrkUoKvMePnCtrV/6bR1QFBHdiw
 I5/q4yhFjwvRh5ffiU5McqlYcM4NK1s3lHD+fk6oKf8XOgPzo6b1nBFky+P4Whw+
 Sy/uBM2TrafRlvRrctz0EdhKSQede/Q7OVFcJKOvUUwnYKRqmrM5qPIQvDg6s4zZ
 7NQWK7SsfWFKFUd1mljqN9Pe//fLIxfHKqyQwWtoPJwPmVaKSUTF/UDHu5HR4VVn
 wd4MxZj1ck3Dv/mtvAcJNxKw3+imD+284wPkpsrityMDpdYRHoc=
 =AgiL
 -----END PGP SIGNATURE-----

Merge tag 'cocci-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jlawall/linux

Pull Coccinelle updates from Julia Lawall:

 - Clean up a number of the semantic patches in the scripts/coccinelle
   directory, particularly with respect to functions that no longer
   exist in the kernel (Sang-Heon Jeon)

   He and I have also done some reorganizations that improve
   performance.

 - Eliminate some false positives (me)

 - Fix an out of date URL (相浦彰)

* tag 'cocci-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jlawall/linux:
  coccinelle: ifnulldev_put: update error message
  coccinelle: ifnulldev_put: update outdated helper names
  coccinelle: atomic_as_refcounter: drop atomic_long_dec_and_lock
  coccinelle: kfree_mismatch: drop vmalloc_exec
  coccinelle: pool_zalloc-simple: drop the pci_pool_alloc rules
  coccinelle: zalloc-simple: drop the kmem_alloc rules
  coccinelle: alloc_cast: drop removed allocators
  coccinelle: remove obsolete pci_free_consistent.cocci
  scripts: coccinelle: devm_free: reduce false positives
  coccinelle: misc: struct_size: drop unneeded parentheses
  coccinelle: mini_lock: improve performance when searching loops
  coccinelle: api: check for macro context
  coccinelle: update Coccinelle website URL
  coccinelle: misc: minmax: avoid unhelpful isomorphisms
  coccinelle: misc: minmax: check for the presence of if cases
  coccinelle: misc: minmax: drop unneeded parentheses
  coccinelle: misc: minmax: improve performance when no candidate exists
  coccinelle: double_lock: improve performance when no double lock exists
2026-08-30 10:42:40 -07:00
Darrick J. Wong
cbb4c6d9af CREDITS/mailmap: add some info about Darrick J. Wong
Add myself to CREDITS because apparently I've never done that; and
update mailmap so that all my old email addresses get remapped to the
kernel.org redirector.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-08-30 10:29:08 -07:00
Linus Torvalds
a23cbb0574 - Fix UM build regression caused by the removal of the UM
specific timex.h header. (Thomas Weißschuh)
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqTmUsRHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1iFihAAtSFpL6WKwu+eTTdTn/J41OCx6SWMw79O
 GRXfHX7HKVS52IhEnh3OWDVcMAqhNXe8oP/JwzBn4EuCv2LrsuBxpiTp6Bq4Y0Da
 Ws16Nb0CRH966L9E6z0kMZQBMsKrLGXGOE+/t9gzHpkTBzJ4a7Dzh1TZX2YudhRO
 lrnS+7azTb5XyshyMDo0C43AH6rUpt1ffYUz0MyACkbWPzQge42O3neN/2gKMDjq
 TM128Man3V4AX6K/9hC4vnvDNYdHsUqxHhc81826wVX7brF74uoYSiogn/AZK7RY
 CK9GQmAv7xo7AJTqe9AKt5dCy/cvVLp4vLqVYMQMGaFB05PF+1EMY69xrAkWnS4k
 1cYnU0hCHN/86XOShe1s0101UPD6ZxYo/WlU53VGRJMNi7Dm5OMFFOGzAK9WfTIB
 4UyOwuzq/505aA8dr5+Hk5PYikDvDXOA/zmqewAMonDrlTWhPBxWV6k+Dhks8WOt
 MIgjoX8wFlA7h0SUQH12XN/pxEBTgNl4P4k8YyHqVwOKN/XPZJ6XPGt2nIc8hnVc
 m3wHxP27x/68Ugpz1a8oADkC6lhDFgTE5XO6mmpfHKaVp/Fwdjrg8atwltHbixiV
 4mo5ZlcI759lN40EBJM4QbGRP8k5TjRGgkUnv7B2qpBBgAvPnCnlMZo0871J6VVz
 0CeKb6kxT+k=
 =eGi8
 -----END PGP SIGNATURE-----

Merge tag 'timers-urgent-2026-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull timer fix from Ingo Molnar:

 - Fix UM build regression caused by the removal of the UM
   specific timex.h header (Thomas Weißschuh)

* tag 'timers-urgent-2026-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  um: Use asm-generic/timex.h over the host architecture one
2026-08-30 10:19:11 -07:00