In xe_vm_close_and_put(), external-BO VMAs are queued on the contested
list for deferred destruction via xe_vma_destroy_unlocked(). However,
xe_vm_pt_destroy() was previously invoked before processing contested
VMAs, destroying vm->pt_root while those VMAs were still linked to their
respective buffer objects (vm_bo->list.gpuva).
If a concurrent thread evicts one of those shared buffer objects,
xe_bo_trigger_rebind() holding only bo->resv walks the BO's VMAs and, in
fault mode, calls xe_vm_invalidate_vma() -> xe_pt_zap_ptes(). Because
vm->pt_root[tile->id] is already NULL, dereferencing pt->level causes a
NULL ptr deref.
Fix this by deferring xe_vm_free_scratch() and xe_vm_pt_destroy() until
after all contested VMAs have been unlinked and destroyed.
User is reporting hitting a NULL ptr deref in xe_pt_zap_ptes(), which
could be explained by this race.
Assisted-by: LLM
Fixes: b06d47be7c ("drm/xe: Port Xe to GPUVA")
Link: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/9290
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: <stable@vger.kernel.org> # v6.12+
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260918131034.598078-2-matthew.auld@intel.com
Ensure GT C6 state before D3 transition in runtime suspend flow.
This avoids the scenario where GT is being accessed after D3 state
is programmed. If the check for C6 fails, resume the device so it is
not stuck in an intermediate state.
v2: Force resume when we cancel suspend due to C6 check (Sashiko)
v3: Abort the suspend properly when idle check fails (Sashiko)
v4: Only check for C6 during suspend (Rodrigo). Have check in
xe_pci_runtime_suspend() instead of xe_pci_suspend().
v5: Don't checkfor PVC (Sashiko), rename function from_device_c6
to _device_idle
v6: Rename to _wait_all_c6 and move PVC check to gt_idle (Rodrigo)
Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260916190614.3047807-1-vinay.belgaumkar@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Drop the definition of MI_SEMAPHORE_WAIT_TOKEN and replace its uses with
MI_SEMAPHORE_WAIT. Both #define's were representing the same opcode
0x1C and this GPU instruction has always been named "MI_SEMAPHORE_WAIT"
in the official specs.
The "*_TOKEN" definition appears to have been copied from i915's code.
i915 needed two separate definitions to reflect the slightly different
instruction lengths gen8-gen11 platforms (4 dwords) vs gen12 and later
platforms (5 dwords, with the extra dword representing a token), but Xe
only supports newer platforms with the 5-dword form, so only a single
definition is needed and should match the official name given in the
specs.
Bspec: 14054, 45749, 60244
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260917-semaphore_inst-v1-1-61e49405f71d@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
pci_p2pdma_distance() can optionally print details about how it
computed the ACS/PCIe topology distance between the two devices, but
xe_dma_buf_attach() always called it with verbose set to false,
silently hiding useful diagnostics whenever a dma-buf attach unset
attach->peer2peer.
Tie the verbose argument to a dedicated dynamic-debug callsite tagged
"xe dma-buf p2p debug" (in addition to keeping the existing
DRM_UT_DRIVER category as a default), so the extra p2pdma distance
logging can be switched on without a recompile in any of the following
ways:
- Enable the whole DRIVER category for xe via drm.debug, which is
already used elsewhere in the driver:
# echo 0x10 > /sys/module/drm/parameters/debug
- Enable dynamic debug for the whole file:
# echo 'file xe_dma_buf.c +p' > \
/sys/kernel/debug/dynamic_debug/control
- Enable dynamic debug for just this callsite by function name:
# echo 'func xe_dma_buf_attach +p' > \
/sys/kernel/debug/dynamic_debug/control
- Enable dynamic debug by matching the format string used to
register the callsite:
# echo 'format "xe dma-buf p2p debug" +p' > \
/sys/kernel/debug/dynamic_debug/control
- Enable it at boot via the dyndbg= kernel command line parameter:
dyndbg="func xe_dma_buf_attach +p"
Any of the above flips verbose_p2p_dist to true for that attach call,
causing pci_p2pdma_distance() to log its distance calculation without
requiring the whole DRM_UT_DRIVER category to be enabled.
v2:
- Guard the _dynamic_func_call() invocation and
xe_ddebug_flip_bool() with a CONFIG_DYNAMIC_DEBUG check so the
build does not break when dynamic debug is disabled. (Sashiko-bot)
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Assisted-by: LLM
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patch.msgid.link/20260914114617.112012-1-thomas.hellstrom@linux.intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
drm_pagemap_dev_unhold_work() processes an unbatched llist of pagemaps
on the system workqueue. This creates potential latency traps during
heavy teardown cycles due to two compounding factors:
1. The loop contains a per-item drm_dbg() trace. If dynamic debug is
enabled on a slow serial console, synchronous print latency scales
linearly with the batch size and can block worker execution.
2. The llist can accumulate large batches during intensive unmap
operations, monopolizing worker resources without yielding.
This causes below log warning:
"workqueue: drm_pagemap_dev_unhold_work [drm_gpusvm_helper] hogged CPU
for >10000us 19 times, consider switching to WQ_UNBOUND"
Fix this by implementing a latency mitigation strategy:
- Move the teardown work to system_unbound_wq to avoid tying up
per-CPU bound worker pools.
- Add cond_resched() inside the teardown loop to yield the CPU during
large batch teardowns, ensuring system responsiveness.
- Convert the drm_dbg() trace to drm_dbg_ratelimited() to mitigate
serial console bottlenecks while preserving debuggability.
v2:
- Changed workqueue from deprecated system_unbound_wq to system_dfl_wq.
(Arvind Yadav)
Fixes: a26084328a ("drm/pagemap, drm/xe: Manage drm_pagemap provider lifetimes")
Reviewed-by: Arvind Yadav <arvind.yadav@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Varun Gupta <varun.gupta@intel.com>
Link: https://patch.msgid.link/20260918081320.30389-2-varun.gupta@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
The desired behavior for SVM eviction failures, which can occur due to
various uncontrollable races, is for TTM to continue walking the LRU
list and look for another eviction candidate. This is expressed by
returning -ENOSPC from the ->move() callback.
Adjust the SVM eviction failure path because of races in ->move() to
return -ENOSPC so that TTM continues searching for another buffer to
evict.
Fixes: 3ca608dc75 ("drm/xe: Basic SVM BO eviction")
Cc: stable@vger.kernel.org
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Link: https://patch.msgid.link/20260917203158.292823-1-matthew.brost@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
xe_pagefault_print() was firing without indicating why servicing of a
fault failed, making it hard to distinguish benign races (e.g. a VM's
file descriptor closing mid-fault) from real bugs.
Add enum xe_pagefault_error, uniquely identifying the high-level point
at which xe_pagefault_service(), xe_pagefault_handle_vma(), and
xe_svm_handle_pagefault() fail. Since consumer.page_addr is always
page (4K) aligned, steal the reserved low byte to carry this code
without growing struct xe_pagefault. Add xe_pagefault_set_error() /
xe_pagefault_get_error() to encode/decode it, and xe_pagefault_addr()
as the single accessor for the real (masked) faulted address; update
all real address consumers (xe_pagefault_match(), cache alignment,
xe_vm_add_fault_entry_pf()) to go through it instead of reading
consumer.page_addr directly. xe_pagefault_set_start_addr() now
preserves any already-recorded error bits rather than clobbering them.
xe_pagefault_print() decodes the error via a new
xe_pagefault_error_to_str() and reports it in an added "Error:" line.
xe_pagefault_asid_to_vm() now distinguishes an ASID with no VM at all
(XE_PAGEFAULT_ERROR_VM_NOT_FOUND, e.g. the owning file descriptor was
already closed) from a VM found but not in fault mode
(XE_PAGEFAULT_ERROR_VM_NOT_IN_FAULT_MODE).
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Assisted-by: GitHub_Copilot:claude-sonnet-5
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Link: https://patch.msgid.link/20260917203850.321385-1-matthew.brost@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
A TLB invalidation issued on a wedged device fails with
-ENOTRECOVERABLE. xe_tlb_inval_issue() squashes only -ECANCELED, so the
error reaches callers that treat it as unexpected and WARN, tainting the
kernel on a wedge that was deliberately caused:
ggtt_invalidate_gt_tlb() drivers/gpu/drm/xe/xe_ggtt.c
xe_svm_invalidate() drivers/gpu/drm/xe/xe_svm.c
xe_bo_trigger_rebind() drivers/gpu/drm/xe/xe_bo.c
xe_vma_userptr_do_inval() drivers/gpu/drm/xe/xe_userptr.c
igt@xe_exec_reset@gt-reset-fault-injection hits the GGTT one, turning an
otherwise passing run into an abort:
*ERROR* SIGID=102 FATAL (-EIO) WEDGED: Device declared wedged!
Tile0: GT1: Failed to invalidate GGTT (-ENOTRECOVERABLE)
WARNING: drivers/gpu/drm/xe/xe_ggtt.c:588 at ggtt_invalidate_gt_tlb
Workqueue: xe-guc-destroy-wq __guc_exec_queue_destroy_async [xe]
ggtt_node_remove+0xe3/0x100 [xe]
xe_ggtt_remove_bo+0x89/0x2c0 [xe]
xe_ttm_bo_destroy+0xcb/0x330 [xe]
...
xe_lrc_destroy+0x74/0x90 [xe]
xe_exec_queue_fini+0x2d/0x60 [xe]
-ECANCELED and -ENOTRECOVERABLE mean the same thing at this layer: the
message was dropped rather than delivered, and the fence has already
been signalled before the error is returned, so there is nothing left to
wait for. Squash both.
A wedged device is only recovered by a fresh initialisation, so the
error return in xe_bo_trigger_rebind() becomes unreachable by design.
v2: fix all invalidation paths. (Sashiko)
Fixes: 50fa9acac2 ("drm/xe/guc: distinguish wedged from recoverable cancellation")
Assisted-by: Claude:claude-opus-5
Cc: Sk Anirban <sk.anirban@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260914215318.200603-1-shuicheng.lin@intel.com
CRI defines bit 5 of the perf limit reasons register as a power brake
(PWRBRK) indicator. Add PWRBRK_MASK and a reason_pwrbrk sysfs attribute
for CRI in place of reason_ratl.
Signed-off-by: Sk Anirban <sk.anirban@intel.com>
Fixes: 8578e6d054 ("drm/xe/gt_throttle: Drop individual show functions")
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260909114931.1039331-2-sk.anirban@intel.com
Extend the existing "hw_engines" debugfs to iterate over all hardware
engines and dump their idle state: whether the engine pipe is idle or
busy, whether C6 entry is allowed, and (where available) the forcewake
request status.
v2: Use existing debugfs (Matt Brost)
v3: Rename macro (Vinay)
Signed-off-by: Sk Anirban <sk.anirban@intel.com>
Reviewed-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260910165855.1124574-2-sk.anirban@intel.com
Start using recently added KLV helpers for parsing migration
packet to properly detect any malformed KLV formats.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Link: https://patch.msgid.link/20260916215827.13488-4-michal.wajdeczko@intel.com
Instead of retrieving data from the KLV dwords manually, add
simple helpers that will do that together with sanity checks.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Link: https://patch.msgid.link/20260916215827.13488-2-michal.wajdeczko@intel.com
Add region 5 (System Controller) to the list of regions
available on device.
Signed-off-by: Alexander Usyskin <alexander.usyskin@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260914-nvm_add_sysctl_region-v1-1-75d710daf5d0@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
We will soon want to auto provision all VFs in a single step, while
already holding the master mutex. Split existing function into two.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com>
Link: https://patch.msgid.link/20260911183257.14999-4-michal.wajdeczko@intel.com
We will soon want to unprovision all VFs in a single step, while
already holding the master mutex. Split existing function into two.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com>
Link: https://patch.msgid.link/20260911183257.14999-3-michal.wajdeczko@intel.com
We will soon want to provision VFs in a single step, while already
holding the master mutex. Split existing function into two.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com>
Link: https://patch.msgid.link/20260911183257.14999-2-michal.wajdeczko@intel.com
Currently, struct xe_i2c is freed before SGUnit IRQ is disabled in unbind
path, leaving a potential UAF in case I2C IRQ is hit during this small
window. Explicitly disable I2C IRQ in xe_i2c_remove() and fix this.
Fixes: 0bb78ce099 ("drm/xe/i2c: Wire up reset/postinstall for I2C IRQ")
Signed-off-by: Raag Jadav <raag.jadav@intel.com>
Reviewed-by: Heikki Krogerus <heikki.krogerus@linux.intel.com>
Link: https://patch.msgid.link/20260911121547.2407261-1-raag.jadav@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
It turned out that during early probe phase, VFs use detached from
the xe_device, temporary xe_gt objects, which when used as location
in xe_log() macros, will be treated by the dmesg decoration code as
bogus, possibly triggering a WARN, and the output will look like:
[drm] *ERROR* SIGID=104 (-ETIMEDOUT) LOC3.0? GUC: MMIO request ...
instead of expected:
[drm] *ERROR* SIGID=104 (-ETIMEDOUT) Tile0: GT0: GUC: MMIO request ...
Relax the tile/GT id validation and instead of looking for the real
objects, only check if encoded id is within the range of possible
tiles or GTs on the current xe device, using data from the device
descriptor rather then the object list.
Fixes: 1151b9f6f4 ("drm/xe/log: Add component/location decorations to dmesg")
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260904170531.516-7-michal.wajdeczko@intel.com
We can now obtain the const pointer to struct xe_device_desc directly
from the xe, no need to pass it as explicit parameter.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260904170531.516-6-michal.wajdeczko@intel.com
We can now obtain the const pointer to struct xe_device_desc directly
from the xe, no need to pass it as explicit parameter.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260904170531.516-5-michal.wajdeczko@intel.com
We can now obtain the const pointers to struct xe_device_desc and
xe_subplatform_desc directly from the xe, no need to pass them as
explicit parameters.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260904170531.516-4-michal.wajdeczko@intel.com
While we already use struct intel_device_info, which has some fields
copied or derived from the original struct xe_device_desc, we also
want to keep the reference to the whole device descriptor, in case
we would like to access more non-mutable device parameters.
Similarly keep the reference to matched subplatform descriptor.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260904170531.516-3-michal.wajdeczko@intel.com
We don't need anything from struct xe_device_desc in xe_info_init()
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260904170531.516-2-michal.wajdeczko@intel.com
If a test does not specify subplatform details about the fake device
it wants to create, our helper may select a sentinel of the subplatform
descriptors list instead of the first available subplatform.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260905200157.3164-3-michal.wajdeczko@intel.com
If a test does not specify details about the fake device it wants
to create, our helper selects the device descriptor from the first
entry from the pciidlist and uses NULL for the subplatform descriptor.
While this works fine today, as the first device is a TGL which does
not have any subplatforms, it will be broken if we reorder pciidlist.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260905200157.3164-2-michal.wajdeczko@intel.com
VRAM page offlining can clear a page-table BO's vmap before VM teardown
calls xe_pt_clear(). Check for a purged BO under its dma-resv lock before
writing to the mapping.
v4(Sashiko):
- dont leave svm notifier lock early
v3(Sashiko):
- Validate NULL vmap
v2(Sashiko):
- Fix lock ordering
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260909111620.1276402-2-tejas.upadhyay@intel.com
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Drop the local PL_WRITE_MBX_TIMEOUT_MS macro and use the common
PCODE_DEFAULT_TIMEOUT_MS for the power limit write mailbox command.
This removes a duplicate timeout definition and keeps hwmon in sync with
the pcode default (now 10ms).
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Link: https://patch.msgid.link/20260907083520.192418-3-karthik.poosa@intel.com
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
The previous 1ms timeout may be insufficient for some mailbox commands,
leading to spurious -ETIMEDOUT failures under load. Increase the default
timeout to 10ms and introduce PCODE_DEFAULT_TIMEOUT_MS for a shared
timeout definition, in line with pcode mailbox specification guidance.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Link: https://patch.msgid.link/20260907083520.192418-2-karthik.poosa@intel.com
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
__xe_shrinker_walk() walks the SYSTEM and TT LRUs without a runtime PM
reference. Shrinking a bo outside system memory invalidates its GPU
mappings, which needs the device resumed, so while it is runtime
suspended the page table zap trips an assert and the TLB invalidation
returns -ENODEV:
WARNING: drivers/gpu/drm/xe/xe_bo.c:770 at xe_bo_move_notify+0x1fc/0x450 [xe]
xe_bo_shrink+0x20f/0x2b0 [xe]
__xe_shrinker_walk+0x174/0x410 [xe]
xe_shrinker_scan+0x10c/0x1e0 [xe]
do_shrink_slab+0x176/0x7e0
drop_caches_sysctl_handler+0x9c/0xf0
Take a reference before walking a memory type other than XE_PL_SYSTEM
and stop there if it cannot be acquired. Reuse the shrinker's existing
acquire path, which resumes the device directly where reclaim allows
that and otherwise queues the PM worker for a later scan. Stop the walk
once the scan target is met, so a satisfied scan does not wake the
device. System memory is still reclaimed while the device is suspended.
Gate this on xe_device_is_l2_flush_optimized(), the same condition under
which xe_bo_trigger_rebind() issues the invalidation for a non-fault-mode
vm, so reclaim is unaffected elsewhere. The System CCS copy already has
its own reference in xe_bo_shrink().
Only a non-fault-mode vm can reach this, since a fault-mode vm requires
LR mode and that holds a runtime PM reference for the vm's lifetime.
Reproduced with igt@xe_madvise@dontneed-before-exec while the GPU is
runtime suspended.
v2: simplify needs_rpm check. (Matt)
retarget Fixes tag since the issue occurs with the non-fault-mode
path added by 4e7ebff69a.
v3: handle this in xe_shrinker.c instead of xe_bo.c (Thomas)
v4: stop the walk once the scan target is met. (Sashiko)
v5: rebase on the freed page accounting fix. (Sashiko)
v6: reuse the shrinker acquire path so runtime pm can be resumed
directly instead of always queueing a worker. (Thomas)
v7: replace xe_pm_runtime_put() with xe_shrinker_runtime_pm_put(). (Thomas)
Fixes: 4e7ebff69a ("drm/xe/xe3p_lpg: flush shrinker bo cachelines manually")
Assisted-by: Claude:claude-opus-5
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patch.msgid.link/20260909162102.1097006-3-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
__xe_shrinker_walk() and xe_shrinker_walk() return either the number of
pages freed or a negative error, so the two cannot be reported at once.
On error the pages already freed are dropped, and since xe_shrinker_scan()
only accumulates non-negative returns while *scanned is updated by
pointer, the shrinker tells mm that it scanned without freeing.
Accumulate the count into a caller-provided counter and return only the
status, so an error no longer discards what the walk had freed.
Fixes: 00c8efc318 ("drm/xe: Add a shrinker for xe bos")
Assisted-by: Claude:claude-opus-5
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260909162102.1097006-2-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Add xe_sysctrl_check_app_status() to query the state of a System
Controller application using get_app_status_by_id mailbox command.
The helper maps xe_sysctrl_app_id values to firmware application
IDs and returns the reported application state. Add a convenience
wrapper to check diag firmware application readiness.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260904134935.674507-5-anoop.c.vijay@intel.com
Convert the PCI barrier mmap over to use xe_mmio_gem, which is a good
match for this functionality. This has the following advantages:
1. Removes a bunch of code.
2. Replaces the fragile hard coded fake offset design.
3. Adds the first user for xe_mmio_gem, which is preferred over nuking
it. There are also potentially other upcoming usecases wanting this
type of functionality, so having standard component to do this would
be good.
There shouldn't be any big functional change here. From userspace pov,
they still query the fake offset like before, just that now it is no
longer hard coded in the KMD.
v2 (Thomas):
- Prefer scoped_guard(). Also, just annotate ALL locations, even if
not strictly needed. Reflect that in the kernel-doc. This will also
shut up static analysis tools.
Assisted-by: LLM
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-18-matthew.auld@intel.com
Hardware MMIO registers should never be executable; reject VM_EXEC
at mmap time and clear VM_MAYEXEC to prevent later mprotect attempts.
Also drop VM_DONTCOPY so that child processes across fork() can
inherit the mapping and lazily fault in the PFNs, matching standard
DRM GEM semantics and making the existing drm_gem_vm_open() callback
functional.
This aligns with existing PCI_BARRIER, which will use this in the next
patch. We don't want any noticeable behaviour change there, since this
will be user visible. This will also be the first user.
Assisted-by: LLM
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-17-matthew.auld@intel.com
xe_mmio_gem_destroy() currently frees the GEM object directly, bypassing
reference counting. Since existing VMAs hold a reference and the fault
handler accesses the object through vma->vm_private_data, this is
use-after-free. Additionally, nothing prevents the fault handler from
installing PTEs to the real MMIO after destroy.
Fix this with proper synchronization and refcounting. Also, do not set
vm_pgoff to zero. Many DRM drivers do this because helpers like
dma_mmap_pages() interpret vm_pgoff as an intra-buffer page offset;
leaving the DRM fake offset there would break these helpers.
Those drivers can get away with zeroing it because they map eagerly -
all PTEs are established before mmap returns, so vm_pgoff is never
consulted again. Our driver does not use such helpers and the newly
introduced call to drm_vma_node_unmap() relies on vm_pgoff being untouched.
v2: (Matt Auld)
- use dma_resv lock to serialize fault handler with destroy
- SIGBUS on access after destroy
Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-16-matthew.auld@intel.com
Currently, when the fault handler provides a dummy page, it
allocates a new one on every invocation and ties its lifetime to
the drm_device via drmm_add_action_or_reset(). Concurrent faults
after hot-unplug therefore accumulate pages that persist until
device teardown.
Cache a single dummy page in the xe_mmio_gem object and use dma_resv
lock to protect its allocation. Free it with the object.
v2: use dma_resv lock to protect the allocation (Matt Auld)
Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-15-matthew.auld@intel.com
xe_mmio_gem_create() calls drm_vma_node_allow() but nothing ever calls
drm_vma_node_revoke(). The drm_vma_offset_file rb-tree entry allocated
by drm_vma_node_allow() is not freed by drm_gem_object_release(), so
it is leaked on every create/destroy cycle.
Add a struct drm_file * parameter to xe_mmio_gem_destroy() and call
drm_vma_node_revoke() from there, mirroring the drm_vma_node_allow()
call in xe_mmio_gem_create().
Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Suggested-by: Ilia Levi <ilia.levi@intel.com>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Reviewed-by: Ilia Levi <ilia.levi@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-14-matthew.auld@intel.com
Make the iteration over the addresses in the VMA more explicit.
No functional change, as the VMA matches the GEM object exactly.
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-13-matthew.auld@intel.com
Currently vmf_insert_pfn() maps the dummy page as UC, inheriting the
VMA's page protection which was set for the real MMIO region. This
conflicts with the direct map's WB mapping of the same page, creating a
cache type alias which is architecturally undefined on some platforms.
Use vmf_insert_pfn_prot() with a WB pgprot instead. Also simplify to
fault in the requested page instead of the whole VMA.
Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260525125801.975038-6-ilia.levi%40intel.com
Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-12-matthew.auld@intel.com
The fault handler assumes it always operates on a VMA spanning the entire
GEM object. This does not hold when the VMA has been split, e.g. by a
partial munmap or mprotect. In that case the handler may map wrong
physical pages or cause SIGBUS.
Handle this by forbidding VMA split, as partial unmaps are not deemed
useful for MMIO GEMs.
Suggested-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Ilia Levi <ilia.levi@intel.com>
Fixes: 1ffcf8b8ae ("drm/xe: Support for mmap-ing mmio regions")
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260908165046.1393557-11-matthew.auld@intel.com
Add a per-GT debugfs file, multi_queue_active_lrca, that prints, for
every engine supporting multi-queue, the currently active queue ID
(CSMQDEBUG) and the LRCA of the exec queue occupying that slot within
the running multi-queue group.
RING_CURRENT_LRCA only reports the primary queue's LRCA for the group
and does not update to reflect the active queue in multi-queue mode,
which makes it hard to tell which queue is actually running when
debugging multi-queue CSB/context-switch issues. Resolve the active
LRCA by matching the primary LRCA against each queue's group and
picking the queue at the reported active_id position.
v4:
- Extracted multi_queue_active_lrca into a dedicated
multi_queue_debugfs_list to prevent debugfs node registration on
platforms lacking multi-queue support entirely, via
xe_gt_has_multi_queue() gating. (Tejas)
v3:
- Use xe_exec_queue_get_lrc() instead of raw pointer dereference to
safely handle concurrent multi-queue group creation and avoid race
conditions (Sashiko)
v2:
- Maintain alphabetical order for includes and
pf_only_debugfs_list (Tejas)
- Export and reuse xe_lrc_get_multi_queue_active_queue_id() instead
of duplicate MMIO read (Tejas)
Bspec: 60321, 73976
Signed-off-by: Varun Gupta <varun.gupta@intel.com>
Link: https://patch.msgid.link/20260908033417.1019602-2-varun.gupta@intel.com
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
emit_render_cache_flush() sets PIPE_CONTROL0_HDC_PIPELINE_FLUSH to
flush the L2/HDC data cache before fence signalling, but it never
requests a flush of the LSC untyped L1 data cache via the 'Untyped
Data-Port Cache Flush Enable' bit in PIPE_CONTROL DWord0[11].
Per the Bspec, in 3D pipeline mode HDC Pipeline Flush is documented to
also flush/invalidate the untyped L1 cache, but only depending on how
HDC_CHICKEN0[13:11] is programmed. Starting with MTL, this coupling
between HDC Pipeline Flush and the untyped L1 cache flush no longer
holds in practice, regardless of how HDC_CHICKEN0 is programmed, so
relying on it is not safe on newer platforms such as BMG. Mesa's Vulkan
driver (anv) has been assuming the kernel flushes both caches between
submissions, and hit user-visible corruption in apps such as Llama.cpp
because of this gap; it now works around it by flushing both caches
again from userspace at the end of every command buffer.
Correctness between submissions on the same queue is userspace's
responsibility and belongs in Mesa, not the kernel. However, for
security we must ensure stale data can't leak through the untyped L1
dataport cache once memory is reclaimed or evicted, which requires the
KMD to flush it before releasing memory for reuse.
Prior to MTL, HDC_CHICKEN0 could be programmed (as already done for
DG2 via Wa_22010960976/Wa_14013347512) to reliably keep HDC Pipeline
Flush coupled to the untyped L1 cache flush, so those platforms are
unaffected. Mesa's own anv driver found that on MTL the HW
disconnected the two independently of how HDC_CHICKEN0 is programmed,
and could not bring the old behavior back even by writing the register
by hand; see Mesa commit 7c2ff46a4fc3 ("anv: don't prevent L1 untyped
cache flush in 3D mode"). The kernel can't reliably request the flush
from the CS on MTL either, so restrict the new PIPE_CONTROL bit to
GRAPHICS_VERx100 >= 2000 (Xe2 and later), where it can be relied on.
Explicitly set PIPE_CONTROL0_UNTYPED_DATAPORT_CACHE_FLUSH together
with PIPE_CONTROL0_HDC_PIPELINE_FLUSH in emit_render_cache_flush() on
Xe2 and later, so the L1 data cache is known clean before memory is
released for reuse, without depending on undocumented
platform-specific HDC_CHICKEN0 behavior.
Bspec: 56551
Link: 7c2ff46a4f
Fixes: 9f8f93bee3 ("drm/xe: Emit a render cache flush after each rcs/ccs batch")
Reported-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/8909
Cc: José Roberto de Souza <jose.souza@intel.com>
Cc: intel-xe@lists.freedesktop.org
Cc: <stable@vger.kernel.org> # v6.8+
Assisted-by: GitHub_Copilot:claude-sonnet-5
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260903114552.48634-1-thomas.hellstrom@linux.intel.com
Route the GT reset failure log through the structured SIGID logging helper.
Failure to complete the full graphics reset means GT reset did not finish
as expected. Use the GT component, which maps to XE_SIGID_GT_TDR, and pass
the reset errno value to the helper.
Also update the message to describe the actual failure. The driver does not
clear GRDOM_FULL directly. The reset should complete and clear it within
the timeout.
Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Suggested-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20260903054610.3640788-5-arvind.yadav@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
The proper SIGID for GuC submission failures is GT_TDR, but using the
generic GT component may hide the real source of the report.
Add GUCSUBMIT as a DRIVER component and map it to XE_SIGID_GT_TDR.
Cc: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Suggested-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20260903054610.3640788-2-arvind.yadav@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
dma_iova_try_alloc() reserves one contiguous IOVA for the whole range and
links each page at the next offset, so the device addresses run
contiguously from entry 0 and one entry describes them all. A 2 MiB range
of 4 KiB pages then drops the same 8 KiB array as a THP backed one.
Fold only when state_offset covers the full range, which proves no device
page was mapped in between, and only single page entries, so the order
kept is 0 and stays true. Widening it instead would tell a consumer to use
a huge page for npages separate CPU pages, which hangs Vega20 on amdgpu.
The kept entry no longer bounds the segment, so skip the unmap walk when
it has nothing to do, keyed off dpagemap rather than the flags, which are
not published yet on the error unwind. Consumers need the same
distinction, so drm_gpusvm_pages_first_dma() returns it alongside the
array from one read of the flags; xe passes it to xe_res_first_dma().
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-7-honghuan@amd.com
drm_gpusvm_get_pages() sizes the dma_addr array for one drm_pagemap_addr
per page, but the mapping loop advances by page order, so a range backed
by one huge page needs a single entry. For a 2 MiB THP that is an 8 KiB
array holding 16 bytes of address.
Union that entry with the array pointer, discriminated by a new
inline_dma_mapping flag. When drm_gpusvm_dma_map_pages() ends up with one
entry it stores it inline and frees the array, after the last error
unwind, which still walks the array form. An unchecked dma_addr read is
now type confusion rather than a compile error, so reads go through the
new drm_gpusvm_pages_first_dma() accessor, including the two
xe_pt_stage_bind() paths.
Only get_pages() and the free path write the union, never the notifier,
and both run under the driver lock that every address reader already
holds. The unlocked short circuit in drm_gpusvm_pages_valid_unlocked()
goes for the same reason: it cannot resolve the union, and every instance
it rejects has to be reset before the allocation loop reuses it.
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Honglei Huang <honghuan@amd.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260905133142.3628027-6-honghuan@amd.com
