Today the driver reports faults with ad-hoc drm_err()/xe_gt_err()
strings that have no stable shape. That is readable for a human, but it
gives fleet tooling nothing durable to match on: the wording changes
between releases, lines can be rate-limited or dropped under an error
storm, and there is no consistent way to ask "which recognised fault
just happened?".
Introduce a signature identifier (SIGID): a small, stable integer that
names one recognised Xe fault site and serves as the primary handle for
triage. A SIGID maps, through published end-user documentation, to a
description and a recommended action; the driver only has to emit the
right SIGID next to the usual human-readable text.
Signed-off-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Assisted-by: Copilot:Opus-4.8
Co-developed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Riana Tauro <riana.tauro@intel.com>
Cc: Stuart Summers <stuart.summers@intel.com>
cc: Badal Nilawar <badal.nilawar@intel.com>
Reviewed-by: Badal Nilawar <badal.nilawar@intel.com>
Link: https://patch.msgid.link/20260814171801.13347-3-michal.wajdeczko@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
In upcoming patches we want to define macros that will work with
either xe_device or xe_tile or xe_gt pointers. To make them work
and to allow compiler optimizations, introduce set of helpers
that will return either expected pointer type or NULL or make
necessary conversions to/from the struct xe/device/pci_dev.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Link: https://patch.msgid.link/20260814171801.13347-2-michal.wajdeczko@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Enable package and vram temperatures support from hwmon on CRI.
The required MMIO offsets of these are same as on BMG.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260813193546.511093-1-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
When PUNIT (power management unit) errors are detected that persist across
warm resets, mark the device as wedged with DRM_WEDGE_RECOVERY_COLD_RESET
and notify userspace that a complete device power cycle is required to
restore normal operation.
Signed-off-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260805071152.1225416-9-mallesh.koujalagi@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
When ``WEDGED=cold-reset`` is sent, it indicates that the device has
encountered an error condition that cannot be resolved through other
recovery methods such as driver rebind or bus reset, and requires a
complete device power cycle to restore functionality.
Signed-off-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260805071152.1225416-8-mallesh.koujalagi@intel.com
Acked-by: Maxime Ripard <mripard@kernel.org>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Introduce DRM_WEDGE_RECOVERY_COLD_RESET (BIT(4)) recovery method to handle
scenarios requiring device power cycle.
This method addresses cases where other recovery mechanisms
(driver reload, PCIe reset, etc.) are insufficient to restore device
functionality. When set, it indicates to userspace that only device power
cycle can recover the device from its current error state.
Signed-off-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link:
https://patch.msgid.link/20260805071152.1225416-7-mallesh.koujalagi@intel.com
Acked-by: Maxime Ripard <mripard@kernel.org>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
This is supposed to print the error code but there is a copy and
paste bug so it prints "bo" instead of "err".
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Signed-off-by: Dan Carpenter <error27@gmail.com>
Link: https://patch.msgid.link/an1tu0z3T-qX1ogn@stanley.mountain
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
When an interrupt is received for correctable errors indicating that error
counter has crossed its threshold, read the current counter value and
deliver a drm-ras error-event to userspace for each affected component.
Also send drm-ras error-event to userspace for uncorrectable errors on
receiving an AER.
To avoid sending duplicate events when the same component appears multiple
times in the response, send the error-event once per component.
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260810112008.1858731-6-riana.tauro@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Define a new netlink event 'error-event' and a new multicast group
'error-report' in drm_ras. Each event contains device name, node and
error information to identify the error triggering the event.
Add drm_ras_nl_error_event() to trigger an event from the driver.
Userspace must subscribe to 'error-report' to receive 'error-event'
notifications.
Usage:
$ sudo ynl --family drm_ras --subscribe error-report
Cc: Jakub Kicinski <kuba@kernel.org>
Cc: Zack McKevitt <zachary.mckevitt@oss.qualcomm.com>
Cc: Lijo Lazar <lijo.lazar@amd.com>
Cc: Hawking Zhang <Hawking.Zhang@amd.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Paolo Abeni <pabeni@redhat.com>
Cc: Eric Dumazet <edumazet@google.com>
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260810112008.1858731-5-riana.tauro@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
DRM RAS interface provides a standardized way for GPU/accelerator
drivers to expose error counters and other reliability nodes to
user space via Generic Netlink.
Since this is cross-driver infrastructure carrying its own uAPI, add a
dedicated MAINTAINERS entry covering the implementation, headers,
netlink spec and documentation, maintained through drm-misc.
Cc: Simona Vetter <simona@ffwll.ch>
Cc: David Airlie <airlied@redhat.com>
Cc: Donald Hunter <donald.hunter@gmail.com>
Cc: Jakub Kicinski <kuba@kernel.org>
Cc: David S. Miller <davem@davemloft.net>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Paolo Abeni <pabeni@redhat.com>
Cc: Simon Horman <horms@kernel.org>
Cc: Raag Jadav <raag.jadav@intel.com>
Cc: Riana Tauro <riana.tauro@intel.com>
Cc: Aravind Iddamsetty <aravind.iddamsetty@linux.intel.com>
Cc: Zack McKevitt <zachary.mckevitt@oss.qualcomm.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: David Airlie <airlied@redhat.com>
Link: https://patch.msgid.link/20260805094949.536651-2-rodrigo.vivi@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
In cases where min frequency was actually greater than BMG_MIN_FREQ,
we were not using the updated min frequency as there was a missing
call to pc_action_query_task_state() between the two settings of
min frequency. Since we know what min_freq was last set, use that
cached value while comparing to BMG_MIN_FREQ to fix this issue.
v2: pc->freq_ready is not set until after pc_adjust_freq_bounds(). Stay
with pc_action_query_task_state() instead.
v3: Update commit message (Stuart)
Fixes: bdde16c9ac ("drm/xe/bmg: Update Wa_14022085890")
Signed-off-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Reviewed-by: Stuart Summers <stuart.summers@intel.com>
Link: https://patch.msgid.link/20260805234649.2076384-1-vinay.belgaumkar@intel.com
There is no need to check the queue inside the pagefault handler
as it is immediately checked later by the xe_pagefault_queue_add()
function and we can just look for the NULL lpf entry. There will
be also already a full WARN triggered, so no need keep drm_warn().
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260811200906.10400-3-michal.wajdeczko@intel.com
It is safer and easier to use guard() instead of explicit lock
unlock calls.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260811200906.10400-2-michal.wajdeczko@intel.com
drmm_mutex_init() can fail while registering its managed cleanup action.
On failure, the reset path destroys the mutex, so continuing OA setup
leaves an unusable lock that later paths may acquire.
Return the error from per-GT OA initialization and abort device-wide OA
initialization if the metrics lock cannot be initialized.
Fixes: a9f905ae7b ("drm/xe/oa/uapi: Initialize OA units")
Fixes: cdf02fe1a9 ("drm/xe/oa/uapi: Add/remove OA config perf ops")
Signed-off-by: Linmao Li <lilinmao@kylinos.cn>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260713083053.321091-1-lilinmao@kylinos.cn
xe_oa_emit_oa_config() releases the sync entries and the syncs array
only on its success path. When it fails before the point of no return
(fence allocation, config buffer allocation or batch submission), it
returns without touching stream->syncs.
The stream open path handles such failures in the caller, but
xe_oa_config_locked() propagates the error without any cleanup, so the
syncs array and the fence references held by the parsed entries are
leaked. The next config ioctl overwrites stream->syncs, making the
memory unreachable for good.
Clean up the parsed syncs when xe_oa_emit_oa_config() fails, matching
the cleanup done by the stream open error path.
Fixes: 9920c8b88c ("drm/xe/oa: Add syncs support to OA config ioctl")
Signed-off-by: Linmao Li <lilinmao@kylinos.cn>
Reviewed-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Link: https://patch.msgid.link/20260731011932.3426219-1-lilinmao@kylinos.cn
Problem: CTX_TIMESTAMP MMIO reads could be stale if a context
switched out between check and read; LRC stores a sentinel while
a context starts that must not be treated as a real timestamp.
Fix: Check the LRC-stored sentinel before and after the MMIO read;
return the LRC value if the context switched out to avoid TOCTOU.
Note: Keep XE_LRC_CTX_TIMESTAMP_ACTIVE in xe_lrc.h as the
canonical sentinel.
Fixes: d243ef6a39 ("drm/xe/lrc: Refactor xe_lrc_timestamp to simplify logic")
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/7956
Assisted-by: GitHub-Copilot:claude-sonnet-5
Signed-off-by: Gajendra Uttamchand <gajendra.uttamchand@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Acked-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260810071812.213358-4-gajendra.uttamchand@intel.com
has_drm_ras flag is meant to facilitate drm_ras feature. Move it to the
correct layer where it belongs.
Fixes: 63dfab5786 ("drm/xe/xe_ras: Add drm_ras feature flag")
Signed-off-by: Raag Jadav <raag.jadav@intel.com>
Reviewed-by: Riana Tauro <riana.tauro@intel.com>
Link: https://patch.msgid.link/20260810124101.105832-1-raag.jadav@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Suspend/resume causes the PXP keys to become invalid, but doesn't
actually kill the session. The driver also doesn't explicitly kill and
re-start the session until a new PXP request comes in, which means that
the "zombie" session can potentially stick around if there are no new
requests from userspace. While this is not an issue for PXP, HDCP has a
new behavior starting on PTL where a communication is sent to GSC if a
session is active at suspend time (even if it doesn't have a valid key),
which can lead to delays in the suspend flow if we suspend while the
zombie session is still active.
To avoid this, we can trigger a termination on resume and kill the
zombie session immediately, instead of delaying the termination to the
next PXP request. Due to restrictions in the rpm suspend/resume flow, we
can't call the termination flow from within the resume call itself, so
the pxp irq worker is expanded to cover this scenario.
The existing logic in the worker doesn't work as-is for the new flow,
because the pm_get_if_active will fail if the worker runs before the
pci_resume call has completed (which is possible, since we queue it
from within that call) or after we're started to suspend again.
Given that we always want to run the worker after a resume (differently
from the irq case, where we want to skip if we're suspended), we can
solve this by just taking the PM reference before queueing the worker.
As part of this rework, the pxp->events variable has been moved to atomic,
to avoid having to take xe->irq.lock from non-irq related paths.
Fixes: b1dcec9bd8 ("drm/xe/ptl: Enable PXP for PTL")
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Julia Filipchuk <julia.filipchuk@intel.com>
Cc: Alan Previn <alan.previn.teres.alexis@intel.com>
Reviewed-by: Alan Previn <alan.previn.teres.alexis@intel.com>
Link: https://patch.msgid.link/20260720222757.3876338-2-daniele.ceraolospurio@intel.com
We will soon want to provision all VFs, including their GuC doorbell
IDs config, in one step. Split existing doorbells bulk configuration
function into two functions, one of which will expect the master lock
to be already taken. Use this new function in the fair GuC doorbells
provisioning.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Reviewed-by: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Link: https://patch.msgid.link/20260807164826.5650-1-michal.wajdeczko@intel.com
We will soon want to provision all VFs, including their GuC context
IDs config, in one step. Split existing GuC IDs bulk configuration
function into two functions, one of which will expect the master
lock to be already taken. Use this new function in the fair GuC IDs
provisioning.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com>
Link: https://patch.msgid.link/20260805154733.604-3-michal.wajdeczko@intel.com
We will soon want to provision all VFs, including their GGTT config,
in one step. Split existing GGTT bulk configuration function into
two functions, one of which will expect the master lock to be already
taken. Use this new function in the fair GGTT provisioning.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Piotr Piórkowski <piotr.piorkowski@intel.com>
Link: https://patch.msgid.link/20260805154733.604-2-michal.wajdeczko@intel.com
The ring buffer and the LRC context image are both mapped WC, so the ring
tail update can become visible to the device before the ring contents it
is meant to publish.
The GuC CT send does contain an xe_device_wmb(), so sending the H2G would
flush the ring contents. The problem is that it comes too late:
xe_lrc_set_ring_tail() publishes the tail before the H2G is sent, and the
device samples the tail from the context image independently of it, either
at context switch-in or while the context is already resident. A submitter
which is interrupted between updating the tail and sending its H2G
therefore leaves the device free to observe the new tail while the ring
contents behind it are not yet visible:
1. Thread A emits a job into the ring, sets the tail to T_A and sends
the H2G, which flushes A's ring contents. The GuC starts scheduling
the context in, but it is not executing yet.
2. Thread B emits a job into ring[T_A..T_B]. Those writes are not yet
visible to the device.
3. Thread B updates the ring tail to T_B. That write targets a
different page and becomes visible first.
4. Thread B is interrupted before it sends its H2G, so the flush which
would have published ring[T_A..T_B] has not happened yet.
5. The context is switched in and samples the ring tail from the
context image, picking up T_B rather than T_A.
6. The GPU executes A's job, advances HEAD to T_A, and continues on to
ring[T_A..T_B], which still holds the previous wrap's contents, so
the CS parses stale commands.
The result is command stream corruption, which typically manifests as a
hang or a spurious pagefault rather than anything that points back at the
submission path.
Kernel jobs are by far the most likely to hit this. Kernel queues such as
the migration queue are shared and can be driven by many threads
concurrently, producing back-to-back submissions on an LRC which is
already executing. User queues are typically tied to a single submitting
thread, so the same interleaving is much harder to produce.
Add an xe_device_wmb() at the end of xe_lrc_write_ring() so that it covers
every ring tail publication site, and so the invariant is local: once
xe_lrc_write_ring() returns, the ring contents are visible to the device.
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8651
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/7810
Cc: stable@vger.kernel.org
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Assisted-by: GitHub_Copilot:claude-opus-5
Reviewed-by: Stuart Summers <stuart.summers@intel.com>
Link: https://patch.msgid.link/20260807171716.140475-1-matthew.brost@intel.com
Add a new GT statistic, PARALLEL_PAGEFAULT_COUNT, to record when
multiple page fault workers are active concurrently.
When a worker dequeues a fault, scan peer workers for an active
cache entry and increment the counter if another fault is already
in flight. This provides basic visibility into parallel fault
handling behavior for performance analysis and tuning.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-13-matthew.brost@intel.com
Pagefault storms can generate long chains of acknowledgments back to the
GuC. Sending each ack as a full CT submission forces a barrier,
descriptor update and doorbell per fault.
Extend xe_guc_ct_send_locked() with a “defer-flush” mode that copies the
message into the H2G ring but defers publishing the descriptor and
ringing the doorbell. Add xe_guc_ct_send_flush() to publish pending
writes and notify GuC once per batch. Wire this into the pagefault
producer via new ack_fault_begin/ack_fault_end callbacks and CT lock
wrappers.
To avoid excessive flush latency while still amortizing MMIO costs, use
a simple periodic flush heuristic for GuC pagefault acks: batch most
acks as defer-flush and force a publish at a fixed interval (e.g., every
16th ack), with a final flush at end-of-batch.
Also increase the H2G CTB size to 16K to better absorb bursts.
Assisted-by: ChatGPT:gpt-5 # Documentation
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Francois Dugast <francois.dugast@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-12-matthew.brost@intel.com
Add a debugfs entry to dump Xe page fault queue state. The output
includes queue geometry (entry size, total size, head/tail), per-entry
allocation state counts, and whether each page fault worker cache is
currently valid.
This is intended to help debug page fault storms, chaining, and retry
behaviour without needing tracing.
Assisted-by: ChatGPT:gpt-5 # Documentation
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-11-matthew.brost@intel.com
Add GT stats to quantify pagefault chaining behavior during fault storms.
Track total chained faults, faults chained directly from the IRQ
handler, cases where IRQ chaining also drained the queue, chained faults
that had to be requeued due to range mismatch, and cases where the last
serviced range allowed immediate ack.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-10-matthew.brost@intel.com
Some Xe platforms can generate pagefault storms where many faults target
the same address range in a short time window (e.g. many EU threads
faulting the same page). The current worker/locking model effectively
serializes faults for a given range and repeatedly performs VMA/range
lookups for each fault, which creates head-of-queue blocking and wastes
CPU in the hot path.
Introduce a page fault chaining cache that coalesces faults targeting
the same ASID and address range.
Each worker tracks the active fault range it is servicing. Fault entries
reside in stable queue storage, allowing the IRQ handler to match new
faults against the worker cache and directly chain cache hits onto the
active entry without allocation or waiting for dequeue. Once the leading
fault completes, the worker acknowledges the entire chain.
A small allocation state is added to each entry so queue, worker, and
IRQ paths can safely reference the same fault object. This prevents
reuse while the fault is active and guarantees that chained faults
remain valid until acknowledged.
Fault handlers also record the serviced range so subsequent faults can
be acknowledged without re-running the full resolution path.
This removes repeated fault resolution during fault storms and
significantly improves forward progress in SVM workloads.
Since threaded prefetches now use a dedicated prefetch workqueue
(usm.prefetch_wq) rather than sharing the page fault workqueue, page
fault servicing can no longer deadlock on vm->lock, so this cache does
not need an -EAGAIN retry path for a failed vm->lock acquisition.
Assisted-by: ChatGPT:gpt-5 # Documentation
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Francois Dugast <francois.dugast@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-9-matthew.brost@intel.com
Add a GT stat measuring total time spent servicing pagefault workqueue
iterations. The counter accumulates the runtime of the pagefault worker
in microseconds, allowing correlation of fault storms and chaining
behavior with CPU time spent in the fault handler.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-8-matthew.brost@intel.com
Pack the engine class and instance fields into a single u8 to save space
in struct xe_pagefault. This also makes future extensions easier.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-7-matthew.brost@intel.com
Add a module parameter to control the number of page-fault work threads,
making it easy to experiment with how different numbers of work threads
impact performance.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-6-matthew.brost@intel.com
With fine-grained page-fault locking, it no longer makes sense to
maintain multiple page-fault queues, as we no longer hash queues based
on the VM’s ASID. Multiple workers can pull page faults from a single
queue, eliminating any head-of-queue blocking. Refactor the structures
and code to use a single shared queue.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-5-matthew.brost@intel.com
The migrate_vma_* functions are very CPU-intensive; as a result,
prefetching SVM ranges is limited by CPU performance rather than paging
copy engine bandwidth. To accelerate SVM range prefetching, the step
that calls migrate_vma_* is now threaded. A dedicated prefetch
workqueue is used for threading so prefetch work is never mixed with
page fault or garbage collector work.
Running xe_exec_system_allocator --r prefetch-benchmark, which tests
64MB prefetches, shows an increase from ~4.35 GB/s to 12.25 GB/s with
this patch on drm-tip. Enabling high SLPC further increases throughput
to ~15.25 GB/s, and combining SLPC with ULLS raises it to ~16 GB/s. Both
of these optimizations are upcoming.
Since the dedicated prefetch workqueue is not shared with page fault
or SVM garbage collector work, page fault servicing and garbage
collection can keep using a plain down_read() on vm->lock: there is no
risk of a blocked reader starving a worker that a pending writer is
waiting to flush, because that flushing is now confined to the
separate prefetch workqueue.
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Francois Dugast <francois.dugast@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-4-matthew.brost@intel.com
Prefetch-only VM bind IOCTLs do not modify VMAs or use userptr pages.
Downgrade vm->lock to read mode once setup is complete.
Lays the groundwork for prefetch IOCTLs to use threaded migration.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Francois Dugast <francois.dugast@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-3-matthew.brost@intel.com
Enable page faults to be serviced while holding vm->lock in read mode.
Introduce additional locks to:
- Ensure only one page fault thread services a given range or VMA
- Serialize SVM garbage collection
- Protect SVM range insertion and removal
While these locks may contend during page faults, expensive operations
like migration can now run in parallel within a single VM.
In addition to new locking, ranges must be reference-counted after
lookup, as another thread could immediately remove them from the GPU SVM
tree, potentially dropping the last reference.
Lastly, decouple the VM’s ASID from the page fault queue selection to
allow parallel page fault handling within the same VM.
Lays the groundwork for prefetch IOCTLs to use threaded migration too.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com>
Link: https://patch.msgid.link/20260806185202.3922432-2-matthew.brost@intel.com
On CRI and newer platforms, the power-limit interval encoding is changed
from the legacy 1.x * 2^y format to a U5.2 value in milliseconds.
Update the interval calculation accordingly for CRI and future platforms.
v2: Round the fractional bit to 1 for values greater than 0.5 ms in the
show path, as the minimum supported time is 1 ms.
No special handling is needed in the store path since sysfs inputs are
only in milliseconds. (Badal)
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Badal Nilawar <badal.nilawar@intel.com>
Link: https://patch.msgid.link/20260703153320.1368225-1-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
On Pre-Xe3p platform, the GAM write the UM queue through DPA using UC.
if GuC reads the queue via GGTT (WB), stale data may be observed
when the cacheline has been polluted by another agent.
To match the GAM's UC writes, configure the GuC mapping as UC as well.
Fixes: 9c57bc0865 ("drm/xe/lnl: Drop force_probe requirement")
Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: <stable@vger.kernel.org> # v6.12+
Signed-off-by: Jia Yao <jia.yao@intel.com>
Reviewed by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260804165057.129529-4-jia.yao@intel.com
On iGPU, the UM queue BO is allocated in system memory. On dGFX, the BO
was previously created in system memory and later reallocated in
xe_guc_realloc_post_hwconfig(). Allocate the UM queue BO directly in
VRAM on dGFX, where it is ultimately required.
Fixes: 9c57bc0865 ("drm/xe/lnl: Drop force_probe requirement")
Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: <stable@vger.kernel.org> # v6.12+
Signed-off-by: Jia Yao <jia.yao@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260804165057.129529-3-jia.yao@intel.com
Move the UM queues into a dedicated BO (ads->um_queue_bo) and avoid
CPU memset operations on it, which eliminates the CPU as a potential
cacheline-polluting agent and helps maintain consistency between GAM
writes and GuC reads.
We also need to ensure the base_dpa for the queue is contiguous on hw
where this is used instead of a GGTT address. Another good reason to
split this out to a separate BO.
Fixes: 9c57bc0865 ("drm/xe/lnl: Drop force_probe requirement")
Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: <stable@vger.kernel.org> # v6.12+
Signed-off-by: Jia Yao <jia.yao@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260804165057.129529-2-jia.yao@intel.com
- Remove dependency on deprecated DRM simple KMS helpers
. Replaces drm_simple_encoder_init() with drm_encoder_init() across dp,
dpi, dsi, vidi and hdmi.
. Adds per-driver drm_encoder_funcs with .destroy = drm_encoder_cleanup,
removing the intermediate layer between the drivers and atomic modesetting.
- Add error handling to drm_encoder_init()
. Checks the return code at every call site instead of ignoring it.
. Prevents silent encoder creation failures and drm_encoder_cleanup() on an
uninitialized encoder.
Bug fix for Exynos HDMI DDC adapter refcounting
- Take an i2c adapter module reference
. Switches to of_get_i2c_adapter_by_node() / i2c_put_adapter() so the
adapter module reference is held while the DDC bus is in use.
. Makes an unload attempt fail gracefully with -EBUSY instead of blocking
uninterruptibly in i2c_del_adapter().
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEoxi+6c5pRPV/gdXcxWAb7Og/+bYFAmpu3mwACgkQxWAb7Og/
+bZS2wwAj9i6AMtZRTtBuA2tieGVabQDTg91AHWHEhkH6+oWOCl+lsM5NTi9vuls
OyyF729TEcfIFMEM/mHwqanX1AZaKBHoBmsTgAmA6UM92cwXzrBMC6I7ueXjPBT7
ZzVxilgOWpls+RYVks6kA+XLOlk9CCvTXOuOZocwFqCAuKM0L6nw/udfo2KdqJdb
bql24T26g1Sj4AgasirewOERA3vFqKdrA5dlf72VhkXReoq/8VfMIDc1BGx2dPjB
VsRY9U/t6wvoZEwbpmGnvjBYwPSs64Y5Qw2v6Z8Q8oZbpK5Igl5LNpIPWKwEyl46
cOBHsPoG0/SC+/PO2U1v5hyy0c4gl+PiWUnrAZypthT/7anRzh8FA6cKZpAmB1dh
9iQJ+hS+nodQNZEvSqJ4lmsVhUnJmZbuNrZ9pvUFhYqtsI+GN6J4p0SqyGmpZZ6k
WGhT/JTLeueoADHcokcvBntf2bSst2kUFY87UUvvyTrlYl2+nSNPpW8VRx4lhFHP
I4CX9Oze
=/qfD
-----END PGP SIGNATURE-----
Merge tag 'exynos-drm-next-for-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/daeinki/drm-exynos into drm-next
Cleanup and modernization for Exynos encoders
- Remove dependency on deprecated DRM simple KMS helpers
. Replaces drm_simple_encoder_init() with drm_encoder_init() across dp,
dpi, dsi, vidi and hdmi.
. Adds per-driver drm_encoder_funcs with .destroy = drm_encoder_cleanup,
removing the intermediate layer between the drivers and atomic modesetting.
- Add error handling to drm_encoder_init()
. Checks the return code at every call site instead of ignoring it.
. Prevents silent encoder creation failures and drm_encoder_cleanup() on an
uninitialized encoder.
Bug fix for Exynos HDMI DDC adapter refcounting
- Take an i2c adapter module reference
. Switches to of_get_i2c_adapter_by_node() / i2c_put_adapter() so the
adapter module reference is held while the DDC bus is in use.
. Makes an unload attempt fail gracefully with -EBUSY instead of blocking
uninterruptibly in i2c_del_adapter().
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Inki Dae <inki.dae@samsung.com>
Link: https://patch.msgid.link/20260802061039.73544-1-inki.dae@samsung.com
Currently, we xe_ras_process_errors() inside xe_ras_init() to handle boot
time errors. But this can potentially result in declaring the device as
wedged quite early in the driver load sequence, which is problematic due to
the lack of registered drm device or required wedged cleanup hooks at this
point.
Call xe_ras_process_errors() only after the prerequisites are available.
Fixes: d9732e498f ("drm/xe/xe_ras: Query errors from system controller on probe")
Signed-off-by: Raag Jadav <raag.jadav@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Tested-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Link: https://patch.msgid.link/20260730110635.925537-1-raag.jadav@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
When guc_exec_queue_revert_pending_state_change() sets needs_cgp_sync for
a queue, replay that update during unpause by injecting a CGP_SYNC_MSG
static message at the head of the scheduler queue.
The message handler __guc_exec_queue_process_msg_cgp_sync() calls
guc_exec_queue_send_cgp_sync() which re-reads the current priority from
q->multi_queue.priority and re-issues the CGP write + CGP_SYNC H2G, so
the queue's CGP entry is restored to the correct value after migration.
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260803230528.3428029-10-niranjana.vishwanathapura@intel.com
When a CGP_SYNC (or REGISTER_CONTEXT_MULTI_QUEUE with CGP) is in flight
during VF migration, GuC loses the message and never sends CGP_SYNC_DONE.
Two failure modes exist:
1. The send was already issued (CGP_SYNC_DONE not received):
group->sync_pending stays set and cgp_update_q points to the queue
that owns the outstanding sync.
2. The wait woke early (send not issued):
The queue returned from xe_guc_exec_queue_group_cgp_sync() without
sending after vf_recovery() became true.
Track which kind of sync is outstanding (registering_cgp / updating_cgp)
and the owning queue (cgp_update_q) so
guc_exec_queue_revert_pending_state_change()
can recover both cases:
- A registration-time CGP bails or is lost → clear registered flag so
run_job re-registers after unpause (re_register / registering_cgp paths).
- A dynamic CGP update bails or is lost → set needs_cgp_sync so replay
re-issues the update after unpause (re_update / updating_cgp paths).
Tag all registration call sites with CGP_SYNC_REGISTRATION so the bail
path distinguishes them from dynamic updates.
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260803230528.3428029-9-niranjana.vishwanathapura@intel.com
The CGP_SYNC wait in xe_guc_exec_queue_group_cgp_sync() only woke on
sync_pending clearing or GuC stop. During VF migration the CT wakes all
waiters but sync_pending may still be set (the GuC lost the in-flight
message), causing the timeout path to fire and ban the group.
Add vf_recovery() as a third wake condition and exclude it from the
error path so a migration-woken wait returns success rather than
triggering a GT reset.
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260803230528.3428029-8-niranjana.vishwanathapura@intel.com
Extract xe_sched_submission_stop() + cancel_delayed_work_sync() into a
new guc_exec_queue_pause_prepare() helper, then call it in a first pass
over all queues before the existing revert loop in
xe_guc_submit_pause_vf().
In a multi-queue group a secondary's run_job() can register the primary
queue, which must not race an in-progress revert of a sibling queue.
Stopping every scheduler before reverting any queue prevents that race.
No functional change outside the VF migration path.
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260803230528.3428029-7-niranjana.vishwanathapura@intel.com
VFs were missing the call to apply the global scheduling policy.
Call xe_guc_submit_enable() during vf_uc_load_hw() to ensure VFs
get the same policy enforcement as PF.
Fixes: 26caeae9fb ("drm/xe/guc: Set RCS/CCS yield policy")
Suggested-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Link: https://patch.msgid.link/20260709075945.1337660-1-marcin.bernatowicz@linux.intel.com
Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
