Ensure GT C6 state before D3 transition in runtime suspend flow.
This avoids the scenario where GT is being accessed after D3 state
is programmed. If the check for C6 fails, resume the device so it is
not stuck in an intermediate state.
v2: Force resume when we cancel suspend due to C6 check (Sashiko)
v3: Abort the suspend properly when idle check fails (Sashiko)
v4: Only check for C6 during suspend (Rodrigo). Have check in
xe_pci_runtime_suspend() instead of xe_pci_suspend().
v5: Don't checkfor PVC (Sashiko), rename function from_device_c6
to _device_idle
v6: Rename to _wait_all_c6 and move PVC check to gt_idle (Rodrigo)
Cc: Badal Nilawar <badal.nilawar@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260916190614.3047807-1-vinay.belgaumkar@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Drop the definition of MI_SEMAPHORE_WAIT_TOKEN and replace its uses with
MI_SEMAPHORE_WAIT. Both #define's were representing the same opcode
0x1C and this GPU instruction has always been named "MI_SEMAPHORE_WAIT"
in the official specs.
The "*_TOKEN" definition appears to have been copied from i915's code.
i915 needed two separate definitions to reflect the slightly different
instruction lengths gen8-gen11 platforms (4 dwords) vs gen12 and later
platforms (5 dwords, with the extra dword representing a token), but Xe
only supports newer platforms with the 5-dword form, so only a single
definition is needed and should match the official name given in the
specs.
Bspec: 14054, 45749, 60244
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260917-semaphore_inst-v1-1-61e49405f71d@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
pci_p2pdma_distance() can optionally print details about how it
computed the ACS/PCIe topology distance between the two devices, but
xe_dma_buf_attach() always called it with verbose set to false,
silently hiding useful diagnostics whenever a dma-buf attach unset
attach->peer2peer.
Tie the verbose argument to a dedicated dynamic-debug callsite tagged
"xe dma-buf p2p debug" (in addition to keeping the existing
DRM_UT_DRIVER category as a default), so the extra p2pdma distance
logging can be switched on without a recompile in any of the following
ways:
- Enable the whole DRIVER category for xe via drm.debug, which is
already used elsewhere in the driver:
# echo 0x10 > /sys/module/drm/parameters/debug
- Enable dynamic debug for the whole file:
# echo 'file xe_dma_buf.c +p' > \
/sys/kernel/debug/dynamic_debug/control
- Enable dynamic debug for just this callsite by function name:
# echo 'func xe_dma_buf_attach +p' > \
/sys/kernel/debug/dynamic_debug/control
- Enable dynamic debug by matching the format string used to
register the callsite:
# echo 'format "xe dma-buf p2p debug" +p' > \
/sys/kernel/debug/dynamic_debug/control
- Enable it at boot via the dyndbg= kernel command line parameter:
dyndbg="func xe_dma_buf_attach +p"
Any of the above flips verbose_p2p_dist to true for that attach call,
causing pci_p2pdma_distance() to log its distance calculation without
requiring the whole DRM_UT_DRIVER category to be enabled.
v2:
- Guard the _dynamic_func_call() invocation and
xe_ddebug_flip_bool() with a CONFIG_DYNAMIC_DEBUG check so the
build does not break when dynamic debug is disabled. (Sashiko-bot)
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Assisted-by: LLM
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patch.msgid.link/20260914114617.112012-1-thomas.hellstrom@linux.intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
drm_pagemap_dev_unhold_work() processes an unbatched llist of pagemaps
on the system workqueue. This creates potential latency traps during
heavy teardown cycles due to two compounding factors:
1. The loop contains a per-item drm_dbg() trace. If dynamic debug is
enabled on a slow serial console, synchronous print latency scales
linearly with the batch size and can block worker execution.
2. The llist can accumulate large batches during intensive unmap
operations, monopolizing worker resources without yielding.
This causes below log warning:
"workqueue: drm_pagemap_dev_unhold_work [drm_gpusvm_helper] hogged CPU
for >10000us 19 times, consider switching to WQ_UNBOUND"
Fix this by implementing a latency mitigation strategy:
- Move the teardown work to system_unbound_wq to avoid tying up
per-CPU bound worker pools.
- Add cond_resched() inside the teardown loop to yield the CPU during
large batch teardowns, ensuring system responsiveness.
- Convert the drm_dbg() trace to drm_dbg_ratelimited() to mitigate
serial console bottlenecks while preserving debuggability.
v2:
- Changed workqueue from deprecated system_unbound_wq to system_dfl_wq.
(Arvind Yadav)
Fixes: a26084328a ("drm/pagemap, drm/xe: Manage drm_pagemap provider lifetimes")
Reviewed-by: Arvind Yadav <arvind.yadav@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Varun Gupta <varun.gupta@intel.com>
Link: https://patch.msgid.link/20260918081320.30389-2-varun.gupta@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
The desired behavior for SVM eviction failures, which can occur due to
various uncontrollable races, is for TTM to continue walking the LRU
list and look for another eviction candidate. This is expressed by
returning -ENOSPC from the ->move() callback.
Adjust the SVM eviction failure path because of races in ->move() to
return -ENOSPC so that TTM continues searching for another buffer to
evict.
Fixes: 3ca608dc75 ("drm/xe: Basic SVM BO eviction")
Cc: stable@vger.kernel.org
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Link: https://patch.msgid.link/20260917203158.292823-1-matthew.brost@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
xe_pagefault_print() was firing without indicating why servicing of a
fault failed, making it hard to distinguish benign races (e.g. a VM's
file descriptor closing mid-fault) from real bugs.
Add enum xe_pagefault_error, uniquely identifying the high-level point
at which xe_pagefault_service(), xe_pagefault_handle_vma(), and
xe_svm_handle_pagefault() fail. Since consumer.page_addr is always
page (4K) aligned, steal the reserved low byte to carry this code
without growing struct xe_pagefault. Add xe_pagefault_set_error() /
xe_pagefault_get_error() to encode/decode it, and xe_pagefault_addr()
as the single accessor for the real (masked) faulted address; update
all real address consumers (xe_pagefault_match(), cache alignment,
xe_vm_add_fault_entry_pf()) to go through it instead of reading
consumer.page_addr directly. xe_pagefault_set_start_addr() now
preserves any already-recorded error bits rather than clobbering them.
xe_pagefault_print() decodes the error via a new
xe_pagefault_error_to_str() and reports it in an added "Error:" line.
xe_pagefault_asid_to_vm() now distinguishes an ASID with no VM at all
(XE_PAGEFAULT_ERROR_VM_NOT_FOUND, e.g. the owning file descriptor was
already closed) from a VM found but not in fault mode
(XE_PAGEFAULT_ERROR_VM_NOT_IN_FAULT_MODE).
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Assisted-by: GitHub_Copilot:claude-sonnet-5
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Link: https://patch.msgid.link/20260917203850.321385-1-matthew.brost@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
A TLB invalidation issued on a wedged device fails with
-ENOTRECOVERABLE. xe_tlb_inval_issue() squashes only -ECANCELED, so the
error reaches callers that treat it as unexpected and WARN, tainting the
kernel on a wedge that was deliberately caused:
ggtt_invalidate_gt_tlb() drivers/gpu/drm/xe/xe_ggtt.c
xe_svm_invalidate() drivers/gpu/drm/xe/xe_svm.c
xe_bo_trigger_rebind() drivers/gpu/drm/xe/xe_bo.c
xe_vma_userptr_do_inval() drivers/gpu/drm/xe/xe_userptr.c
igt@xe_exec_reset@gt-reset-fault-injection hits the GGTT one, turning an
otherwise passing run into an abort:
*ERROR* SIGID=102 FATAL (-EIO) WEDGED: Device declared wedged!
Tile0: GT1: Failed to invalidate GGTT (-ENOTRECOVERABLE)
WARNING: drivers/gpu/drm/xe/xe_ggtt.c:588 at ggtt_invalidate_gt_tlb
Workqueue: xe-guc-destroy-wq __guc_exec_queue_destroy_async [xe]
ggtt_node_remove+0xe3/0x100 [xe]
xe_ggtt_remove_bo+0x89/0x2c0 [xe]
xe_ttm_bo_destroy+0xcb/0x330 [xe]
...
xe_lrc_destroy+0x74/0x90 [xe]
xe_exec_queue_fini+0x2d/0x60 [xe]
-ECANCELED and -ENOTRECOVERABLE mean the same thing at this layer: the
message was dropped rather than delivered, and the fence has already
been signalled before the error is returned, so there is nothing left to
wait for. Squash both.
A wedged device is only recovered by a fresh initialisation, so the
error return in xe_bo_trigger_rebind() becomes unreachable by design.
v2: fix all invalidation paths. (Sashiko)
Fixes: 50fa9acac2 ("drm/xe/guc: distinguish wedged from recoverable cancellation")
Assisted-by: Claude:claude-opus-5
Cc: Sk Anirban <sk.anirban@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260914215318.200603-1-shuicheng.lin@intel.com
CRI defines bit 5 of the perf limit reasons register as a power brake
(PWRBRK) indicator. Add PWRBRK_MASK and a reason_pwrbrk sysfs attribute
for CRI in place of reason_ratl.
Signed-off-by: Sk Anirban <sk.anirban@intel.com>
Fixes: 8578e6d054 ("drm/xe/gt_throttle: Drop individual show functions")
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260909114931.1039331-2-sk.anirban@intel.com
Extend the existing "hw_engines" debugfs to iterate over all hardware
engines and dump their idle state: whether the engine pipe is idle or
busy, whether C6 entry is allowed, and (where available) the forcewake
request status.
v2: Use existing debugfs (Matt Brost)
v3: Rename macro (Vinay)
Signed-off-by: Sk Anirban <sk.anirban@intel.com>
Reviewed-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260910165855.1124574-2-sk.anirban@intel.com
Start using recently added KLV helpers for parsing migration
packet to properly detect any malformed KLV formats.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Link: https://patch.msgid.link/20260916215827.13488-4-michal.wajdeczko@intel.com
Instead of retrieving data from the KLV dwords manually, add
simple helpers that will do that together with sanity checks.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Link: https://patch.msgid.link/20260916215827.13488-2-michal.wajdeczko@intel.com
Instead of open coding it via the inefficient amdgpu_ring_write(), which
which the compiler is not able to optimise much, we can add a new
amdgpu_ring_fill() helper which pads using memset32. We do this by moving
the existing implementation from amdgpu_ring_insert_nop(), which is used
by the GFX rings, and then individually convert all the SDMA insert_nop
implementations.
For SDMA this should have a lesser benefit than for GFX (only SDMA v4.0
uses the 256 byte ring padding while the rest use 16), but it does not
harm to be consistent.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A subset of supported devices can have a mismatch between the logical and
device engine instance numbers which the driver currently solves by a
combination of lookup table generated at driver init time, and a
conditionally set vfunc checked and used at runtime to do the lookup.
As the requirement to do these lookups is relatively widespread in the
code base, plus the fact the mapping table is already allocated for all
possible hw ips and instances, we can significantly streamline them by
initializing the said lookup table unconditionally to the predominant
"identity map" (logical instance numbers equal to device instances),
and allow affected parts to re-configure the table based on their
configuration.
The effect is that for each lookup site we replace the conditional
followed by a indirect function call with a simple two-dimensional table
lookup.
Bloat-o-meter agrees the savings are significant:
add/remove: 13/12 grow/shrink: 23/279 up/down: 51626/-198973 (-147347)
...
Total: Before=10139692, After=9992345, chg -1.45%
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
SDMA CSA address is static while the driver is loaded so lets store it in
the SDMA instance allowing us to simplify and inline the
amdgpu_sdma_get_csa_mc_addr() helper.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_ras_late_init() sends the MCA configuration to PMFW, but it runs
before RAS is ready on a device that still needs a hive reset, so the
request is dropped and CE polling stays disabled. A reset loses the
setting too, since PMFW is reloaded and nothing programs it again.
The request is a single SMU message and does not depend on the RAS block
being up, so amdgpu_ras_mgr_set_debug_mode() no longer requires it. Send
it from amdgpu_ras_resume(), which covers modprobe, resume from suspend,
GPU reset and reset on init, and drop it from amdgpu_ras_late_init().
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
dcn30_apply_idle_power_optimizations() derives the MALL frame cache
hysteresis timer with
tmr_delay = (uint32_t)(div_u64(..., denom) - 64LL);
div_u64() returns a u64, so when the quotient is smaller than 64 the
subtraction wraps instead of going negative and tmr_delay ends up huge.
The loop that follows tries to squeeze it into the 6 bit register field
by doubling denom, but that only makes the quotient smaller, so tmr_delay
can never converge. tmr_scale is bumped past 3 and the function gives up
with
/* Delay exceeds range of hysteresis timer */
ASSERT(false);
even though the requested delay is too *short* to encode, not too long.
With mall_additional_timer_percent left at its default of 0, the quotient
drops below 64 once the refresh rate used for the calculation goes above
~243 Hz. Every DCN 3.0 display above that loses MALL static screen
entirely and splats a WARN once per boot. Reproduced on Navi 23
(RX 6600) driving 1920x1080, resetting /sys/kernel/debug/clear_warn_once
between modes:
refresh MALL ASSERT
144 Hz enabled no
240 Hz enabled no
280 Hz skipped yes
360 Hz skipped yes
Commit 3bb68cec4d ("drm/amd/display: Add Overflow check to skip MALL")
already covered the other end of the range, where a large stutter period
makes the delay too long to encode. Cover the short end by clamping to
0, which selects the shortest hysteresis the register can express,
65.28us * 64 = ~4.18ms. That is marginally longer than what the formula
asks for at these refresh rates, and erring long is the safe direction:
it only delays MALL entry, it can never enter early.
The numerator does not change between iterations, only denom does, so
compute it once and keep both call sites inside 100 columns.
The genuinely out of range case at very low refresh rates still reaches
the ASSERT, which is where it belongs.
Fixes: 52f2e83e2f ("drm/amdgpu/display: add MALL support (v2)")
Signed-off-by: Francis Marlou Pacaro <pacaro.francis.marlou.n@gmail.com>
Reviewed-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This tab was accidentally added in commit 6c9e0328d1 ("drm/amd/pm:
refactor PPT limits by controller and power source"). Delete it.
Signed-off-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_amdkfd_check_and_lock_kfd() and amdgpu_amdkfd_unlock_kfd() no
longer have any callers since the compute partition switch was reworked.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_pci_remove() calls drm_dev_unplug() before fini_sw(), so
drm_dev_enter() is already false there and the iounmap() guarded by it
is skipped. This .remove path runs on both hot-unplug and plain rmmod,
so the register BAR ioremap mapping leaks one instance per unload.
Unmap rmmio unconditionally (guard only on non-NULL) and drop the now
unused idx.
Fixes: 62d5f9f711 ("drm/amdgpu: Unmap MMIO mappings when device is not unplugged")
Signed-off-by: Chengjun Yao <Chengjun.Yao@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The compute partition switch locked the kfd device, freed it during the
pre-switch teardown, and then unlocked at the end - by which point
adev->kfd.dev referenced the newly reinitialized device, so the unlock
landed on the wrong object and the per-device lock was never balanced.
Add a dedicated prepare function to unwind and clean up kfd device.
Don't use the global lock during teardown. A new device is created after
partition switch; hence there is no need to do 'unlock' on a different
object after the switch.
User is not supposed to run any process on the device during partition
switch. Keep acquire/release semantics for init_complete so that the
device remains unavailable for a process till its reinitialization is
complete.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
After "drm/amdgpu: Fix GPU PCIe link capability reporting" there are
now a few areas of walking topology.
Use the cached GPU-side link device for ASPM queries and switch state
caching instead of walking the PCI topology independently in each path.
Validate the internal downstream/upstream switch pair when initializing
the cached link devices so all consumers share the existing topology
checks.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_amdkfd_clear_kfd_mapping() assumes that a non-NULL kfd_dev
has a fully populated node array. This is not true when KFD device
initialization fails after probe.
For example, kgd2kfd_device_init() sets num_nodes before checking
PCIe atomics support. On Polaris systems without the required atomics,
it returns before allocating nodes[0], but the kfd_dev remains attached
to the amdgpu device. A later GPU reset then dereferences nodes[0]->id.
Require the authoritative KFD initialization flag before walking the
node array, matching the existing KFD reset and teardown paths.
Fixes: 70cadefcc6 ("drm/amdgpu: unmap all user mappings of framebuffer and doorbell before mode1 reset")
Closes: https://gitlab.freedesktop.org/drm/amd/-/work_items/5833
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
dc->clk_mgr is already dereferenced inside dcn401_prepare_bandwidth(),
so the dc->clk_mgr NULL check that follows is redundant and misleading.
Drop it.
Fixes: 53845307d5 ("drm/amd/display: Register DCN as a PMFW DF C-state client on DCN42")
Reported-by: Dan Carpenter <error27@gmail.com>
Cc: Roman Li <roman.li@amd.com>
Cc: Alex Hung <alex.hung@amd.com>
Cc: Tom Chung <chiahsuan.chung@amd.com>
Cc: Ivan Lipski <ivan.lipski@amd.com>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Aurabindo Pillai <aurabindo.pillai@amd.com>
Cc: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Cc: Wayne Lin <wayne.lin@amd.com>
Cc: Dan Wheeler <daniel.wheeler@amd.com>
Cc: Leo Chen <leo.chen@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fix a hardware RTL bug in the TCP XNACK scoreboard reset path where a wave
receiving a no-retry FAULT from UTC and subsequently terminating may leave
its scoreboard entry uncleared when reset events arrive back-to-back.
This can cause a later, unrelated wave using the same slot to incorrectly
inherit the XNACK error state, resulting in its VMEM operations being
dropped even though the wave never faulted.
Signed-off-by: Gang Ba <Gang.Ba@amd.com>
Reviewed-by: Jay Cornwall <jay.cornwall@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
atom_get_vbios_date(), atom_get_vbios_pn(), atom_get_vbios_version()
and atom_get_vbios_build() all walk the BIOS image using offsets and
counts taken from the image itself, checked only against each other
and never against ctx->bios_size:
- atom_get_vbios_date() reads 14 bytes at a fixed offset that
check_atom_bios()'s minimum size (0x49) does not cover.
- atom_get_vbios_pn() dereferences an image-supplied u16 string
offset unchecked, then walks and copies from it with only a length
cap, not an image cap.
- atom_get_vbios_version() takes its search bounds from image u16s;
atom_find_str_in_rom() already clamps the search itself, but the
match is then advanced by a fixed 18 bytes and STRLEN_NORMAL bytes
are copied from there with no check that this still lands inside
the image.
- atom_get_vbios_build() walks from an image-supplied config-string
offset toward the ROM header; a config offset at or past the
header makes the pointer difference negative, wrapping through the
u16 'len' and passing strscpy() a bound far larger than the
32-byte destination.
Bound each getter's reads, walks and copies to ctx->bios_size, falling
back to an empty string on a malformed image.
Fixes: 29b4c589b4 ("drm/amdgpu: Add vbios info ioctl interface")
Fixes: d6fa802661 ("drm/amdgpu: Add vbios build number interface")
Assisted-by: gkh_clanker_t1000
Signed-off-by: Hari Mishal <harimishal1@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
dc->clk_mgr is checked for NULL earlier in dcn50_init_hw() and
dcn60_init_hw(), but dcn50_initialize_min_clocks() and
dcn401_initialize_min_clocks() are called without any guard,
causing Smatch to report potential NULL dereferences.
Guard both call sites with the same pattern used throughout
both functions:
if (dc->clk_mgr && dc->clk_mgr->funcs)
Also fix dcn50_initialize_min_clocks() which calls
get_dispclk_from_dentist without checking the function pointer,
unlike the dcn401 equivalent which guards that call.
Fix kernel-doc in dcn60_hwseq.c by adding missing parameter descriptions
for @probe in dcn60_update_probe_status() and @type in
is_probe_measurement_type_for_hubbub().
Fixes: 7f7d7ea1fa ("drm/amd/display: Add new sources for DCN6")
Reported-by: Dan Carpenter <error27@gmail.com>
Cc: Aurabindo Pillai <aurabindo.pillai@amd.com>
Cc: Ivan Lipski <ivan.lipski@amd.com>
Cc: Dan Wheeler <daniel.wheeler@amd.com>
Cc: Roman Li <roman.li@amd.com>
Cc: Alex Hung <alex.hung@amd.com>
Cc: Tom Chung <chiahsuan.chung@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The low 6 bits of cp_hqd_eop_control store the base-2 logarithm
of the EOP ring size. This was calculated as
order_base_2(q->eop_ring_buffer_size / 4) - 1
But order_base_2 can in theory return 0, so this could underflow
(although in practice the ring buffer size cannot be less than 4096).
Change this to
order_base_2(q->eop_ring_buffer_size / 8)
using properties of logarithms.
Also add to the above comment to make the mathematics more clear.
Reviewed-by: Kent Russell <kent.russell@amd.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The low 6 bits of cp_hqd_eop_control store the base-2 logarithm
of the EOP ring size. This was calculated as
ffs(q->eop_ring_buffer_size / sizeof(unsigned int)) - 1 - 1
But ffs can in theory return 1 or 0, so this could underflow
(although in practice the ring buffer size cannot be less than 4096).
Change this to
ffs(q->eop_ring_buffer_size / sizeof(unsigned int) / 4)
using properties of logarithms.
Reviewed-by: Kent Russell <kent.russell@amd.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Commit eb53125a7a ("drm/amd: Add dedicated helper for
amdgpu_device_find_parent()") made amdgpu_device_gpu_bandwidth() query
the first device outside the dGPU. That is the host side of the
physical link, not the GPU side.
As a result, the ASIC and platform capability masks can both be based
on the host port. drm_amdgpu_info_device then exposes the host
capabilities to userspace, such as Gen5 x16 for a Gen4 x8 GPU.
Cache both ends of the physical link during device initialization.
Use link_dev for the GPU capability and link_partner for the platform
capability and _PR3 detection.
Reported-by: "Marek Olšák" <maraeo@gmail.com>
Closes: https://lore.kernel.org/amd-gfx/CAAxE2A4VhsAzzO1QjBjUg+NgnbD04ZzMyN6xsUJxjKJHH6hxiw@mail.gmail.com/
Suggested-by: Lijo Lazar <lijo.lazar@amd.com>
Fixes: eb53125a7a ("drm/amd: Add dedicated helper for amdgpu_device_find_parent()")
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead of searching for the matching SDMA instance inside many of the
SDMA ring vfuncs lets just use the instance number already stored in the
ring and replace the function calls and array walks with direct lookups.
As these helpers are used only from SDMA vfuncs we can remove safely
remove the "is this a SDMA ring" checks as well.
We could add an assert that ring->type queried is AMDGPU_RING_TYPE_SDMA
but as currently there are no debug build only assert macros in this
driver for now I opted not to so the helper can stay a trivial static
inline without polluting the callers with out of line logging.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
There is an include loop between amdgpu.h and amdgpu_sdma.h which
prevents adding any interesting static inline helpers to the latter.
Break it by moving the type definitions to a new amdgpu_sdma_types.h
header.
This is required so static inline helpers which need to dereference
struct amdgpu_device can be added to amdgpu_sdma.h.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
atomctrl_get_smc_sclk_range_table() copies ucSclkEntryNum entries from
the VBIOS SMU_Info table into pp_atom_ctrl_sclk_range_table. Both the
source (asSclkFcwRangeEntry[]) and destination (table->entry[]) arrays
are fixed at MAX_SCLK_RANGE (8) entries, but ucSclkEntryNum is an
unchecked u8 read straight from the VBIOS, so it can be up to 255. The
destination is stack-allocated in both callers
(polaris10_get_sclk_range_table(), vegam_get_sclk_range_table()), so a
count above 8 overflows a stack buffer, not just a heap read.
Reject counts above MAX_SCLK_RANGE rather than clamping to it. Only
one version of this VBIOS struct (ATOM_SMU_INFO_V2_1) has ever
existed, so a count above 8 corresponds to no legitimate table format
and means either a corrupted or a tampered VBIOS.
Fixes: a23eefa2f4 ("drm/amd/powerplay: enable dpm for baffin.")
Assisted-by: gkh_clanker_t1000
Signed-off-by: Hari Mishal <harimishal1@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On GFX 12.1, ring buffers may be mapped with a non-coherently
cached memory type. In that case the queue fetch can observe stale
packets, since the packet producers (CPU or other GPUs) do not keep the
device-side cache coherent.
Set CP_HQD_PQ_CONTROL.SCOPE=3 (SYSTEM) so the queue fetches use
system scope and pick up the latest packets. Apply this to compute queue
MQDs and to MES-managed queues.
Without this, the queue can consume stale packets for long periods,
causing severe slowdowns and, in the worst case, application hangs.
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Joseph Greathouse <joseph.greathouse@amd.com>
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
scripts/kernel-doc reports missing and excess member descriptions
for struct amdgpu_display_manager:
struct member 'power_module' not described
struct member 'hdmi_frl_status_polling_wq' not described
struct member 'hdmi_frl_status_polling_delay_ms' not described
struct member 'boot_time_crc_info' not described
Excess struct member 'dm_boot_time_crc_info' description
Add @power_module to the main struct doc, document
hdmi_frl_status_polling_wq/work/delay_ms with separate inline
comments, and rename @dm_boot_time_crc_info to @boot_time_crc_info
to match the actual member name.
Signed-off-by: Omokefe Emmanuel Onanaroghene <emmaonana18@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fix the spelling in comments. No functional change.
Signed-off-by: Ruslan Vagner <rusya92266@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
nbio_v7_9_handle_ras_controller_intr_no_bifring() dereferences ras and obj
without checking either for NULL. Both amdgpu_ras_get_context() and
amdgpu_ras_find_obj() can return NULL, e.g. during the window between
adev->nbio.ras being set (early in amdgpu_ras_init(), by design, to
enable the fatal-error interrupt as soon as possible) and the PCIE_BIF
ras object actually being created in RAS late_init. Any interrupt in that
window crashes in hard-IRQ context.
This is analogous to commit d190b459b2 ("drm/amdgpu: the warning
dereferencing obj for nbio_v7_4"), which fixed the same issue in the
nbio_v7_4 handler.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Fixes: 7692e1ee24 ("drm/amdgpu: add RAS fatal error handler for NBIO v7.9")
Reviewed-by: Tao Zhou <tao.zhou1@amd.com>
Signed-off-by: Dmitriy Chumachenko <Dmitry.Chumachenko@cyberprotect.ru>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Now, sdma_v4_4_2_reset_queue() does basically the same
as amdgpu_sdma_reset_queue_legacy() so let's use that.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Consolidate one part of the SDMA ring initialization with a new
amdgpu_sdma_ring_init() helper.
Only code which is completely IP version agnostic is moved to the helper.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Root cause:
- In userq restore/rearm, root PD was locked with amdgpu_vm_lock_pd(vm, &exec, 1).
- Later in the same drm_exec context, amdgpu_evf_mgr_rearm() attaches
BOOKKEEP eviction fences to all locked BOs.
- In this flow, root PD validation/update can also consume move-fence slots,
so reserving only +1 at PD lock time is not sufficient under stress.
Fix:
- Reserve caller-side slots at the actual source by switching to:
amdgpu_vm_lock_pd(vm, &exec, TTM_NUM_MOVE_FENCES + 1)
This keeps reservation policy in callers, consistent with Christian Konig
feedback and with amdgpu_gem_object_open() where callers reserve
TTM_NUM_MOVE_FENCES + 1 before attaching eviction fences.
Credit:
- Christian Konig: confirmed fence-slot reservation belongs to callers and
asked to investigate why +1 was still not enough.
- Prike Liang: clarified the initial variant was workaround-only and
developed that initial version independently.
Cc: Prike Liang <prike.liang@amd.com>
Cc: Sunil Khatri <sunil.khatri@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Root cause analysis showed that the previous workaround was fixing the
symptom in the wrong layer.
Problem:
kernel BUG at drivers/dma-buf/dma-resv.c:319!
BUG_ON(fobj->num_fences >= fobj->max_fences)
Workqueue: events amdgpu_userq_restore_worker
- amdgpu_evf_mgr_attach_fence() had been temporarily taught to reserve/add
fence bookkeeping internally as a workaround.
- This hid the real contract violation and made reservation policy implicit
inside a helper.
Review clarification:
- The helper-side dma_resv_add_fence-related workaround was only an intended
proposal sent for review discussion, not the final direction.
- Final fix keeps fence-slot reservation explicit in callers and leaves the
helper free of policy decisions.
Findings from investigation:
- drm_exec_prepare_obj() both locks and reserves fence slots.
- drm_exec_lock_obj() only locks and does not reserve slots.
- Existing attach_fence callers in the GEM/userq restore flows already reserve
TTM_NUM_MOVE_FENCES + 1 before adding the bookkeeping eviction fence.
- The WPTR mapping path in mes_userq_create_wptr_mapping() used
drm_exec_lock_obj() and then called amdgpu_evf_mgr_attach_fence(), so this
specific path could add a fence without pre-reserved space.
Fix:
- Switch the WPTR mapping path to drm_exec_prepare_obj(...,
TTM_NUM_MOVE_FENCES + 1), restoring the caller-side reservation contract.
- Remove the helper-side reservation workaround from
amdgpu_evf_mgr_attach_fence().
Why this is the right layer:
- Reservation policy stays explicit at call sites that define how many fences
may be added in that execution context.
- The helper only performs validate + add_fence when called with a properly
prepared reservation object, matching the existing drm_exec model.
Fixes: 7264bc10c7 (drm/amdgpu: dont pin wptr bo instead use eviction fence)
Cc: Sunil Khatri <sunil.khatri@amd.com>
Cc: Prike Liang <prike.liang@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Reviewed-by: Prike Liang <prike.liang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The Apple iMac10,1 and iMac11,2 (RV730/DCE3.2) drive their internal eDP
panel through an LVTMA power sequencer whose enable bit (PWRSEQ_EN in
LVTMA_PWRSEQ_CNTL) is programmed exactly once, by EFI at cold boot.
Apple's minimal ATOM tables never touch it afterwards, so any GPU
re-initialisation - S3 resume or a GPU-lockup reset - clears PWRSEQ_EN
and the panel comes back dark while the rest of the GPU resumes fine.
The driver never programs this register itself; on every other radeon
board the platform firmware maintains it across a re-POST, which is why
this only bites the Apple machines. Re-assert PWRSEQ_EN in the eDP
DPMS-on path when it is found cleared; the hardware then runs the
DIGON->DE->VARY_BL->BLON power-on sequence on its own. Both failure
paths pass through here, as radeon_gpu_reset() restores the modeset
state via radeon_resume_kms(), so a single site covers resume and reset.
The write is gated to Apple parts and to exactly DCE3.2 (ASIC_IS_DCE32
is >= RV730, so DCE4+ must be excluded - Apple also shipped DCE5, e.g.
the 2011 MacBook Pro, where the register lives at a different offset).
It is skipped when the sequencer is already enabled, so lit panels,
DCE4+ boards and non-Apple boards are untouched. DCE3.2 places the
register 0x18 above the documented R600 offset.
Only iMac11,2 is verified on hardware; iMac10,1 shares the same RV730
design and Apple EFI behaviour.
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/5719
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patch.msgid.link/20260914022002.2005150-1-mario.limonciello@amd.com
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When a HELLO arrived on an already ESTABLISHED connection,
amdgpu_ualink_process_hello_msg() set the state to PENDING before
calling amdgpu_ualink_handle_connection_reset(). That function only
runs the imp/exp XA cleanup when it observes the state as ESTABLISHED,
so the premature change skipped the cleanup and left stale NPA
mappings and pinned BOs from the previous connection.
Leave the state as ESTABLISHED and let
amdgpu_ualink_handle_connection_reset() perform the transition, so the
cleanup runs.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by : Philip Yang <philip.yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_dma_buf_map() adds VRAM to the allowed domains for a peer2peer
attachment. GTT is only a fallback placement when VRAM is preferred, so
ttm_bo_validate() migrates the buffer from GTT into VRAM. While the
exporting device is runtime suspended its SDMA rings are down and the
move fails:
amdgpu: Move buffer fallback to memcpy unavailable
An importer on a second GPU reaches this holding no runtime PM
reference on the exporter, e.g. a compositor on the APU submitting a
frame that references a buffer exported by an idle dGPU:
amdgpu_cs_ioctl -> amdgpu_cs_parser_bos -> amdgpu_cs_bo_validate
-> ttm_bo_validate -> amdgpu_bo_move -> dma_buf_map_attachment
-> amdgpu_dma_buf_map -> ttm_bo_validate -> amdgpu_bo_move
Pinning a dma-buf into VRAM has the same requirement, which
commit 030631e97b ("drm/amdgpu: revert "take runtime pm reference
when we attach a buffer" v2") called out as the one case that would
need the reference back.
Take it in attach and drop it in detach. pm_runtime_get_if_active()
never resumes the device, so it cannot deadlock against the reservation
taken during resume, which is why the old pm_runtime_get_sync() had to
go. If the device is not active, clear peer2peer instead: the buffer
then stays in GTT, which remains accessible while the GPU is powered
down. If runtime PM is disabled, take a plain reference so the put in
detach stays balanced.
Fixes: 030631e97b ("drm/amdgpu: revert "take runtime pm reference when we attach a buffer" v2")
Suggested-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Mike Lothian <mike@fireburn.co.uk>
Assisted-by: Claude:Opus-5 [Claude Code]
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Point to the new repo on gitlab.
Cc: christian.koenig@amd.com
Cc: felix.kuehling@amd.com
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Remove the caller_handles_kernel_queues argument from
the amdgpu_sdma_reset_engine() function and make it
always handle kernel queues.
Now the SDMA recovery sequence is more consistent
between callers for the KFD as follows.
Before recovery: first the KFD is suspended,
then the SDMA queue contents are backed up.
After recovery: first the SDMA queue contents
are restored, then the KFD is resumed.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The code was exactly the same between SDMA v5.0 and v5.2
furthermore the exact same implementation can be shared
between all SDMA versions that don't use MES.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
