gfx_v12_1_get_xccs_per_xcp() supplies the compute partition layout that
gfx_v12_1_cp_resume() needs on a VF. Once VFI is enabled the VF reaches
registers through RLCG, which cannot access the GFX_IMU registers, so the
GFX_IMU_PARTITION_SWITCH read stops working. Take the same field from
RLC_IMU_AID_CONFIG instead, and read it with a direct MMIO access rather
than RREG32_SOC15, which would route back into the RLCG path.
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reading or writing a register through the amdgpu_regs2 debugfs file
with GRBM bank selection, which umr does for every banked access (a
single "umr -b x x x -r <reg>" is enough, as is the wave dump with
"-O halt_waves"), makes lockdep report a circular locking dependency,
after which lockdep stays off until reboot:
WARNING: possible circular locking dependency detected
umr/63306 is trying to acquire lock:
ffff8882cf4b8350 (&mm->mmap_lock){++++}-{4:4}, at: __might_fault+0x9b/0x140
but task is already holding lock:
ffff8881aec010d0 (&amdgpu_grbm_lock_key){+.+.}-{4:4}, at: amdgpu_debugfs_regs2_op.isra.0+0x20d/0xd00 [amdgpu]
...
Chain exists of:
&mm->mmap_lock --> &amdgpu_srbm_lock_key --> &amdgpu_grbm_lock_key
...
__might_fault+0xc5/0x140
amdgpu_debugfs_regs2_op.isra.0+0x441/0xd00 [amdgpu]
full_proxy_write+0x10e/0x1d0
vfs_write+0x1d0/0xf50
ksys_write+0x116/0x250
amdgpu_debugfs_regs2_op() and amdgpu_debugfs_process_reg_op() call
get_user()/put_user() for every dword while holding grbm_idx_mutex or
srbm_mutex, and pm.mutex if requested, so a fault there takes mmap_lock
inside those locks. The lock hierarchy in amdgpu_lockdep.c puts
srbm_mutex and grbm_idx_mutex below notifier_lock, which
amdgpu_cs_submit() takes with the BO reservations held, and
dma_resv_lockdep() orders the reservations inside mmap_lock. So
mmap_lock is outside grbm_idx_mutex and srbm_mutex, and these two
debugfs paths nest it the other way round. The copies under the locks
are as old as the bank selection in both files; commit 1d0f5838b1
("drm/amdgpu: Add lockdep annotations for lock ordering validation")
only made them visible to lockdep.
Copy the user buffer in before any of the locks is taken and the result
out after all of them are dropped, as amdgpu_debugfs_wave_read(),
amdgpu_debugfs_gpr_read() and amdgpu_debugfs_gprwave_read() already do.
The buffer is sized by the count passed to read() or write(), so an
access larger than PAGE_SIZE is now rejected with -EINVAL; umr reads and
writes one register at a time. A fault on the user buffer of a write now
fails with -EFAULT before any register is written, instead of after a
partial write.
Fixes: 5662815950 ("drm/amd/amdgpu: Add bank selection for MMIO debugfs (v3)")
Fixes: 37df9560cd ("drm/amd/amdgpu: New debugfs interface for MMIO registers (v5)")
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Signed-off-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The vblank counter callback reads display registers and scanout
position without holding reset_domain->sem. During GPU recovery,
display commit work can call the callback while reset owns the
semaphore for writing. The hardware access then triggers the
lockdep_assert_held() in amdgpu_device_skip_hw_access() and sets
TAINT_WARN when lockdep is enabled.
Try to acquire the reset semaphore for reading before accessing the
hardware, and hold it across the counter and scanout-position reads.
If reset is already in progress, return DRM's cached software vblank
count instead. This avoids blocking behind reset and avoids accessing
display hardware while it is being reset.
The cached count is less accurate than the hardware count, but is
used only when reset prevents a safe hardware read. This addresses
the reset-time warning from this vblank path; it does not suppress or
fix unrelated lockdep warnings.
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Sunpeng Li <sunpeng.li@amd.com>
Signed-off-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Reviewed-by: Sunpeng Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On the A+A VF the GPU reaches its HBM over the system memory path at a
system physical address, so page table entries covering VRAM have to
carry AMDGPU_PTE_SYSTEM and the addresses they hold have to be physical
rather than MC addresses.
Set the SYSTEM bit in the PDB0 entries and in amdgpu_ttm_tt_pde_flags(),
and convert the VRAM address in amdgpu_gmc_get_pde_for_bo() with
amdgpu_gmc_vram_mc2pa().
NPA mappings still require VSCT=0011 (PTE.S=0). Clear SYSTEM in
gmc_v12_1_get_npa_flags() and amdgpu_ualink_peer_remote_init() so NPA
PTEs do not inherit the VRAM SYSTEM bit.
v2:
- Use (sriov_vf || passthrough) and amdgpu_virt_vram_is_spm() for
PDB0/TTM SYSTEM bits and mc2pa.
v3:
- Rely on amdgpu_virt_vram_is_spm() without redundant VF/passthrough
checks.
v4:
- Explicitly clear AMDGPU_PTE_SYSTEM for NPA (VSCT=0011).
Signed-off-by: Zhigang Luo <zhigang.luo@amd.com>
Signed-off-by: Fangsheng Huang <Fangsheng.Huang@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Philip Yang <philip.yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The A+A VF FB BAR is not usable, so resolve its aperture from SRAT in
GMC early_init. Skip later aperture assignment for SPM-backed devices,
and restrict the existing mmhub-based override to native A+A and APU.
The GPU reaches its HBM at a system physical address, so
vram_base_offset becomes aper_base as well, which makes PDB0 map GPU
address 0 onto that address.
The XGMI node segment size comes from the same place: once early_init
has resolved the aperture, use its size rather than the value derived
from the register, which does not describe the VF partition.
v2:
- Use (sriov_vf || passthrough) and amdgpu_virt_vram_is_spm()
instead of connected_to_cpu for the SRAT aperture and node segment
size.
v3:
- Resolve the SRAT aperture in GMC early_init and skip later aperture
assignment in mc_init for SPM-backed devices.
- Rely on amdgpu_virt_vram_is_spm() without redundant VF/passthrough
checks.
Signed-off-by: Zhigang Luo <zhigang.luo@amd.com>
Signed-off-by: Fangsheng Huang <Fangsheng.Huang@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On A+A the VF has no usable FB BAR, so the VBIOS copy cannot be read
through BAR 0. It lives in HBM, which the SRAT describes as
specific-purpose memory reachable at a system physical address.
Add a dedicated amdgpu_read_bios_from_spm() path that resolves the HBM
range from SRAT and maps it as write-back system memory. Keep the
existing SR-IOV dynamic critical region path when it is enabled,
matching the VRAM reader.
v2:
- Use a separate amdgpu_read_bios_from_spm() helper and leave
amdgpu_read_bios_from_vram() unchanged.
- Keep the SR-IOV dynamic critical region path for SPM-backed VBIOS.
- Fetch VBIOS after the COMMON block and make it optional for
SPM-backed SR-IOV VF and passthrough configurations.
v3:
- Keep the SRAT range local to the VBIOS path; GMC owns aper_base and
aper_size.
- Map SPM with memremap(), memcpy(), and memunmap().
- Rely on amdgpu_virt_vram_is_spm() without redundant VF/passthrough
checks.
Signed-off-by: Zhigang Luo <zhigang.luo@amd.com>
Signed-off-by: Fangsheng Huang <Fangsheng.Huang@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On A+A, HBM is reached at a host physical address, so VRAM mappings
need the system-memory path. That is true for both an SR-IOV VF and a
passthrough VM; native PF A+A keeps the mmhub/sysvm path.
Add AMDGPU_VIRT_VRAM_IS_SPM and amdgpu_virt_vram_is_spm() so later
patches can identify SPM-backed VRAM without repeating the cap test.
v2:
- Rename the helper to amdgpu_virt_vram_is_spm(); it only tests
AMDGPU_VIRT_VRAM_IS_SPM.
- Set the cap in ip_early_init when an SR-IOV VF or passthrough dGPU
has no FB BAR0, after AMD_IS_APU is known.
Signed-off-by: Zhigang Luo <zhigang.luo@amd.com>
Signed-off-by: Fangsheng Huang <Fangsheng.Huang@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
ip_discovery_size_in_kb is in KB but was compared against
DISCOVERY_TMR_SIZE, a byte count, so the check never fired.
Compare both sides in KB against a new 16KB SR-IOV bound, since 10KB does
not cover new ASICs which need 0x2840, and drop the duplicate check below.
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why & How]
Immediate (tearing) flips breaks self refresh state machine assumptions,
so disable PSR/Replay for them.
Tested-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On A+A the GPU HBM is exposed as specific-purpose memory described by
SRAT, and the GPU reaches it at a system physical address over the
internal link, so the driver needs a way to discover that range.
Add amdgpu_acpi_find_gpu_memory_in_srat(), which resolves the GPU
proximity domain from the SRAT GENERIC_AFFINITY entry matching the PCI
device handle, then returns the enabled, non-hotpluggable memory range
of that domain.
No caller yet, the users follow.
v2:
- A+A VF/passthrough is identified later by missing FB BAR0
(AMDGPU_VIRT_VRAM_IS_SPM), not by xgmi.connected_to_cpu.
v3:
- Rename best_base/best_len to mem_base/mem_len; first match, not a
comparison.
- Drop the unused legacy packed-BDF PCI handle format.
- Log "no acpi generic initiator found" without extra PCI BDF.
- Use dev_err for error-return logs.
v4:
- Reverse Christmas-tree local declarations, one per line.
- Reject NULL base_addr/length with -EINVAL.
Signed-off-by: Zhigang Luo <zhigang.luo@amd.com>
Signed-off-by: Fangsheng Huang <Fangsheng.Huang@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
mes_userq_mqd_create() returns -ENOMEM when copying the user MQD fails,
even though memdup_user() returns -EFAULT for a bad pointer or -EINTR
when interrupted by a signal. Return PTR_ERR() instead.
While at it drop the error messages. Neither case is a driver error and
userspace shouldn't be able to spam the log with invalid IOCTL
parameters.
v2: drop the error messages (Christian)
Fixes: 2c695d7c07 ("drm/amdgpu: enable compute/gfx usermode queue")
Fixes: 543b614537 ("drm/amdgpu: enable SDMA usermode queues")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_userq_get_doorbell_index() pins any BO passed as doorbell_handle
into the doorbell domain. For a regular BO, TTM moves it with
ttm_bo_move_null() and its contents are lost. Since an importer on the
same device gets the exporter's GEM object, a client can do this to a
buffer another client shared with it.
Reject BOs that were not created in the doorbell domain.
v2: drop the error message, use a local abo variable (Christian)
Fixes: f09c1e6077 ("drm/amdgpu: generate doorbell index for userqueue")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Temporarily increment the PM reference count during page handling to
ensure the GPU does not enter a sleep state.
v2 (Shahyan): Added commit description
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: Shahyan Soltani <shahyan.soltani@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Wake up the GPU while setting up user pages and validating the buffer.
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Co-developed-by: Shahyan Soltani <shahyan.soltani@amd.com>
Signed-off-by: Shahyan Soltani <shahyan.soltani@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For now use the same logic as the existing one, but it's not 100%
correct since:
1) most operations don't need the GPUs
2) some operations that need it are actually deferred
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Co-developed-by: Shahyan Soltani <shahyan.soltani@amd.com>
Signed-off-by: Shahyan Soltani <shahyan.soltani@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Move the duplicated VMID-mask-above-first_kfd_vmid computation out of
amdgpu_mes.c and amdgpu_amdkfd.c into a shared helper, called once
from amdgpu_vmid_mgr_init() to set vm_manager.vmid_uq_mask_gfxhub/mmhub.
MES and KFD now just read those fields instead of each computing them.
Signed-off-by: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_vmid_mgr is really the per-VMHUB structure used for VMIDs handed
out to kernel queues only, not to user queues. Rename it to
amdgpu_kq_vmid_mgr to make that unambiguous.
Signed-off-by: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why]
DM was declaring it's own list of has_alpha DRM formats to check against
for enabling alpha blending, when the format->has_alpha already does
that for us.
The self-declared list missed some has_alpha formats, causing
direct-scanout planes with such formats to appear opaque.
[How]
Drop the self-declared list, instead use the format->has_alpha field
directly.
Signed-off-by: Leo Li <sunpeng.li@amd.com>
Reviewed-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Submitting a cs ioctl will now wake up the GPU. The guard will
automatically drop the rpm ref when the cs ioctl goes out of scope.
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Co-developed-by: Shahyan Soltani <shahyan.soltani@amd.com>
Signed-off-by: Shahyan Soltani <shahyan.soltani@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
In struct amdgpu_vm_manager fix the indentation of
member "fault_info".
Signed-off-by: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
vmid_mask_gfxhub/vmid_mask_mmhub makes more sense logically in
vm_manager than the mes ip struct.
Signed-off-by: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On GFX 12.1 AID A0, extended-scope fine-grained local HBM must be
mapped MTYPE_UC when the device is in DPX, QPX or CPX compute partition
mode. In SPX mode the local MTYPE (MTYPE_NC by default) is kept.
Remote VRAM and system memory already use MTYPE_UC.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: David Yat Sin <david.yatsin@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add a helper to extract the AID revision from adev->rev_id, matching
the existing SOC_V1_0_DIE_REV_XCD(), and use it instead of open-coded
rev_id bit checks for AID A1 in gmc_v12_1 and kfd_svm.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: David Yat Sin <david.yatsin@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit 7e9fca7726.
This series breaks the build when CONFIG_KUNIT=y and
CONFIG_DRM_AMD_DC_KUNIT_TEST=n. When CONFIG_KUNIT=y,
the functions are not static, but when CONFIG_DRM_AMD_DC_KUNIT_TEST=n,
they should be.
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit b6b813e046.
This series breaks the build when CONFIG_KUNIT=y and
CONFIG_DRM_AMD_DC_KUNIT_TEST=n. When CONFIG_KUNIT=y,
the functions are not static, but when CONFIG_DRM_AMD_DC_KUNIT_TEST=n,
they should be.
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit 5a98f454a6.
This series breaks the build when CONFIG_KUNIT=y and
CONFIG_DRM_AMD_DC_KUNIT_TEST=n. When CONFIG_KUNIT=y,
the functions are not static, but when CONFIG_DRM_AMD_DC_KUNIT_TEST=n,
they should be.
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On a MacBookPro14,3 with a Radeon Pro 555 (Polaris11), the framebuffer
is at MC address 0 when amdgpu loads after a cold boot, as the firmware
leaves it (MC_VM_FB_LOCATION = 0x007f0000), while the VBIOS ASIC_Init
table places it at 0xF4_0000_0000 (0xf47ff400). amdgpu reads the
location once, at init, so after a re-POST (S3 resume or GPU reset)
the framebuffer has moved and the driver keeps programming the old
one: the SMU is handed a table that was never written and the GPU
does not come back, which leaves the internal panel black.
Resetting the ASIC on load makes ASIC_Init run before the driver reads
the location, so the driver uses the VBIOS placement from the start and
every later re-POST puts the framebuffer back where it already is.
Add the Radeon Pro 555 used in this machine to the existing VI reset
quirk table.
Tested on a MacBookPro14,3 on 6.18.49 with the quirk table backported
(the kernel also carries unrelated local PCI and ACPI patches for this
machine). The framebuffer is at 0x000000F400000000 after both cold and
warm boot, and the GPU survived 9 S3 cycles (lid close and rtcwake, one
of them with the lid closed for about 7.5 minutes and a USB-C disk
attached), each followed by a few minutes of 3D load; no ring timeouts
or VM faults were reported. The reset adds about 0.23 s to amdgpu init.
Suggested-by: Christian König <christian.koenig@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://lore.kernel.org/all/20260924132952.25054-1-fbeltranmillalen@gmail.com/
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Francisco Beltrán Millalén <fbeltranmillalen@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Only amdgpu_ctx_set_stable_pstate() communicates with the hw.
v3 (Shahyan): fixed typo in commit description
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: Shahyan Soltani <shahyan.soltani@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fix a hardware RTL bug in the TCP XNACK scoreboard reset path on MI300,
where a wave receiving a no-retry FAULT from UTC and subsequently
terminating may leave its scoreboard entry uncleared when reset events
arrive back-to-back.
This can cause a later, unrelated wave using the same scoreboard slot
to incorrectly inherit the XNACK error state, resulting in its VMEM
operations being dropped even though the wave never faulted.
Signed-off-by: Gang Ba <Gang.Ba@amd.com>
Reviewed-by: Jay Cornwall <jay.cornwall@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
PRT mappings have a bo_va but no backing BO. Reject them when validating
queue buffers and when looking up the WPTR for MES or USERQ_SIGNAL, before
passing the BO to the reservation or CPU mapping helpers.
Repeat the check at the WPTR lookups because the VM reservation is dropped
between initial validation and use.
Without this, pointing a user queue's WPTR at a PRT (sparse) VA mapping
makes mes_userq_create_wptr_mapping() dereference a NULL bo (base.bo) and
oops in the reservation helper:
BUG: kernel NULL pointer dereference, address: 0000000000000158
Oops: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 4558 Comm: amd_fuzzing Tainted: G E 7.1.0+
RIP: 0010:drm_exec_lock_obj+0x2b/0x140 [drm_exec]
CR2: 0000000000000158
Call Trace:
drm_exec_prepare_obj+0x25/0x60 [drm_exec]
mes_userq_create_wptr_mapping+0xc4/0x1c0 [amdgpu]
mes_userq_mqd_create+0x351/0x870 [amdgpu]
amdgpu_userq_ioctl+0x7ef/0xf80 [amdgpu]
drm_ioctl_kernel+0xac/0x110 [drm]
drm_ioctl+0x2c6/0x5b0 [drm]
amdgpu_drm_ioctl+0x4f/0x90 [amdgpu]
__x64_sys_ioctl+0xa0/0xf0
do_syscall_64+0xa3/0x4d0
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_vm_flush_compute_tlb() walks the XCCs of the compute partition one
at a time, and amdgpu_gmc_flush_gpu_tlb_pasid() submits the invalidation
to one XCC's KIQ and then busy-waits for its fence before the caller can
move on to the next. On a partition that owns every XCC of the device
that is num_xcc round trips back to back.
The XCCs do not depend on each other here. Each has its own KIQ ring,
ring lock and fence sequence, the page tables are already updated before
any invalidation is issued, and no invalidation needs another XCC to have
finished first.
Split the submit out of amdgpu_gmc_flush_gpu_tlb_pasid() and add
amdgpu_gmc_flush_gpu_tlb_pasid_xccs(), which queues the invalidation on
every XCC in the mask before collecting any of the fences, so the round
trips overlap. amdgpu_gmc_flush_gpu_tlb_pasid() becomes a one bit mask
call into it, so both entry points share the reset-domain handling and
the KIQ dispatch.
Note that amdgpu_vm_flush_compute_tlb() no longer stops at the first XCC
that fails. Every XCC in the mask is flushed regardless and the first
error is returned, which is a deliberate behaviour change for callers:
one XCC failing to queue its invalidation no longer leaves the remaining
XCCs unflushed.
Measured on gfx950 with all 8 XCCs in one SPX partition, revoking host
memory access for a pasid mapped on every XCC. Traced with ftrace,
amdgpu_vm_flush_compute_tlb() drops from 141 us to 65 us, against 23 us
for the slowest individual XCC flush. The KFD SVM ioctl carrying that
flush drops from 0.14 ms to 0.05 ms at p50.
Assisted-by: Cursor:claude-opus-5
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Taking the eviction lock is actually just one step which we need to do
in the critical section handling.
Rename the functions to reflect that, use the update parameters instead of the
vm to save the GFP flags.
v2: rebased and reordered
v3: fix rebase artefact, fix error handling in amdgpu_vm_pt_alloc
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Natalie Vock <nat@pixelcluster.dev>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
MGCG registers are PF-only. Skip programming them under SR-IOV.
Signed-off-by: Jane Jian <Jane.Jian@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Golden register init, the SDMA RB command switch setup, the GFXHUB
L2 protection fault controls and the guest-side FB_LOCATION programming
all touch registers a VF does not own, so leave them to the PF.
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The VM invalidation engine registers are always accessed with instance 0.
Under SR-IOV the instance is forwarded to RLCG-VFI as the target XCC, so
invalidating any GFXHUB other than the first one hits the registers of
XCC 0.
Pass the GFXHUB index as the register instance, and print the XCC ID in
the VFI access messages.
Signed-off-by: Jane Jian <Jane.Jian@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
During a GPU reset, evict_process_queues_cpsch() calls remove_queue_mes()
per queue, which fails with -EIO because the recovery path already holds
the reset domain write lock. The loop aborts on the first queue, leaving
the rest with is_active still set while the reset wipes the MES queue
table.
When the process later exits, process_termination_cpsch() sends
REMOVE_QUEUE for doorbells MES no longer knows about. MES rejects them,
the driver reports it as a timeout and concludes MES is unrecoverable,
so it requests another GPU reset.
Failed to evict queue 2
GPU reset(1) succeeded!
failed to remove queue from MES, doorbell=0x1002
MES might be in unrecoverable state, issue a GPU reset
GPU reset begin!. Source: 3
Mark every queue as reset in kgd2kfd_post_reset(). This clears is_active,
corrects the queue counts and sets is_reset so the queue is skipped later.
Add a verbose argument to set_queue_as_reset() so the full reset path
does not log a message for every queue.
Signed-off-by: YuanShang <YuanShang.Mao@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Set MES1 DATA as the last entry of the SR-IOV firmware list for MP0 15.0.8.
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Included psp_13_0_15 ip block for uniras
Signed-off-by: Ce Sun <cesun102@amd.com>
Reviewed-by: Tao Zhou <tao.zhou1@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Ed Maste <emaste@FreeBSD.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add the per-IP hardware wiring that drives the mes_dbgext core on gfx11
(mes_v11_0 / gfx_v11_0) and gfx12 (mes_v12_0 / gfx_v12_0): handle
MES_MISC_OP_SETUP_MES_DBGEXT in the misc-op path, unmask the MES debug
interrupt, select the flat (gfx11) or zoned (gfx12) buffer format in
sw_init(), start/stop the feature from hw_init()/hw_fini(), and drain
the log from the EOP interrupt handler while the feature is active.
Signed-off-by: Yongqiang Sun <Yongqiang.Sun@amd.com>
Reviewed-by: Shaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add the core MES debug-extension logging layer in amdgpu_mes: allocate
and initialize the firmware log buffer, and print messages to dmesg
via interrupt or polling collection modes, with start/stop/notify
and an amdgpu_mes_dbgext debugfs control.
Expose the mes_dbgext_buffer_size and mes_dbgext_options module
parameters to configure the buffer size and collection mode, and wire
the debugfs init (amdgpu_debugfs_mes_init).
Signed-off-by: Yongqiang Sun <Yongqiang.Sun@amd.com>
Reviewed-by: Shaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Assume that the AC limits are the maximum of all power states,
and the DC limits are the maximum of battery power states.
This shouldn't make any difference in practice, but is
cleaner and more robust against bogus information in the VBIOS.
Fixes: e6c5d36756 ("drm/amd/pm/si: Fix updating clock limits from power states")
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Link: https://patch.msgid.link/20260923120354.1027996-2-timur.kristof@gmail.com
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm_gem_handle_create_tail() skips the close() callback when open()
returns an error, so a bo_va added before a failing step is never
removed. It stays linked on bo->vm_bo and a later eviction of that BO
dereferences the VM after it has been torn down.
Validate and fence the BO before adding the bo_va, drop the bo_va when
attaching the eviction fence fails, and propagate the allocation failure
from amdgpu_vm_bo_add().
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
An imported dma-buf BO backed by a VRAM-capable exporter holds no host
pages of its own, yet when validated into TTM_PL_TT it is charged against
the GTT page budget; importing many such buffers can exhaust that budget
and fail with -ENOMEM even though no host memory is used. Set
AMDGPU_GEM_CREATE_PREEMPTIBLE on these imports so they land in the PREEMPT
domain instead, which is GTT-like and still bound (keeping the exporter
resident) but carries no host-page limit.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use amdgpu_vm_bo_move() for updating the NPA BO memory accounting stats
when the NPA BO is moved out from NPA domain into an empty placement.
Without this, when the app finishes, it spits out a warning:
"VM memory stats for proc .. task .. is non-zero when fini"
This was only affecting the memory accounting stats, we were not leaking
any memory here.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Acked-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The export cleanup worker checks amdgpu_ualink_check_conn_ready() once
when building npa_release_bitmap, but not again in the send loop. Cleanup
workers run concurrently on the unbounded NPA WQ (one per exported
handle), so when a peer's egress dies many workers have already seeded
npa_release_bitmap from a connection that was still ESTABLISHED and all
issue an LSDMA copy to the dead peer, flooding the log with repeated
NPA-REVOKE send failures.
Re-check the connection immediately before each send, as the tlb
shootdown, force_retry_rpcs and unmap_all_npa_addr loops already do, so
once the first worker marks a peer NOT_READY the rest skip it. With the
re-check in place, seed npa_release_bitmap with a single bitmap_copy() of
importers_bitmap instead of a separate build loop; peers that fail the
re-check or the send are dropped from the bitmap in the send loop.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why]
mod_power_remove_stream() shifts the remaining power_entity slots down
but does not move replay_events, and mod_power_add_stream() does not
initialize it. replay_events therefore stay bound to the map slot
instead of the stream.
When several streams are disabled in one atomic commit,
amdgpu_dm_mod_power_update_streams() removes them one after another.
The eDP stream can then be looked up in a slot whose stale
replay_events already have replay_event_hw_programming set, so
amdgpu_dm_replay_set_event() returns early ("already in desired state")
without calling mod_power_set_replay_event(). Replay is not disabled
before the eDP panel is powered off. After DPMS on, the sink reports
neither replay state nor frame lock (DPCD 0x378 = 0x00, no error bits),
so the HPD IRQ recovery does not trigger and the panel stays black
until a full modeset.
Seen with an eDP panel using FreeSync Replay plus two DP-MST displays:
DPMS off/on of all outputs leaves eDP black, while DPMS of eDP alone
works. Doing an eDP-only DPMS first makes the next all-output DPMS
fail reliably.
[How]
Shift replay_events together with the PSR cached fields in
mod_power_remove_stream() and initialize it to replay_event_vsync in
mod_power_add_stream(), matching the psr_event_vsync initial value used
for PSR (both vsync events are driven together by
amdgpu_dm_crtc_set_static_screen_optimze()).
Tested on 7.3.0-rc3 (238650ef6c): the reproducer above now recovers
reliably, and Replay still engages when the screen is idle.
The issue was debugged with help from an AI assistant (Claude), which
analysed ftrace/kprobe traces and the driver source, pointed to the
missing replay_events handling and suggested this change. I collected
the traces and built and tested the fix on the affected hardware.
Fixes: 4cef2ac4c7 ("drm/amd/display: Introduce power module on Linux")
Assisted-by: Claude:claude-opus-5
Signed-off-by: Simon Polack <spolack+git@mailbox.org>
Reviewed-by: Ray Wu <ray.wu@amd.com>
Tested-by: Daniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When CONFIG_DRM_AMD_DC_DCE is disabled, most functions insde dce110_hwseq.c
are no longer referenced, causing a lot of warnings such as
drivers/gpu/drm/amd/amdgpu/../display/dc/hwss/dce110/dce110_hwseq.c:3510:13: error: 'dce110_enable_analog_link_output' defined but not used [-Werror=unused-function]
3510 | static void dce110_enable_analog_link_output(
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/drm/amd/amdgpu/../display/dc/hwss/dce110/dce110_hwseq.c:3391:13: error: 'dce110_set_cursor_attribute' defined but not used [-Werror=unused-function]
3391 | static void dce110_set_cursor_attribute(struct pipe_ctx *pipe_ctx)
Replace the #ifdef with an IS_ENABLED() check that leads to an empty function
in this configuration, so the compiler can eliminate the unused functions
and not complain about it.
Fixes: cd2cfe4d15 ("drm/amd/display: gate DCE ASIC code behind CONFIG_DRM_AMD_DC_DCE")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Fangzhi Zuo <jerry.zuo@amd.com>
Tested-by: Daniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The Sun XVR-300 is a FireMV 2200 PCIe (RV370) board with an
FCode-only Open Firmware ROM and no COMBIOS to parse, so the
generic connector table leaves both DVI-I outputs undetected. Both
XVR-300 SKUs report device ID 5b64 with an ATI subsystem ID rather
than Sun's, so match the OF node name prefix instead. The table
comes from the board's FCode.
Signed-off-by: Imre Kaloz <kaloz@kernel.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
