Add soc_v1_0 reset handler for mode2 reset
v2: Remove XGMI parallel reset dispatch and xgmi_update_topology()
call, reset on soc_v1_0 is per-device only. (Lijo)
v3: Remove ras as will be plugged later as separate ip. Remove vcn from
ip mask as vcn not part of reset sequence yet (Lijo)
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The default SMU message timeout may be too short for the
PrepareMp1ForUnload message on smu_v15_0_8, leading to spurious
-ETIME errors under load or slow firmware response conditions.
Introduce SMU_V15_0_8_MSG_TIMEOUT_US (5 seconds) and apply it to:
- smu_v15_0_8_notify_unload(): switch from smu_cmn_send_smc_msg()
to smu_cmn_send_smc_msg_with_params_ext() to pass the explicit
timeout.
v2: Use c2pmsgctl and args directly (Lijo)
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When VBIOS is not available, fallback to use 100M for refclock value.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add the get_rom_offset callback to retrieve the VBIOS ROM offset
from REGS_ROM_OFFSET_CTRL for vbios imaging fetching from spirom.
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Enable translate further for gc v12_1 to resolve
the tlb reach issue
v2: ensure PTE.P is set to 1 for leaf PTE
v3: Do not clear the P bit when it has been set
on a PDB0 entry by the common VM update code
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Horatio Zhang <hongkun.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_vm_sdma_update() keeps the number of free dwords of the current
job in a local variable. When the job runs out of room it is committed
and a new one is allocated, but that local variable is not refreshed and
still describes the job which was just submitted.
The stale value is then used to decide how many PTEs fit into the new
job. If it happens to be copy_pte_num_dw + 7, or one more than that, the
result is zero. copy_pte() encodes the transfer size as count * 8 - 1,
so a count of zero asks the engine for a 1GB copy starting inside the IB
pool. That runs off the end of the pool, faults on the first unmapped
GART page above it and hangs the ring.
Re-initialize the estimate from the newly allocated job, which is empty
and therefore has its full budget available.
Cc: stable@vger.kernel.org
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
VM fault status is 64 bits on gfxhub v12_1 and
mmhub v4_2, with fields such as FED located in the
upper status register.
Add an optional HI32 fault status callback to help
identify FED status on gfxhub v12_1 and mmhub v4_2
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Horatio Zhang <hongkun.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use global SMN addresses for MMHUB v4_2_0 register access
across all instances.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Update the address mask to support 52-bits addresses when validating the
PDE addresses for GFX 12.1. Currently the address mask was incorrect and
only permitted 48-bits addresses. Additionally, drop the BUG_ON() and
replace it with code to reset the invalid address and valid bit if the
address verification fails. This should help generate a page fault instead
of crashing the kernel.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Philip Yang <philip.yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Clean up gfx_v12_1_ring_emit_vm_flush function, since
PFP_SYNC_ME cmd not support on gc v12_1, and it does
not have gfx ring.
Signed-off-by: Likun Gao <Likun.Gao@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Work around this in both the amdgpu GMC and KFD SVM paths by explicitly
selecting MTYPE_NC for remote memory when amdgpu_mtype_local is 0. Remove
this once rev_id-based AID detection is reliable.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit bfeb7b28e9.
Revert the rev 0x1 (B0) special case for IP_VERSION(12, 1, 0) that
forced noretry_default off. Align GFX12.1.0 with the GC >= 10.3.0
noretry default.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The current combination of PTE flags to translate a retry
fault into a no-retry fault is not valid for GFX12. Use a
valid no-retry PTE flags combination that works for GFX 12.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
gmc v12_1 currently derives the number of mmhub vmhubs
from the aid_mask when populating vmhubs_mask. This
breaks when mmhub is represented by mid and mmhub
inst_mask does not match the aid_mask.
Iterate over adev->mmhub.inst_mask instead so vmhubs_mask
reflects the discovered mmhub instances.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Mangesh Gadre <Mangesh.Gadre@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
mmhub v4_2_0 used adev->aid_mask as the mid_mask
when programming MMHUB instances. That assumption
is not valid for products with more AIDs than MIDs.
Track discovered MMHUB instances in mmhub.inst_mask
and use that mask for mmhub v4_2_0 programming paths.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For GMC v12.1.0, the IH cookie NodeID spacing is 8 (MID0=0x0, MID1=0x8),
not 4. The wrong divisor caused NPA page faults (node_id=8) to select
mmhub2 instead of the correct mmhub1.
Fix the divisor from 4 to 8, and add a comment documenting the NodeID
to DIE mapping for reference.
Warn once if page fault from the vmhub but vmhub print fault status
function not initialized because not all hubs are initialized.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
GFX12.1 doesn't follow the pair-wise CU mask requirement as
is the case for GFX 10,GFX 11 and GFX 12.
Currently, the ioctl call is returning error because of this
incorrect condition of checking for a pair of CUs to be enabled
in the CU mask.
Fix the issue by removing this check for GFX 12.1.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On process teardown, kfd_process_notifier_release_internal() called
kfd_process_dequeue_from_all_devices() (which issues the MES
SET_SHADER_DEBUGGER process_ctx_flush=1 that retires the process context)
before kfd_dbg_trap_disable(). For a process under a debugger, the debug
teardown path (kfd_dbg_trap_disable -> kfd_dbg_trap_deactivate) then issues
several SET_SHADER_DEBUGGER process_ctx_flush=0 packets (reset wave launch
mode, clear debug flags, reset MES debug mode). Because those packets carry
the process_context_addr, MES re-adds the process to its scheduler list
after the flush had retired it.
Teardown then frees proc_ctx_bo, and a later MES scheduling pass dereferences
the freed process context, producing a GPUVM page fault (vmid 0, GC_UTCL2)
on the freed page. This is intermittent because it depends on an MES pass
landing on the process before its GART page is reused. It is specific to the
debugger path, where queues are added with skip_process_ctx_clear=1 so the
last REMOVE_QUEUE does not clear the context and delisting relies on the
flush.
Reorder teardown so debug is disabled first, and the process context is
flushed/retired last, so nothing re-adds the process after it is retired.
kfd_dbg_trap_disable() now runs while queues still exist, which matches the
normal live debug-disable ordering.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
status_ptr was declared as u64 * but the WB allocation is only one u32.
Writing through a u64 * pointer zeroed 8 bytes, corrupting the adjacent
WB slot. Fix by using u32 * to match the actual allocation size.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The heavy weight shootdown need special handling in MES sched pipe
Signed-off-by: Shaoyun Liu <shaoyun.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The optimized QUE_SLEEP was only programmed on the HWS packet-manager
path via a PM4 WRITE_DATA packet, which is skipped when MES is enabled.
Add gfx12.1 get_iq_wait_times/build_dequeue_wait_counts_packet_info
callbacks and program CP_IQ_WAIT_TIME2 through the MES WRITE_REG op at
start_cpsch, per XCC.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Unmap the KIQ pipe during driver unload so stale pointers do
not carry over to the next driver load.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Declare MEC_1 and RLC firmware in gfx_v12_1 with MODULE_FIRMWARE()
so modinfo and distro firmware packaging pick them up.
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A workaround sequence which toggles MGCG override is required to enable
hardware MGCG. As this is not the final solution, don't change the CGCG
flags.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
RLC fw disables GFX CGCG and CGLS on GC 12.1.0 by default, so clear the
corresponding CG flags and keep GUI idle interrupts disabled during gfx init.
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
MEC will have the specific FW binary for silicon rev 0.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
To avoid allocation failures in high-density queue scenarios, move user
mode compute MQD allocations for GFX12.1 from GTT sub-allocator to
dedicated kernel memory via amdgpu_amdkfd_alloc_kernel_mem.
This ensures that compute queue creation is not bottlenecked by the
fixed size of the sub-allocator pool.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For GC info table v1.5, calculate the WGP totals
for SA and SA1 independently, then initialize
max_cu_per_sh with the greater value.
If both totals are equal, keep the SA value.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Shiwu Zhang <shiwu.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Initialize amdgpu_gfx_config using values from
gc_info_v1_5.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The GB_ADDR_CONFIG_READ is initialized to zero.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Both of them are not required for gfx_v12_1.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use the default all zero settings.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The underlying functionality was removed
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The underlying functionality was removed
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
add_hw_queue, remove_hw_queue, set_hw_resources_1 and inv_tlbs_pasid
read master_xcc_ids[MES_PIPE_INST(xcc_id, SCHED)]. That only worked
for xcc_id == 0; with a non-zero target (e.g. partition-1 master in
DPX) it read an uninitialised slot. Index by xcc_id consistently, and
fix the related shared_cmd_buf_gpu_addr[] index in
set_hw_resources_1() to use MES_PIPE_INST(master_xcc_id, pipe).
All indexing stays in logical XCC space, so this remains correct with
harvested XCCs; the existing ip_map / GET_INST() translation handles
the logical-to-physical conversion at the HW register boundary.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On multi-node configurations without XGMI interconnect (e.g. A+A),
num_physical_nodes can be greater than 1 even though XGMI is not supported.
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Control register is access restricted. Use the read-only copy to read
the partition settings.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
DPX is only valid partition mode in NPS2.
Updated check which was making CPX as
available compute partition mode instead of DPX
Signed-off-by: Mangesh Gadre <Mangesh.Gadre@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
NICs attached to remote GPUs are unable to access VRAM coherently.
Hence, use DEVICE_PRIVATE for GFX12.1
v2: Squash in Fix migrate_vma DEVICE_COHERENT setting
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Correct regGFX_IMU_PARTITION_SWITCH value to suitable for harvest
situation for gfx v12.1.
Signed-off-by: Likun Gao <Likun.Gao@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Jinzhou Su <jinzhou.su@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Check LSDMA PIO error status bits after each transfer completes to
detect silent copy failures caused by issues such as IFoE (Infinity
Fabric over Ethernet) errors.
The following error conditions are checked:
- Write return NACK / general error
- Read return NACK / general error
- Invalid address
- Zero count
- DRAM ECC error
Log the error via dev_warn() and clear only the set error bits
(write-1-to-clear) so that subsequent transfers can proceed once the
underlying issue is resolved.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add mutex to serialize LSDMA PIO copy and fill operations for
concurrent access from multiple threads.
A mutex is used rather than a spinlock because the underlying
lsdma_v7_1_wait_pio_status() polls with udelay(), which can
busy-wait for up to usec_timeout microseconds.
It is optional to call mutex_destroy for clarity and debug
correctness, it is no-op for release build, leave it as lsdma
block no ip callback funcs to call mutex_destroy.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On GFX 12.1, all SDMA engines are the same from HW perspective.
As a result, report all SDMA engines as regular/PCIe SDMA engines.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A temporary workaround to unblock usermode tests.
Will revert once issue resolved.
Signed-off-by: Feifei Xu <Feifei.Xu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Add JPEG v5.0.2 SRIOV code path
- Add JPEG MMSCH init table transfer
- Skip unsupported actions such as power gating
and per queue reset for SRIOV VF.
Signed-off-by: Bokun Zhang <Bokun.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Add VCN v5.0.2 SRIOV code path
- Add VCN MMSCH init table transfer
- Skip unsupported actions such as power gating
and per queue reset for SRIOV VF.
Signed-off-by: Bokun Zhang <Bokun.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
jpeg_v5_0_2_is_idle() initializes ret to false and then accumulates ring
idle status using &=. Since false & condition always remains false, the
function can never report the JPEG block as idle.
Initialize ret to true so the function returns true only when all JPEG
rings report RB_JOB_DONE.
Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
