GFX12.1 doesn't follow the pair-wise CU mask requirement as
is the case for GFX 10,GFX 11 and GFX 12.
Currently, the ioctl call is returning error because of this
incorrect condition of checking for a pair of CUs to be enabled
in the CU mask.
Fix the issue by removing this check for GFX 12.1.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On process teardown, kfd_process_notifier_release_internal() called
kfd_process_dequeue_from_all_devices() (which issues the MES
SET_SHADER_DEBUGGER process_ctx_flush=1 that retires the process context)
before kfd_dbg_trap_disable(). For a process under a debugger, the debug
teardown path (kfd_dbg_trap_disable -> kfd_dbg_trap_deactivate) then issues
several SET_SHADER_DEBUGGER process_ctx_flush=0 packets (reset wave launch
mode, clear debug flags, reset MES debug mode). Because those packets carry
the process_context_addr, MES re-adds the process to its scheduler list
after the flush had retired it.
Teardown then frees proc_ctx_bo, and a later MES scheduling pass dereferences
the freed process context, producing a GPUVM page fault (vmid 0, GC_UTCL2)
on the freed page. This is intermittent because it depends on an MES pass
landing on the process before its GART page is reused. It is specific to the
debugger path, where queues are added with skip_process_ctx_clear=1 so the
last REMOVE_QUEUE does not clear the context and delisting relies on the
flush.
Reorder teardown so debug is disabled first, and the process context is
flushed/retired last, so nothing re-adds the process after it is retired.
kfd_dbg_trap_disable() now runs while queues still exist, which matches the
normal live debug-disable ordering.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
status_ptr was declared as u64 * but the WB allocation is only one u32.
Writing through a u64 * pointer zeroed 8 bytes, corrupting the adjacent
WB slot. Fix by using u32 * to match the actual allocation size.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The heavy weight shootdown need special handling in MES sched pipe
Signed-off-by: Shaoyun Liu <shaoyun.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The optimized QUE_SLEEP was only programmed on the HWS packet-manager
path via a PM4 WRITE_DATA packet, which is skipped when MES is enabled.
Add gfx12.1 get_iq_wait_times/build_dequeue_wait_counts_packet_info
callbacks and program CP_IQ_WAIT_TIME2 through the MES WRITE_REG op at
start_cpsch, per XCC.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Unmap the KIQ pipe during driver unload so stale pointers do
not carry over to the next driver load.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Declare MEC_1 and RLC firmware in gfx_v12_1 with MODULE_FIRMWARE()
so modinfo and distro firmware packaging pick them up.
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A workaround sequence which toggles MGCG override is required to enable
hardware MGCG. As this is not the final solution, don't change the CGCG
flags.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
RLC fw disables GFX CGCG and CGLS on GC 12.1.0 by default, so clear the
corresponding CG flags and keep GUI idle interrupts disabled during gfx init.
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
MEC will have the specific FW binary for silicon rev 0.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
To avoid allocation failures in high-density queue scenarios, move user
mode compute MQD allocations for GFX12.1 from GTT sub-allocator to
dedicated kernel memory via amdgpu_amdkfd_alloc_kernel_mem.
This ensures that compute queue creation is not bottlenecked by the
fixed size of the sub-allocator pool.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For GC info table v1.5, calculate the WGP totals
for SA and SA1 independently, then initialize
max_cu_per_sh with the greater value.
If both totals are equal, keep the SA value.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Shiwu Zhang <shiwu.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Initialize amdgpu_gfx_config using values from
gc_info_v1_5.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The GB_ADDR_CONFIG_READ is initialized to zero.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Both of them are not required for gfx_v12_1.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use the default all zero settings.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The underlying functionality was removed
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The underlying functionality was removed
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
add_hw_queue, remove_hw_queue, set_hw_resources_1 and inv_tlbs_pasid
read master_xcc_ids[MES_PIPE_INST(xcc_id, SCHED)]. That only worked
for xcc_id == 0; with a non-zero target (e.g. partition-1 master in
DPX) it read an uninitialised slot. Index by xcc_id consistently, and
fix the related shared_cmd_buf_gpu_addr[] index in
set_hw_resources_1() to use MES_PIPE_INST(master_xcc_id, pipe).
All indexing stays in logical XCC space, so this remains correct with
harvested XCCs; the existing ip_map / GET_INST() translation handles
the logical-to-physical conversion at the HW register boundary.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On multi-node configurations without XGMI interconnect (e.g. A+A),
num_physical_nodes can be greater than 1 even though XGMI is not supported.
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Control register is access restricted. Use the read-only copy to read
the partition settings.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
DPX is only valid partition mode in NPS2.
Updated check which was making CPX as
available compute partition mode instead of DPX
Signed-off-by: Mangesh Gadre <Mangesh.Gadre@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
NICs attached to remote GPUs are unable to access VRAM coherently.
Hence, use DEVICE_PRIVATE for GFX12.1
v2: Squash in Fix migrate_vma DEVICE_COHERENT setting
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Correct regGFX_IMU_PARTITION_SWITCH value to suitable for harvest
situation for gfx v12.1.
Signed-off-by: Likun Gao <Likun.Gao@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Jinzhou Su <jinzhou.su@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Check LSDMA PIO error status bits after each transfer completes to
detect silent copy failures caused by issues such as IFoE (Infinity
Fabric over Ethernet) errors.
The following error conditions are checked:
- Write return NACK / general error
- Read return NACK / general error
- Invalid address
- Zero count
- DRAM ECC error
Log the error via dev_warn() and clear only the set error bits
(write-1-to-clear) so that subsequent transfers can proceed once the
underlying issue is resolved.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add mutex to serialize LSDMA PIO copy and fill operations for
concurrent access from multiple threads.
A mutex is used rather than a spinlock because the underlying
lsdma_v7_1_wait_pio_status() polls with udelay(), which can
busy-wait for up to usec_timeout microseconds.
It is optional to call mutex_destroy for clarity and debug
correctness, it is no-op for release build, leave it as lsdma
block no ip callback funcs to call mutex_destroy.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On GFX 12.1, all SDMA engines are the same from HW perspective.
As a result, report all SDMA engines as regular/PCIe SDMA engines.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A temporary workaround to unblock usermode tests.
Will revert once issue resolved.
Signed-off-by: Feifei Xu <Feifei.Xu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Add JPEG v5.0.2 SRIOV code path
- Add JPEG MMSCH init table transfer
- Skip unsupported actions such as power gating
and per queue reset for SRIOV VF.
Signed-off-by: Bokun Zhang <Bokun.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Add VCN v5.0.2 SRIOV code path
- Add VCN MMSCH init table transfer
- Skip unsupported actions such as power gating
and per queue reset for SRIOV VF.
Signed-off-by: Bokun Zhang <Bokun.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
jpeg_v5_0_2_is_idle() initializes ret to false and then accumulates ring
idle status using &=. Since false & condition always remains false, the
function can never report the JPEG block as idle.
Initialize ret to true so the function returns true only when all JPEG
rings report RB_JOB_DONE.
Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Replace external find calls with GNU Make’s $(rwildcard …)
to collect files. Depending on ‘find’ can fail if it’s
unavailable or returns no matches, which left CFLAGS_*
unset. In that case, floating-point code was built
with -mno-sse, leading to build failures.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The fatal error interrupt is broadcast to every device of the hive and
they all race for amdgpu_ras_global_ras_isr(), which hands -EBUSY to
everyone but the winner. Treating that as a failure returns before the
device is marked, so seven devices out of eight are left without their
fatal and SRAM ECC state, the one that actually logged the error among
them. KFD then tells the processes on those devices that the reset was
a plain hang.
-EBUSY only means the reset has already been asked for. Record the
state anyway and leave the request to the winner.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
kfd_signal_reset_event() picks between KFD_HW_EXCEPTION_ECC and
KFD_HW_EXCEPTION_GPU_HANG from the SRAM ECC flag, and only delivers the
memory exception event for the former. Nothing raises that flag on the
RAS module paths, so a reset caused by an uncorrectable or a consumed
poison error is reported to every process on the device as a plain hang
and the runtime carries on instead of tearing the workload down.
Raise it the way the per IP callbacks used to.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The failure paths free the RAS manager but keep the flags that advertise
it as the owner of RAS and of the RAS firmware load. Every entry point
gated on those hits the NULL manager, hw_init turns that into -EINVAL
and the whole probe fails.
Clear both once the manager is gone, and route the allocation failure
through the same place.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Tao Zhou <tao.zhou1@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
ras_mp1_set_debug_mode() reports -EOPNOTSUPP where the control does not
exist, an SR-IOV guest among them. Treating that as fatal fails
ras_core_sw_init() and takes RAS down on every VF.
Nothing else in the MP1 block depends on it, so carry on.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Tao Zhou <tao.zhou1@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Different from detect_and_reset, reset_hw_queue performs per queue
reset. Per queue reset setting requires reset_queue_only=1 and hung
queue's doorbell_offset in RESET packet. This packet must be sent to MES
pipe 0(SCHED_PIPE).
We also need to set enable_compute_pipe_reset=1 in SET_HW_RESOURCES so
when queue reset fails, MES will proceed pipe reset.
This patch also fixes a bug in mes_v12_1_reset_hw_queue function. Instead
of using MESAPI__RESET offset for api_status, the existing code uses
MESAPI__REMOVE_QUEUE.
Signed-off-by: Amber Lin <amber.lin@amd.com>
Reviewed-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
removing MES oversubscription timer will affect both amdgpu/amdkfd, so
add the similar timer for amdgpu userq as well.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
the issue only happens with oversubscription when gpu has no
workload, the root cause is mes oversubscription timer, so
disable mes timer and make a similar timer in kfd to resolve
the issue.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
new api will be used to workaround a HW scheduler issue
for 100% usage when gpu has no workload.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This version brings along the following updates:
- Fix HPO test regressions.
- Refactor DPP_SET_INPUT_TRANSFER_FUNC to drop pipe_ctx params.
- Use fast update path for address-only plane flips.
- Split OPTC_PIPE_CONTROL_LOCK into smaller helpers.
- Fix DPREFCLK override when SMU isn't available.
- Fix CalculateFlipSchedule calculation.
- Add hook to disable alt-ch in PMO.
- Update alt-ch size calculations.
- Enable min dispclk ODM on DCN42.
- Remove RMCM tetrahedral cube from dc_plane_state.
- Fix mismatch number of OPP/DPP accounting.
- Adjust vblank_nom policy for HW SDP throttling.
- Add passthrough visual confirm.
- Expand amdgpu_dm KUnit test coverage across cursor, MST, EDID, HDMI,
IRQ, i2c, framebuffer, tiling/modifier and HPD paths.
- Various fixes for Kunit tests related to HDMI FRL, IRQ and cursors.
Reviewed-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Taimur Hassan <Syed.Hassan@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why&How]
An interrupt racing DM teardown can reach amdgpu_dm_irq_schedule_work()
after the IRQ workqueue is destroyed, passing a NULL wq to queue_work()
and crashing in __queue_work(). Bail out early when adev->dm.irq_wq is
NULL so a registered handler is simply not queued.
Reviewed-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why]
The requirement is that VSTARTUP has to come before vsync, line 0.
This means that the VSTARTUP has to be in the blank front porch.
The current implementation in DML2.1 does meet this requirement by
adjusting vblank_nom to the full vblank_avail but this has two
undesirable side effects:
1. When vblank is nominal this pushes VUPDATE on the line after VBLANK
start. For Replay this gives only 1 line worth of time to program the
SDP and blank the DPG. If we miss this window then we have to wait
an entire frame before we can retry IPX entry. Residency is impacted.
2. The prefetch schedule is suboptimal. Instead of leveraging a very
long vblank for power we wake early and block c-state earlier.
[How]
Use the same policy as prior ASIC - clamp the vblank_nom adjustment to
be the maximum of vblank_nom or the size of the back porch + 2 lines.
The back porch includes the vsync width in it in the calculation.
Reviewed-by: Dillon Varone <dillon.varone@amd.com>
Signed-off-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why&How]
Wrapper currently populates the vblank nom as the entire blank for a
given timing. This should be constrained to a sane maximum but still
controlled by the caller's (wrapper) policy, which for now will be the
existing bounding box value.
Reviewed-by: Austin Zheng <Austin.Zheng@amd.com>
Signed-off-by: Dillon Varone <Dillon.Varone@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Added a new visual_confirm mode VISUAL_CONFIRM_DM_PASSTHROUGH so the confirm
color is supplied by the DM instead of computed by DC.
- Relocated struct tg_color above dc_flip_addrs so it can be embedded by value.
Reviewed-by: Joshua Aberback <joshua.aberback@amd.com>
Signed-off-by: Gaghik Khachatrian <gaghik.khachatrian@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add KUnit tests for amdgpu_dm_create_i2c() covering the OEM i2c adapter
and the hardware i2c bus creation paths.
Assisted-by: Copilot:Claude-Opus-4.8
Reviewed-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Bhawanpreet Lakha <bhawanpreet.lakha@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add KUnit tests for amdgpu_dm_i2c_xfer() covering a failed hardware
command submission and the OEM i2c path with no device present.
Assisted-by: Copilot:Claude-Opus-4.8
Reviewed-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Bhawanpreet Lakha <bhawanpreet.lakha@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
