drm/amdgpu: Flush the pasid on all XCCs in parallel

Browse source

amdgpu_vm_flush_compute_tlb() walks the XCCs of the compute partition one
at a time, and amdgpu_gmc_flush_gpu_tlb_pasid() submits the invalidation
to one XCC's KIQ and then busy-waits for its fence before the caller can
move on to the next. On a partition that owns every XCC of the device
that is num_xcc round trips back to back.

The XCCs do not depend on each other here. Each has its own KIQ ring,
ring lock and fence sequence, the page tables are already updated before
any invalidation is issued, and no invalidation needs another XCC to have
finished first.

Split the submit out of amdgpu_gmc_flush_gpu_tlb_pasid() and add
amdgpu_gmc_flush_gpu_tlb_pasid_xccs(), which queues the invalidation on
every XCC in the mask before collecting any of the fences, so the round
trips overlap. amdgpu_gmc_flush_gpu_tlb_pasid() becomes a one bit mask
call into it, so both entry points share the reset-domain handling and
the KIQ dispatch.

Note that amdgpu_vm_flush_compute_tlb() no longer stops at the first XCC
that fails. Every XCC in the mask is flushed regardless and the first
error is returned, which is a deliberate behaviour change for callers:
one XCC failing to queue its invalidation no longer leaves the remaining
XCCs unflushed.

Measured on gfx950 with all 8 XCCs in one SPX partition, revoking host
memory access for a pasid mapped on every XCC. Traced with ftrace,
amdgpu_vm_flush_compute_tlb() drops from 141 us to 65 us, against 23 us
for the slowest individual XCC flush. The KFD SVM ioctl carrying that
flush drops from 0.14 ms to 0.05 ms at p50.

Assisted-by: Cursor:claude-opus-5
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

...

This commit is contained in:

David Yat Sin 2026-09-18 17:29:34 -04:00 • committed by Alex Deucher

parent 0dc93a683a

commit d3859b81fd

3 changed files with 173 additions and 78 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

237

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c

View file

8

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h

View file

10

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c

View file