drm/amdgpu/vce: fix integer overflow in image size

Browse source

Fix a security vulnerability where malicious VCE command streams
with oversized dimensions (e.g. 65536×65536) cause 32-bit integer
overflow, wrapping the calculated buffer size to 0. This bypasses
validation and allows GPU firmware to perform out-of-bound memory
access.

The fix uses 64-bit arithmetic to detect overflow and rejects
invalid dimensions before they reach the hardware.

V2: remove redundant check
V3: modify max height value
V4: remove size64

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

...

This commit is contained in:

Boyuan Zhang 2026-05-25 11:34:27 -04:00 • committed by Alex Deucher

parent dbb02b4755

commit cbe408dba5

1 changed files with 14 additions and 3 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

17

drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c

View file