drm/amd/display: Refactor stream validation

Browse source

[Why]
amdgpu_dm_create_validate_stream_for_sink() drove its RGB -> YUV422 ->
YUV420 chroma fallback by recursing and toggling the shared
aconnector->force_yuv420_output / force_yuv422_output fields, resetting
them after each recursive call. Those fields have no locking and the
function runs concurrently on the same connector from two paths: the
connector probe worker (->mode_valid) and a compositor's atomic check
(dm_update_crtc_state). When both run at once, one thread can clear the
override just before the other tests its exit condition, so the exit is
missed and validation loops indefinitely, hanging the modeset path.

[How]
- Replace the recursion with an explicit loop over the chroma encodings
  wrapping the existing bpc walk.
- Carry the encoding/bpc selection on the stack, passed by value into
  create_stream_for_sink() / fill_stream_properties_from_drm_display_mode(),
  instead of mutating shared connector state.
- Derive the supported encodings and bit depths into bitmaps and drive
  validation from them, gating each candidate on the sink's advertised
  capability so unsupported encodings are never retried.
- Move encoding selection entirely to the caller and pass the chosen
  dc_pixel_encoding into fill_stream_properties_from_drm_display_mode().

v2: sqaush in KUnit test fixes, merge with drm-misc changes (Alex)

Assisted-by: Copilot:claude-opus-4.8
Reviewed-by: Jerry Zuo <jerry.zuo@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Signed-off-by: Roman Li <roman.li@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

...

This commit is contained in:

Ivan Lipski 2026-07-21 17:47:11 -04:00 • committed by Alex Deucher

parent be784e161a

commit 0b0ff65d3c

3 changed files with 346 additions and 132 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

317

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_connector.c

View file

8

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_connector.h

View file

155

drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_connector_test.c

View file