drm/amd/display: Use u16 for drm_bpp in DSC calculations

Browse source

[Why]
DSC calculations fail because the u16 bits_per_pixel from
the DRM struct is being casted to the u8 drm_bpp parameters
and locals. Integer wraparound is happening because this
value is greater than 255.

[How]
Use u16 to match what's in the structure instead of u8.

Signed-off-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Reviewed-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Acked-by: Qingqing Zhuo <qingqing.zhuo@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

...

This commit is contained in:

Nicholas Kazlauskas 2020-05-21 12:32:45 -04:00 • committed by Alex Deucher

parent b7efa4f5cd

commit ca8e2084e5

1 changed files with 4 additions and 4 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

8

drivers/gpu/drm/amd/display/dc/dsc/rc_calc.c

View file