The Wayland shm formats (almost equivalently, DRM fourcc pixel format
codes) are specified as being little endian, but Pixman's format codes
are generally native-endian. The existing conversions all assume
native=little; this commit introduces format conversions for when
native=big.
Not all DRM/wayland fourcc formats have a big endian pixman_format_code
equivalent, but the most popular (A|X)(RGB|BGR)8888 formats all do.
This fixes a regression from the earlier commit switching from Cairo
to Pixman, where downscaling images inaccurately computed pixel values
in the result.
Now the code uses a convolution filter when downscaling, to get a more
accurate combined color for a downscaled pixel. The 2-lobe Lanczos
filter now used is very close to the Catmull-Rom filter applied by the
CAIRO_FILTER_BEST setting. Downscaling images now runs roughly as
slowly as the Cairo implementation did.
Bilinear filtering is still used for upscaling; while it may not give
results as nice as what Cairo had, rendering quickly becomes even more
important which the screenshot being produced is large.
The previous code using Cairo ultimately used Pixman to draw the
transformed output buffers onto a common image; switching to use
Pixman directly makes it easier to remove the Cairo dependency in the
future, and increases the number of buffer formats which the
rendering code can directly use.
This change also sometimes speeds up the image rendering, by using
Pixman's OP_SRC composition instead of the slower OP_OVER when we
are guaranteed that an output image will not share pixels with another
output's image when drawn on the common image. Cairo sometimes did
this, but not always.
The rendered images may change very slightly due to the new way of
computing transforms, which has slightly different rounding artifacts.
This fixes grim to treat the rotations as counterclockwise and applied
to the surface. This became necessary because wlroots recently corrected
its rotation similarly in https://github.com/swaywm/wlroots/pull/2023.
The format was incorrectly set to 0 instead of what we got from the
compositor. WL_SHM_FORMAT_ARGB8888 was defined to be 0 so the bug
was hidden.
We now choose the Cairo format depending on the Wayland format.
Fixes#17
