mesa: Skip 3-byte array formats in _mesa_array_format_flip_channels
authorMichel Dänzer <mdaenzer@redhat.com>
Fri, 24 Apr 2020 09:40:14 +0000 (11:40 +0200)
committerMarge Bot <eric+marge@anholt.net>
Mon, 27 Apr 2020 14:14:00 +0000 (14:14 +0000)
Byte swapping makes no sense for 3-byte formats: Swapping the order of 2
or 4 bytes at a time would inevitably result in bytes getting mixed up
between neighbouring pixels.

Fixes crash with a debugging build on a big endian machine due hitting
the unreachable() at the end of the function.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2665
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4735>

src/mesa/main/formats.c

index 114ea5218beb1a2472fc2c65fc9eb2e550815ecd..0bca52fde50c697eb3728cda42e6e46f0aa95d38 100644 (file)
@@ -404,7 +404,7 @@ _mesa_array_format_flip_channels(mesa_array_format format)
    num_channels = _mesa_array_format_get_num_channels(format);
    _mesa_array_format_get_swizzle(format, swizzle);
 
-   if (num_channels == 1)
+   if (num_channels == 1 || num_channels == 3)
       return format;
 
    if (num_channels == 2) {