mesa: Fix _mesa_float_to_unorm() on 32-bit systems.
authorKenneth Graunke <kenneth@whitecape.org>
Fri, 23 Aug 2019 18:10:30 +0000 (11:10 -0700)
committerKenneth Graunke <kenneth@whitecape.org>
Tue, 27 Aug 2019 23:57:02 +0000 (23:57 +0000)
This fixes the following CTS test on 32-bit systems:
GTF-GL46.gtf30.GL3Tests.packed_depth_stencil.packed_depth_stencil_init

It does glGetTexImage of a 16-bit SNORM image, requesting 32-bit UNORM
data.  In get_tex_rgba_uncompressed, we round trip through float to
handle image transfer ops for clamping.  _mesa_format_convert does:

   _mesa_float_to_unorm(0.571428597f, 32)

which translated to:

   _mesa_lroundevenf(0.571428597f * 0xffffffffu)

which produced different results on 64-bit and 32-bit systems:

   64-bit: result = 0x92492500
   32-bit: result = 0x80000000

This is because the size of "long" varies between the two systems, and
0x92492500 is too large to fit in a signed 32-bit integer.  To fix this,
we switch to the new _mesa_i64roundevenf function which always does the
64-bit operation.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=104395
Fixes: 594fc0f8595 ("mesa: Replace F_TO_I() with _mesa_lroundevenf().")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
src/mesa/main/format_utils.h

index 78365cab5332c8ffae68325117499c69ae6e703f..3a528ceb3db613c820dc85923d45c6115daad08d 100644 (file)
@@ -87,7 +87,7 @@ _mesa_float_to_unorm(float x, unsigned dst_bits)
    else if (x > 1.0f)
       return MAX_UINT(dst_bits);
    else
-      return _mesa_lroundevenf(x * MAX_UINT(dst_bits));
+      return _mesa_i64roundevenf(x * MAX_UINT(dst_bits));
 }
 
 static inline unsigned