mesa: Fix _mesa_float_to_unorm() on 32-bit systems.
authorKenneth Graunke <kenneth@whitecape.org>
Fri, 23 Aug 2019 18:10:30 +0000 (11:10 -0700)
committerKenneth Graunke <kenneth@whitecape.org>
Tue, 27 Aug 2019 23:57:02 +0000 (23:57 +0000)
commite18cd5452aa4434fb22105eb939843381771b91c
tree96b3a142095370074a67a622c7b416cddf6a80a1
parentb59914e179a9e5930af37e7f7c0d8eafd682caff
mesa: Fix _mesa_float_to_unorm() on 32-bit systems.

This fixes the following CTS test on 32-bit systems:
GTF-GL46.gtf30.GL3Tests.packed_depth_stencil.packed_depth_stencil_init

It does glGetTexImage of a 16-bit SNORM image, requesting 32-bit UNORM
data.  In get_tex_rgba_uncompressed, we round trip through float to
handle image transfer ops for clamping.  _mesa_format_convert does:

   _mesa_float_to_unorm(0.571428597f, 32)

which translated to:

   _mesa_lroundevenf(0.571428597f * 0xffffffffu)

which produced different results on 64-bit and 32-bit systems:

   64-bit: result = 0x92492500
   32-bit: result = 0x80000000

This is because the size of "long" varies between the two systems, and
0x92492500 is too large to fit in a signed 32-bit integer.  To fix this,
we switch to the new _mesa_i64roundevenf function which always does the
64-bit operation.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=104395
Fixes: 594fc0f8595 ("mesa: Replace F_TO_I() with _mesa_lroundevenf().")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
src/mesa/main/format_utils.h