mesa: Use GLdouble for depthMax in final unpack conversions.

The final step of _mesa_unpack_depth_span is to take the temporary
GLfloat depth values and convert them to the desired format.  When
converting to GL_UNSIGNED_INTEGER with depthMax > 0xffffff, we use
double-precision math to avoid overflow and precision problems.

Or at least that's the idea.  Unfortunately

   GLdouble z = depthValues[i] * (GLfloat) depthMax;

actually causes single-precision multiplication, since both operands are
GLfloats.  Casting depthMax to GLdouble causes the scaling to be done
with double-precision math.

Fixes a regression in oglconform's depth-stencil basic.read.ds test
since c60ac7b17993d28af65b04f9bbbf3ee74c35358c, where the expected and
actual values differed slightly.  For example, 0xcfa7a6 vs. 0xcfa7a4.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=49772
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>

diff --git a/src/mesa/main/pack.c b/src/mesa/main/pack.c
index 83192c1574c54bf4b2e4f243278bfee55d3aadae..672a467aa1c3ad25d358c49ad9c8918a0531e525 100644 (file)
--- a/src/mesa/main/pack.c
+++ b/src/mesa/main/pack.c
@@ -4913,7 +4913,7 @@ _mesa_unpack_depth_span( struct gl_context *ctx, GLuint n,
       else {
          /* need to use double precision to prevent overflow problems */
          for (i = 0; i < n; i++) {
-            GLdouble z = depthValues[i] * (GLfloat) depthMax;
+            GLdouble z = depthValues[i] * (GLdouble) depthMax;
             if (z >= (GLdouble) 0xffffffff)
                zValues[i] = 0xffffffff;
             else