ext: Fix undefined-behavior bug in bitshift

If a small number or zero is passed in, fp64_exp could be very
negative (-1000 for example). The intent of the line is to evaluate to
zero in these cases, but what it actually did was bitshift right by
1000, which is undefined behavior (according to ubsan) that so happens
to result in 0 on GCC/most architectures. This commit changes the code
to check for cases where the bitshift is larger than the width of the
integer.

Change-Id: I8de4bd8ad170f0321d54689460de449b7f8fb60a
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/21859
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Tested-by: kokoro <noreply+kokoro@google.com>

diff --git a/ext/fputils/fp80.c b/ext/fputils/fp80.c
index 6ba89048010889c3cf21e8b391cfb5f8c545c0e6..05acfd91aa65e0898f80c0966c69d7e35a8ce0b5 100644 (file)
--- a/ext/fputils/fp80.c
+++ b/ext/fputils/fp80.c
@@ -162,7 +162,10 @@ fp80_cvtfp64(fp80_t fp80)
              * as normals */
             return build_fp64(sign, fp64_frac, fp64_exp);
         } else if (fp64_exp <= 0) {
-            uint64_t fp64_denormal_frac = fp64_frac >> (-fp64_exp);
+            uint64_t fp64_denormal_frac = -64 < fp64_exp
+                // -64 < fp_exp <= 0, so safe to bitshift by -fp_exp
+                ? fp64_frac >> (-fp64_exp)
+                : 0;
             /* Generate a denormal or zero */
             return build_fp64(sign, fp64_denormal_frac, 0);
         } else {