fix javascript conversion of huge values
authorJacob Lifshay <programmerjake@gmail.com>
Wed, 15 Mar 2023 08:46:44 +0000 (01:46 -0700)
committerJacob Lifshay <programmerjake@gmail.com>
Wed, 15 Mar 2023 08:46:44 +0000 (01:46 -0700)
the old code would convert 1e100 to -1 when it should have been 0.

openpower/sv/int_fp_mv.mdwn

index 5c0afa2dee68c71e1e8788f72185770c12e7ef71..a1c0981ee9ddeca23846684f6f9c49c2d4a6c47e 100644 (file)
@@ -636,9 +636,11 @@ switch(CVM)
             result <- si64_CONVERT_FROM_BFP(range_max)
     default:  # JavaScript semantics
         # CVM = 6, 7 are illegal instructions
-
+        limit <- bfp_CONVERT_FROM_UI64(0xFFFF_FFFF_FFFF_FFFF)
         if IsInf(rnd) or IsNaN(rnd) then
             result <- [0] * 64
+        else if bfp_COMPARE_GT(bfp_ABSOLUTE(rnd), limit) then
+            result <- [0] * 64
         else
             # this works because the largest type we try to
             # convert from has 53 significand bits, and the