analyzer: fix ICE when canonicalizing NaN (PR 93451)
PR analyzer/93451 reports an ICE when canonicalizing the constants
in a region_model, with a failed qsort_chk when attempting to sort
the constants within the region_model.
The svalues in the model were:
sv0: {poisoned: uninit}
sv1: {type: ‘double’, ‘0.0’}
sv2: {type: ‘double’, ‘1.0e+0’}
sv3: {type: ‘double’, ‘ Nan’}
The qsort_chk of the 3 constants fails due to tree_cmp using the
LT_EXPR ordering of the REAL_CSTs, which doesn't work for NaN.
This patch adjusts tree_cmp to impose an arbitrary ordering during
canonicalization for UNORDERED_EXPR cases w/o relying on the LT_EXPR
ordering, fixing the ICE.
gcc/analyzer/ChangeLog:
PR analyzer/93451
* region-model.cc (tree_cmp): For the REAL_CST case, impose an
arbitrary order on NaNs relative to other NaNs and to non-NaNs;
const-correctness tweak.
(ana::selftests::build_real_cst_from_string): New function.
(ana::selftests::append_interesting_constants): New function.
(ana::selftests::test_tree_cmp_on_constants): New test.
(ana::selftests::test_canonicalization_4): New test.
(ana::selftests::analyzer_region_model_cc_tests): Call the new
tests.
gcc/testsuite/ChangeLog:
PR analyzer/93451
* gcc.dg/analyzer/torture/pr93451.c: New test.