analyzer: fix qsort issue with array_region keys (PR 93352)
PR analyzer/93352 reports a qsort failure
"comparator not anti-symmetric: -
2147483648, -
2147483648)"
within the analyzer on code involving an array access of [0x7fffffff + 1].
The issue is that array_region (which uses int for keys into known
values in the array) uses subtraction to implement int_cmp for sorting
the keys, which isn't going to work for boundary values.
Potentially a wider type should be used, but for now this patch fixes
the ICE by using explicit comparisons rather than subtraction to
implement the qsort callback.
gcc/analyzer/ChangeLog:
PR analyzer/93352
* region-model.cc (int_cmp): Rename to...
(array_region::key_cmp): ...this, using key_t rather than int.
Rewrite in terms of comparisons rather than subtraction to
ensure qsort is anti-symmetric when handling extreme values.
(array_region::walk_for_canonicalization): Update for above
renaming.
* region-model.h (array_region::key_cmp): New decl.
gcc/testsuite/ChangeLog:
PR analyzer/93352
* gcc.dg/analyzer/pr93352.c: New test.