Fix HP/PA GNU/Linux "long double" format
This:
$ ./gdb -ex "set architecture hppa1.0" -ex "set osabi GNU/Linux" -ex "ptype 1.0L"
Shows that HPPA/Linux support for long doubles is broken. It causes
GDB to access memory out of bounds. With Valgrind, we see:
The target architecture is assumed to be hppa1.0
==4371== Invalid write of size 8
==4371== at 0x4C2F21F: memset (vg_replace_strmem.c:1224)
==4371== by 0x8451C4: convert_doublest_to_floatformat (doublest.c:362)
==4371== by 0x845F86: floatformat_from_doublest (doublest.c:769)
==4371== by 0x84628E: store_typed_floating (doublest.c:873)
==4371== by 0x6A7C3D: value_from_double (value.c:3662)
==4371== by 0x6AA211: evaluate_subexp_standard (eval.c:745)
==4371== by 0x7F306D: evaluate_subexp_c (c-lang.c:716)
==4371== by 0x6A8C6A: evaluate_subexp (eval.c:79)
==4371== by 0x6A8E87: evaluate_type (eval.c:174)
==4371== by 0x817B8D: whatis_exp (typeprint.c:456)
==4371== by 0x817D68: ptype_command (typeprint.c:508)
==4371== by 0x5F2977: do_cfunc (cli-decode.c:105)
==4371== Address 0x8998d18 is 0 bytes after a block of size 8 alloc'd
==4371== at 0x4C2AA98: calloc (vg_replace_malloc.c:711)
==4371== by 0x8732B6: xcalloc (common-utils.c:83)
==4371== by 0x8732F5: xzalloc (common-utils.c:93)
==4371== by 0x6A37AF: allocate_value_contents (value.c:1036)
==4371== by 0x6A37E5: allocate_value (value.c:1047)
==4371== by 0x6A7BEE: value_from_double (value.c:3656)
==4371== by 0x6AA211: evaluate_subexp_standard (eval.c:745)
==4371== by 0x7F306D: evaluate_subexp_c (c-lang.c:716)
==4371== by 0x6A8C6A: evaluate_subexp (eval.c:79)
==4371== by 0x6A8E87: evaluate_type (eval.c:174)
==4371== by 0x817B8D: whatis_exp (typeprint.c:456)
==4371== by 0x817D68: ptype_command (typeprint.c:508)
The trouble is that hppa_linux_init_abi overrides the default
long_double_bit set by the generic hppa-tdep.c:
set_gdbarch_long_double_bit (gdbarch, 128);
set_gdbarch_long_double_format (gdbarch, floatformats_ia64_quad);
with:
/* On hppa-linux, currently, sizeof(long double) == 8. There has been
some discussions to support 128-bit long double, but it requires some
more work in gcc and glibc first. */
set_gdbarch_long_double_bit (gdbarch, 64);
which misses overriding the long_double_format, so we end with a weird
combination of:
set_gdbarch_long_double_bit (gdbarch, 64);
set_gdbarch_long_double_format (gdbarch, floatformats_ia64_quad);
Weird because floatformats_ia64_quad's totalsize is longer than 64-bits.
The floatformat conversion routines use the struct floatformat's
totalsize (in bits) to know how much to copy/convert, thus the buffer
overruns.
gdb/ChangeLog:
2016-03-09 Pedro Alves <palves@redhat.com>
* hppa-linux-tdep.c (hppa_linux_init_abi): Set the long double
format to floatformats_ieee_double.