libctf: get the encoding of non-ints/fps in the dynamic space right

If you call ctf_type_encoding() on a slice, you are meant to get the
encoding of the slice with the format of the underlying type.  If
you call it on a non-int, non-fp, non-slice, you're meant to get the
error ECTF_INTNOTFP.

None of this was implemented for types in the dynamic space (which, now,
is *all* types in writable containers).  Instead, we were always
returning the encoding as if it were a float, which for all other types
consulted the wrong part of a discriminated union and returned garbage.
(Curiously, existing users were more disturbed by the lack of an error
in the non-int/fp/slice case than they were about getting garbage back.)

libctf/
	* ctf-types.c (ctf_type_encoding): Fix the dynamic case to
	work right for non-int/fps.

diff --git a/libctf/ChangeLog b/libctf/ChangeLog
index 8f41ece32b071c16627b02f12fb7e63d46ba1fdc..74dfbef83738059ee11f201dcb58c45e4e0efed2 100644 (file)
--- a/libctf/ChangeLog
+++ b/libctf/ChangeLog
@@ -1,3 +1,8 @@
+2019-08-09  Nick Alcock  <nick.alcock@oracle.com>
+
+       * ctf-types.c (ctf_type_encoding): Fix the dynamic case to
+       work right for non-int/fps.
+
 2019-08-08  Nick Alcock  <nick.alcock@oracle.com>
 
        * ctf-types.c (ctf_type_name): Don't strlen a potentially-

diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index ec221d734948e12faac78af46abd71c68dfb167e..27cbfb94d00bc748e93375d3c480c1ef80e3eb8b 100644 (file)
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -739,7 +739,27 @@ ctf_type_encoding (ctf_file_t *fp, ctf_id_t type, ctf_encoding_t *ep)
 
   if ((dtd = ctf_dynamic_type (ofp, type)) != NULL)
     {
-      *ep = dtd->dtd_u.dtu_enc;
+      switch (LCTF_INFO_KIND (fp, tp->ctt_info))
+       {
+       case CTF_K_INTEGER:
+       case CTF_K_FLOAT:
+         *ep = dtd->dtd_u.dtu_enc;
+         break;
+       case CTF_K_SLICE:
+         {
+           const ctf_slice_t *slice;
+           ctf_encoding_t underlying_en;
+           slice = &dtd->dtd_u.dtu_slice;
+
+           data = ctf_type_encoding (fp, slice->cts_type, &underlying_en);
+           ep->cte_format = underlying_en.cte_format;
+           ep->cte_offset = slice->cts_offset;
+           ep->cte_bits = slice->cts_bits;
+           break;
+         }
+       default:
+         return (ctf_set_errno (ofp, ECTF_NOTINTFP));
+       }
       return 0;
     }