python: Improve how templated SimObject classes are handled.

When setting up a SimObject's Param structure, gem5 will autogenerate
a header file which attempts to declare the SimObject's C++ type. It
has had at least some level of sophistication there where it would
pull off the namespaces ahead of the class name and handle them
properly, but it didn't know how to handle templates.

This change improves that handling in two ways. First, it adds a new
magical SimObject attribute called 'cxx_template_params' which is used
to specify what the template parameters are as a list. For instance, if
your SimObject was a template which took an integer constant as its
first parameter and a type as its second, this attribute could look
like the following:

cxx_template_params = [ 'int FOO', 'class Bar' ]

Importantly, if there are any default values for these template
parameters, they should *not* be included here, they should be
specified where the class is later defined.

The second new mechanism is to add an internal CxxClass in the
SimObject.cxx_param_decl method. This class accepts the class signature
in the cxx_class attribute and the cxx_template_params and does two
things. First, it strips off namespaces like in the old implementation.
Second, it extracts and processes any template arguments attached to
the class. If these are constants (as determined by the contents of
cxx_template_params), then they are stored verbatim. If they're types,
then they're recursively expanded into a CxxClass and stored that way.
Note that these are the *values* of the template arguments, where as
cxx_template_params lists the *types* and *names* of those arguments.
In our earlier example, if cxx_class was:

cxx_class = 'CoolClasses::ClassName<12, Fruit::Apple>'

Then CxxClass would extract the namespace 'CoolClasses', the class
name 'ClassName', the argument '12', and the argument 'Fruit::Apple'.
That second argument would be expanded into a CxxClass with the
namespace 'Fruit' and the class name 'Apple'.

Importantly here, because there were no default arguments given in
cxx_template_params, all "hidden" arguments which would fall through
to their defaults need to be fully specified in cxx_class.

The CxxClass has a method called declare() which uses the information
extracted earlier to output all of the "stuff" necessary for declaring
the given class, including opening any containing namespaces and
putting template<...> ahead of the actual class declaration with the
template parameters specified.

If any of the template arguments are themselves CxxClass instances,
then they'll be recursively declared immediately before the current
class is.

An alternative solution to this problem might be to include the header
file which actually defines the cxx_class type to avoid having to
come up with a declaration. Unfortunately this doesn't work since it
can set up include loops where the SimObject C++ header file includes
the param header to get access to the Param type, but that includes
the C++ header to get access to the SimObject type.

This also makes it harder for SimObjects to refer to each other, since
they rely on the declaration in the params header files when declaring
a member pointer to that type in their own Param structures.

Change-Id: I68cfc36ddff6d789eb4cdef5178c4619ac2cc8b1
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/17228
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Gabe Black <gabeblack@google.com>

diff --git a/src/python/m5/SimObject.py b/src/python/m5/SimObject.py
index 97f6847396a7d1ed7d3472e390594c8226abe712..b74e93a87ffad2f46a33f44e44695e5a22558fdc 100644 (file)
--- a/src/python/m5/SimObject.py
+++ b/src/python/m5/SimObject.py
@@ -415,6 +415,7 @@ class MetaSimObject(type):
         'cxx_extra_bases' : list,
         'cxx_exports' : list,
         'cxx_param_exports' : list,
+        'cxx_template_params' : list,
     }
     # Attributes that can be set any time
     keywords = { 'check' : FunctionType }
@@ -454,6 +455,8 @@ class MetaSimObject(type):
             value_dict['cxx_exports'] += cxx_exports
         if 'cxx_param_exports' not in value_dict:
             value_dict['cxx_param_exports'] = []
+        if 'cxx_template_params' not in value_dict:
+            value_dict['cxx_template_params'] = []
         cls_dict['_value_dict'] = value_dict
         cls = super(MetaSimObject, mcls).__new__(mcls, name, bases, cls_dict)
         if 'type' in value_dict:
@@ -773,6 +776,7 @@ module_init(py::module &m_internal)
         code('static EmbeddedPyBind embed_obj("${0}", module_init, "${1}");',
              cls, cls._base.type if cls._base else "")
 
+    _warned_about_nested_templates = False
 
     # Generate the C++ declaration (.hh file) for this SimObject's
     # param struct.  Called from src/SConscript.
@@ -790,7 +794,78 @@ module_init(py::module &m_internal)
             print(params)
             raise
 
-        class_path = cls._value_dict['cxx_class'].split('::')
+        class CxxClass(object):
+            def __init__(self, sig, template_params=[]):
+                # Split the signature into its constituent parts. This could
+                # potentially be done with regular expressions, but
+                # it's simple enough to pick appart a class signature
+                # manually.
+                parts = sig.split('<', 1)
+                base = parts[0]
+                t_args = []
+                if len(parts) > 1:
+                    # The signature had template arguments.
+                    text = parts[1].rstrip(' \t\n>')
+                    arg = ''
+                    # Keep track of nesting to avoid splitting on ","s embedded
+                    # in the arguments themselves.
+                    depth = 0
+                    for c in text:
+                        if c == '<':
+                            depth = depth + 1
+                            if depth > 0 and not \
+                                    self._warned_about_nested_templates:
+                                self._warned_about_nested_templates = True
+                                print('Nested template argument in cxx_class.'
+                                      ' This feature is largely untested and '
+                                      ' may not work.')
+                        elif c == '>':
+                            depth = depth - 1
+                        elif c == ',' and depth == 0:
+                            t_args.append(arg.strip())
+                            arg = ''
+                        else:
+                            arg = arg + c
+                    if arg:
+                        t_args.append(arg.strip())
+                # Split the non-template part on :: boundaries.
+                class_path = base.split('::')
+
+                # The namespaces are everything except the last part of the
+                # class path.
+                self.namespaces = class_path[:-1]
+                # And the class name is the last part.
+                self.name = class_path[-1]
+
+                self.template_params = template_params
+                self.template_arguments = []
+                # Iterate through the template arguments and their values. This
+                # will likely break if parameter packs are used.
+                for arg, param in zip(t_args, template_params):
+                    type_keys = ('class', 'typename')
+                    # If a parameter is a type, parse it recursively. Otherwise
+                    # assume it's a constant, and store it verbatim.
+                    if any(param.strip().startswith(kw) for kw in type_keys):
+                        self.template_arguments.append(CxxClass(arg))
+                    else:
+                        self.template_arguments.append(arg)
+
+            def declare(self, code):
+                # First declare any template argument types.
+                for arg in self.template_arguments:
+                    if isinstance(arg, CxxClass):
+                        arg.declare(code)
+                # Re-open the target namespace.
+                for ns in self.namespaces:
+                    code('namespace $ns {')
+                # If this is a class template...
+                if self.template_params:
+                    code('template <${{", ".join(self.template_params)}}>')
+                # The actual class declaration.
+                code('class ${{self.name}};')
+                # Close the target namespaces.
+                for ns in reversed(self.namespaces):
+                    code('} // namespace $ns')
 
         code('''\
 #ifndef __PARAMS__${cls}__
@@ -806,14 +881,12 @@ module_init(py::module &m_internal)
         if cls == SimObject:
             code('''#include <string>''')
 
+        cxx_class = CxxClass(cls._value_dict['cxx_class'],
+                             cls._value_dict['cxx_template_params'])
+
         # A forward class declaration is sufficient since we are just
         # declaring a pointer.
-        for ns in class_path[:-1]:
-            code('namespace $ns {')
-        code('class $0;', class_path[-1])
-        for ns in reversed(class_path[:-1]):
-            code('} // namespace $ns')
-        code()
+        cxx_class.declare(code)
 
         for param in params:
             param.cxx_predecls(code)