From: Gabe Black <gabeblack@google.com>
Date: Sun, 8 Nov 2020 15:17:04 +0000 (-0800)
Subject: scons: Convert ProtoBuf to use a scons Builder and Scanner.
X-Git-Tag: develop-gem5-snapshot~472
X-Git-Url: https://git.libre-soc.org/?a=commitdiff_plain;h=6bbdfa370c150b25a711e7dcf17bab8bf7ceba25;p=gem5.git

scons: Convert ProtoBuf to use a scons Builder and Scanner.

There are several benefits to using a Builder. First, the action we're
executing is shared between all uses of the Builder. The number of
times this particular builder is called is small, but it should still
be a little more efficient.

Second, we can use SCons's emitter mechanism to generate the .pb.cc and
.pb.h target files in a little more general way.

Also, this change adds a Scanner for .proto files which will scan them
for imports and let SCons manage those implicit dependencies properly.
The scanner is a bit simplistic as described in a comment in the
source, but should work pretty well in practice with reasonably
formatted files, and in particular some files I'm working with that
include imports.

Change-Id: Iaf2498e61133d6f713d6ccaf199422b882c5894f
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/37276
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Maintainer: Gabe Black <gabe.black@gmail.com>
Tested-by: kokoro <noreply+kokoro@google.com>
---

diff --git a/src/SConscript b/src/SConscript
index 9ac20eb5d..12f7abf2e 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -65,7 +65,7 @@ Export('env')
 
 build_env = [(opt, env[opt]) for opt in export_vars]
 
-from m5.util import code_formatter, compareVersions
+from m5.util import code_formatter, compareVersions, readCommand
 
 ########################################################################
 # Code for adding source files of various types
@@ -386,6 +386,34 @@ class SimObject(PySource):
 
         bisect.insort_right(SimObject.modnames, self.modname)
 
+
+# This regular expression is simplistic and assumes that the import takes up
+# the entire line, doesn't have the keyword "public", uses double quotes, has
+# no whitespace at the end before or after the ;, and is all on one line. This
+# should still cover most cases, and a completely accurate scanner would be
+# MUCH more complex.
+protoc_import_re = re.compile(r'^import\s+\"(.*\.proto)\"\;$', re.M)
+
+def protoc_scanner(node, env, path):
+    deps = []
+    for imp in protoc_import_re.findall(node.get_text_contents()):
+        deps.append(Dir(env['BUILDDIR']).File(imp))
+    return deps
+
+env.Append(SCANNERS=Scanner(function=protoc_scanner, skeys=['.proto']))
+
+def protoc_emitter(target, source, env):
+    root, ext = os.path.splitext(source[0].get_abspath())
+    return [root + '.pb.cc', root + '.pb.h'], source
+
+env.Append(BUILDERS={'ProtoBufCC' : Builder(
+            action=MakeAction('${PROTOC} --cpp_out ${BUILDDIR} '
+                              '--proto_path ${BUILDDIR} '
+                              '${SOURCE.get_abspath()}',
+                              Transform("PROTOC")),
+            emitter=protoc_emitter
+        )})
+
 class ProtoBuf(SourceFile):
     '''Add a Protocol Buffer to build'''
 
@@ -400,20 +428,7 @@ class ProtoBuf(SourceFile):
         modname,ext = self.extname
         assert ext == 'proto'
 
-        # Currently, we stick to generating the C++ headers, so we
-        # only need to track the source and header.
-        self.cc_file = self.tnode.dir.File(modname + '.pb.cc')
-        self.hh_file = self.tnode.dir.File(modname + '.pb.h')
-
-        # Use both the source and header as the target, and the .proto
-        # file as the source. When executing the protoc compiler, also
-        # specify the proto_path to avoid having the generated files
-        # include the path.
-        env.Command([self.cc_file, self.hh_file], self.tnode,
-                    MakeAction('${PROTOC} --cpp_out ${BUILDDIR} '
-                               '--proto_path ${BUILDDIR} '
-                               '${SOURCE.get_abspath()}',
-                               Transform("PROTOC")))
+        self.cc_file, self.hh_file = env.ProtoBufCC(source=source)
 
         # Add the C++ source file
         Source(self.cc_file, tags=self.tags,