systemc: Teach verify.py how to expect failing error codes.
authorGabe Black <gabeblack@google.com>
Thu, 30 Aug 2018 08:37:00 +0000 (01:37 -0700)
committerGabe Black <gabeblack@google.com>
Wed, 3 Oct 2018 00:20:25 +0000 (00:20 +0000)
Some tests expect to fail. For those tests (and only those tests) we
need to tell verify.py that it's ok if their exit status isn't 0. Also
if those tests *don't* fail, then that will also be flagged as an
error.

This is done by adding an expected_returncode file into the test's
source directory which holds what the expected return code should be.

Change-Id: I239a28e1d98dd3f76b71028660e492f675a0b3cb
Reviewed-on: https://gem5-review.googlesource.com/c/12446
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>

src/systemc/tests/verify.py

index c2605a2cdabd856c37da4d5327d0ab75c66acb30..57e2c7d2976e7ee740ba265766ffa552670624f1 100755 (executable)
@@ -78,6 +78,9 @@ class Test(object):
     def src_dir(self):
         return os.path.join(script_dir, self.path)
 
+    def expected_returncode_file(self):
+        return os.path.join(self.src_dir(), 'expected_returncode')
+
     def golden_dir(self):
         return os.path.join(self.src_dir(), 'golden')
 
@@ -383,11 +386,19 @@ class VerifyPhase(TestPhaseBase):
             with open(test.returncode_file()) as rc:
                 returncode = int(rc.read())
 
+            expected_returncode = 0
+            if os.path.exists(test.expected_returncode_file()):
+                with open(test.expected_returncode_file()) as erc:
+                    expected_returncode = int(erc.read())
+
             if returncode == 124:
                 self.failed(test, 'time out')
                 continue
-            elif returncode != 0:
-                self.failed(test, 'abort')
+            elif returncode != expected_returncode:
+                if expected_returncode == 0:
+                    self.failed(test, 'abort')
+                else:
+                    self.failed(test, 'missed abort')
                 continue
 
             out_dir = test.m5out_dir()