https://bugs.libre-soc.org/show_bug.cgi?id=985 write-up for Test API

realised that whilst implementation is not possible, documentation of the API
is

diff --git a/openpower/test_api.mdwn b/openpower/test_api.mdwn
new file mode 100644 (file)
index 0000000..24d8b34
--- /dev/null
+++ b/openpower/test_api.mdwn
@@ -0,0 +1,36 @@
+# Libre-SOC Test API
+
+A problem with complex specifications is ensuring Compliance.
+Whilst having Compliance Documentation is obviously
+critical (stating in writing
+what results are expected when an instruction is executed)
+actually *checking* that the results are as expected is both
+tedious, protracted and required to be meticulous to such
+an extreme extent that to expect it to be carried out manually
+is absurd.
+The sensible thing to do is to *automate* the process of testing
+Compliance, with a Test Suite.
+
+However even if a Test Suite is available, it firstly may be
+specific to one implementation, or only be installable on one
+type of system, or only work under other certain very specific
+circumstances (such as requiring a bare metal machine but not
+function fully under a Virtual Machine, or vice-versa).
+
+It is clearly of benefit to make it easy to run a
+Compliance Test Suite for a given Instruction Set Architecture,
+but more than that it is important to make it easy to run
+against *multiple* disparate systems, all of which implement
+that same ISA.
+
+Thus we have some requirements: in Software Engineering terms
+we have a Requirements Specification. The Test API must:
+
+1. be comprehensive and thorough
+2. run a full suite of tests sufficient to confirm that an implementation
+ is 100% Compliant with every instruction to be implemented
+3. be generic (runnable or adaptable to run on multiple systems)
+4. be **open** so as to encourage wider adoption, increasing
+ the probability of its use and thus avoiding catastrophic
+ implementation mistakes.
+