From 7057de5c749059581578aab763a12f0a9b1c5419 Mon Sep 17 00:00:00 2001 From: Luke Kenneth Casson Leighton Date: Tue, 4 Jun 2024 10:14:18 +0100 Subject: [PATCH] https://bugs.libre-soc.org/show_bug.cgi?id=985 write-up for Test API realised that whilst implementation is not possible, documentation of the API is --- openpower/test_api.mdwn | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 openpower/test_api.mdwn diff --git a/openpower/test_api.mdwn b/openpower/test_api.mdwn new file mode 100644 index 000000000..24d8b345e --- /dev/null +++ b/openpower/test_api.mdwn @@ -0,0 +1,36 @@ +# Libre-SOC Test API + +A problem with complex specifications is ensuring Compliance. +Whilst having Compliance Documentation is obviously +critical (stating in writing +what results are expected when an instruction is executed) +actually *checking* that the results are as expected is both +tedious, protracted and required to be meticulous to such +an extreme extent that to expect it to be carried out manually +is absurd. +The sensible thing to do is to *automate* the process of testing +Compliance, with a Test Suite. + +However even if a Test Suite is available, it firstly may be +specific to one implementation, or only be installable on one +type of system, or only work under other certain very specific +circumstances (such as requiring a bare metal machine but not +function fully under a Virtual Machine, or vice-versa). + +It is clearly of benefit to make it easy to run a +Compliance Test Suite for a given Instruction Set Architecture, +but more than that it is important to make it easy to run +against *multiple* disparate systems, all of which implement +that same ISA. + +Thus we have some requirements: in Software Engineering terms +we have a Requirements Specification. The Test API must: + +1. be comprehensive and thorough +2. run a full suite of tests sufficient to confirm that an implementation + is 100% Compliant with every instruction to be implemented +3. be generic (runnable or adaptable to run on multiple systems) +4. be **open** so as to encourage wider adoption, increasing + the probability of its use and thus avoiding catastrophic + implementation mistakes. + -- 2.30.2