From 252c0ed935483c21626d24531919436c2d41bb91 Mon Sep 17 00:00:00 2001 From: Luke Kenneth Casson Leighton Date: Tue, 4 Jun 2024 10:40:43 +0100 Subject: [PATCH] continue API doc --- openpower/test_api.mdwn | 57 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/openpower/test_api.mdwn b/openpower/test_api.mdwn index 24d8b345e..734b946a2 100644 --- a/openpower/test_api.mdwn +++ b/openpower/test_api.mdwn @@ -34,3 +34,60 @@ we have a Requirements Specification. The Test API must: the probability of its use and thus avoiding catastrophic implementation mistakes. +The beginnings of this API came out of a need to "bootstrap" +the Libre-SOC implementation of: + +* the Machine-readable Specification - semi-automated conversion + of the Power ISA 3.0 PDF pseudo-code to an actual executable + language - which produced a slow but functional python-based + Simulator (ineptly named ISACaller) +* the nmigen-based hardware implementation (several variants) + again ineptly named TestIssuer +* QEMU + +The reason chosen right at the start of the project for including +QEMU in the list is because it is a functional bare-metal "executor" of +Power ISA instructions, known and confirmed (for the most part) to +be accurate in its implementation (caveat: the Floating-Point +execution is in no way accurate, relying on the *underlying* +Host Operating System and Hardware. Therefore if executed on +a PowerPC system the FP guest execution produces correct answers +but if executed on an Intel AMD or ARM system the guest produces +*wrong answers* in certain cases). + +As the test suite grew (list of "known-good answers to certain instructions" +grew) this list became "expected" results, and in and of themselves +became an important integral part of the API. Initial exploration +was to simply bootstrap against *other implementations* - run a +test against QEMU, then run the exact same instruction(s) against +ISACaller, and if there are discrepancies find out why and fix them. + +However when errors in *QEMU* were discovered, this was the point +where it was realised that "expected results" needed to be meticulously +and laboriously created. The context here : the official IBM / OpenPOWER +ISA Compliance Test Suite is in a **non-machine-readable** format +that would take months to massage into a machine-readable form. +It is deemed that this is too risky to do because it should be +obvious that simple "transcription" errors from manual typing +or mis-reading could be catastrophic. Therefore the Official +Compliance Test Suite is *excluded* until such time as IBM / OpenPOWER +releases a machine-readable Compliance Test Suite. + +At present (time of writing) the Test API can (very slowly) run +tests against QEMU and also run the exact same test(s) against +ISACaller and TestIssuer. Over time a suite of "expected results" +was also extracted from the organically-evolving API into a more +rigorous format, which gives: + +* the starting state of the implementation being tested + (memory contents, register contents, Program Counter) +* the instructions to be executed +* the expected ending-state of both memory and registers, + post-execution. + +The next most logically-obvious thing to do is to extend the API +to allow additional implementations to easily be included, in a +modular extensible fashion, and to formalise and document the API +itself. + + -- 2.30.2