From faa092419bf07a5dcee136806351dc0e35ed253e Mon Sep 17 00:00:00 2001
From: lkcl <lkcl@web>
Date: Fri, 12 Aug 2022 03:01:40 +0100
Subject: [PATCH]

---
 openpower/sv/ldst.mdwn | 201 +++++++++++++++++++++--------------------
 1 file changed, 105 insertions(+), 96 deletions(-)

diff --git a/openpower/sv/ldst.mdwn b/openpower/sv/ldst.mdwn
index 1cfb037c8..919bc5714 100644
--- a/openpower/sv/ldst.mdwn
+++ b/openpower/sv/ldst.mdwn
@@ -49,99 +49,6 @@ as well as Element-width overrides and Twin-Predication.
 *Despite being constructed from Scalar LD/ST none of these Modes
 exist or make sense in any Scalar ISA. They **only** exist in Vector ISAs*
 
-# Vectorisation of Scalar Power ISA v3.0B
-
-OpenPOWER Load/Store operations may be seen from [[isa/fixedload]] and
-[[isa/fixedstore]] pseudocode to be of the form:
-
-    lbux RT, RA, RB
-    EA <- (RA) + (RB)
-    RT <- MEM(EA)
-
-and for immediate variants:
-
-    lb RT,D(RA)
-    EA <- RA + EXTS(D)
-    RT <- MEM(EA)
-
-Thus in the first example, the source registers may each be independently
-marked as scalar or vector, and likewise the destination; in the second
-example only the one source and one dest may be marked as scalar or
-vector.
-
-Thus we can see that Vector Indexed may be covered, and, as demonstrated
-with the pseudocode below, the immediate can be used to give unit stride or element stride.  With there being no way to tell which from the OpenPOWER v3.0B Scalar opcode alone, the choice is provided instead by the SV Context.
-
-    # LD not VLD!  format - ldop RT, immed(RA)
-    # op_width: lb=1, lh=2, lw=4, ld=8
-    op_load(RT, RA, op_width, immed, svctx, RAupdate):
-      ps = get_pred_val(FALSE, RA); # predication on src
-      pd = get_pred_val(FALSE, RT); # ... AND on dest
-      for (i=0, j=0, u=0; i < VL && j < VL;):
-        # skip nonpredicates elements
-        if (RA.isvec) while (!(ps & 1<<i)) i++;
-        if (RAupdate.isvec) while (!(ps & 1<<u)) u++;
-        if (RT.isvec) while (!(pd & 1<<j)) j++;
-        if svctx.ldstmode == elementstride:
-          # element stride mode
-          srcbase = ireg[RA]
-          offs = i * immed              # j*immed for a ST
-        elif svctx.ldstmode == unitstride:
-          # unit stride mode
-          srcbase = ireg[RA]
-          offs = immed + (i * op_width) # j*op_width for ST
-        elif RA.isvec:
-          # quirky Vector indexed mode but with an immediate
-          srcbase = ireg[RA+i]
-          offs = immed;
-        else
-          # standard scalar mode (but predicated)
-          # no stride multiplier means VSPLAT mode
-          srcbase = ireg[RA]
-          offs = immed
-
-        # compute EA
-        EA = srcbase + offs
-        # update RA?
-        if RAupdate: ireg[RAupdate+u] = EA;
-        # load from memory
-        ireg[RT+j] <= MEM[EA];
-        if (!RT.isvec)
-            break # destination scalar, end now
-        if (RA.isvec) i++;
-        if (RAupdate.isvec) u++;
-        if (RT.isvec) j++;
-
-Indexed LD is:
- 
-    # format: ldop RT, RA, RB
-    function op_ldx(RT, RA, RB, RAupdate=False) # LD not VLD!
-      ps = get_pred_val(FALSE, RA); # predication on src
-      pd = get_pred_val(FALSE, RT); # ... AND on dest
-      for (i=0, j=0, k=0, u=0; i < VL && j < VL && k < VL):
-        # skip nonpredicated RA, RB and RT
-        if (RA.isvec) while (!(ps & 1<<i)) i++;
-        if (RAupdate.isvec) while (!(ps & 1<<u)) u++;
-        if (RB.isvec) while (!(ps & 1<<k)) k++;
-        if (RT.isvec) while (!(pd & 1<<j)) j++;
-        if svctx.ldstmode == elementstride:
-            EA = ireg[RA] + ireg[RB]*j   # register-strided
-        else
-            EA = ireg[RA+i] + ireg[RB+k] # indexed address
-        if RAupdate: ireg[RAupdate+u] = EA
-        ireg[RT+j] <= MEM[EA];
-        if (!RT.isvec)
-            break # destination scalar, end immediately
-        if svctx.ldstmode != elementstride:
-            if (!RA.isvec && !RB.isvec)
-                break # scalar-scalar
-        if (RA.isvec) i++;
-        if (RAupdate.isvec) u++;
-        if (RB.isvec) k++;
-        if (RT.isvec) j++;
-
-Note in both cases that [[sv/svp64]] allows RA-as-a-dest in "update" mode (`ldux`) to be effectively a *completely different* register from RA-as-a-source.  This because there is room in svp64 to extend RA-as-src as well as RA-as-dest, both independently as scalar or vector *and* independently extending their range.
-
 # Determining the LD/ST Modes
 
 A minor complication (caused by the retro-fitting of modern Vector
@@ -160,6 +67,12 @@ an alternative table meaning for [[sv/svp64]] mode.  The following modes make se
 * fail-first (where Vector Indexed is banned)
 * Signed Effective Address computation (Vector Indexed only)
 
+More than that however it is necessary to fit the usual Vector ISA
+capabilities onto both Power ISA LD/ST with immediate and to
+LD/ST Indexed. They present subtly different Mode tables.
+
+**LD/ST immediate**
+
 The table for [[sv/svp64]] for `immed(RA)` is:
 
 | 0-1 |  2  |  3   4  |  description               |
@@ -206,6 +119,8 @@ destination.
 
 Note that there are no immediate versions of cache-inhibited LD/ST.
 
+**LD/ST Indexed**
+
 The modes for `RA+RB` indexed version are slightly different:
 
 | 0-1 |  2  |  3   4  |  description              |
@@ -244,13 +159,107 @@ Note that cache-inhibited LD/ST (`ldcix`) when VSPLAT is activated will perform
 If a genuine cache-inhibited LD-VSPLAT is required then a *scalar*
 cache-inhibited LD should be performed, followed by a VSPLAT-augmented mv.
 
-## LD/ST ffirst
+# Vectorisation of Scalar Power ISA v3.0B
+
+OpenPOWER Load/Store operations may be seen from [[isa/fixedload]] and
+[[isa/fixedstore]] pseudocode to be of the form:
+
+    lbux RT, RA, RB
+    EA <- (RA) + (RB)
+    RT <- MEM(EA)
+
+and for immediate variants:
+
+    lb RT,D(RA)
+    EA <- RA + EXTS(D)
+    RT <- MEM(EA)
+
+Thus in the first example, the source registers may each be independently
+marked as scalar or vector, and likewise the destination; in the second
+example only the one source and one dest may be marked as scalar or
+vector.
+
+Thus we can see that Vector Indexed may be covered, and, as demonstrated
+with the pseudocode below, the immediate can be used to give unit stride or element stride.  With there being no way to tell which from the OpenPOWER v3.0B Scalar opcode alone, the choice is provided instead by the SV Context.
+
+    # LD not VLD!  format - ldop RT, immed(RA)
+    # op_width: lb=1, lh=2, lw=4, ld=8
+    op_load(RT, RA, op_width, immed, svctx, RAupdate):
+      ps = get_pred_val(FALSE, RA); # predication on src
+      pd = get_pred_val(FALSE, RT); # ... AND on dest
+      for (i=0, j=0, u=0; i < VL && j < VL;):
+        # skip nonpredicates elements
+        if (RA.isvec) while (!(ps & 1<<i)) i++;
+        if (RAupdate.isvec) while (!(ps & 1<<u)) u++;
+        if (RT.isvec) while (!(pd & 1<<j)) j++;
+        if svctx.ldstmode == elementstride:
+          # element stride mode
+          srcbase = ireg[RA]
+          offs = i * immed              # j*immed for a ST
+        elif svctx.ldstmode == unitstride:
+          # unit stride mode
+          srcbase = ireg[RA]
+          offs = immed + (i * op_width) # j*op_width for ST
+        elif RA.isvec:
+          # quirky Vector indexed mode but with an immediate
+          srcbase = ireg[RA+i]
+          offs = immed;
+        else
+          # standard scalar mode (but predicated)
+          # no stride multiplier means VSPLAT mode
+          srcbase = ireg[RA]
+          offs = immed
+
+        # compute EA
+        EA = srcbase + offs
+        # update RA?
+        if RAupdate: ireg[RAupdate+u] = EA;
+        # load from memory
+        ireg[RT+j] <= MEM[EA];
+        if (!RT.isvec)
+            break # destination scalar, end now
+        if (RA.isvec) i++;
+        if (RAupdate.isvec) u++;
+        if (RT.isvec) j++;
+
+Indexed LD is:
+ 
+    # format: ldop RT, RA, RB
+    function op_ldx(RT, RA, RB, RAupdate=False) # LD not VLD!
+      ps = get_pred_val(FALSE, RA); # predication on src
+      pd = get_pred_val(FALSE, RT); # ... AND on dest
+      for (i=0, j=0, k=0, u=0; i < VL && j < VL && k < VL):
+        # skip nonpredicated RA, RB and RT
+        if (RA.isvec) while (!(ps & 1<<i)) i++;
+        if (RAupdate.isvec) while (!(ps & 1<<u)) u++;
+        if (RB.isvec) while (!(ps & 1<<k)) k++;
+        if (RT.isvec) while (!(pd & 1<<j)) j++;
+        if svctx.ldstmode == elementstride:
+            EA = ireg[RA] + ireg[RB]*j   # register-strided
+        else
+            EA = ireg[RA+i] + ireg[RB+k] # indexed address
+        if RAupdate: ireg[RAupdate+u] = EA
+        ireg[RT+j] <= MEM[EA];
+        if (!RT.isvec)
+            break # destination scalar, end immediately
+        if svctx.ldstmode != elementstride:
+            if (!RA.isvec && !RB.isvec)
+                break # scalar-scalar
+        if (RA.isvec) i++;
+        if (RAupdate.isvec) u++;
+        if (RB.isvec) k++;
+        if (RT.isvec) j++;
+
+Note in both cases that [[sv/svp64]] allows RA-as-a-dest in "update" mode (`ldux`) to be effectively a *completely different* register from RA-as-a-source.  This because there is room in svp64 to extend RA-as-src as well as RA-as-dest, both independently as scalar or vector *and* independently extending their range.
+
+# LD/ST ffirst
 
 LD/ST ffirst treats the first LD/ST in a vector (element 0) as an
-ordinary one.  Exceptions occur "as normal".  However for elements 1
+ordinary one.  Exceptions, if any are needed occur "as normal" exactly as
+they would on any Scalar v3.0 Power ISA LD/ST. However for elements 1
 and above, if an exception would occur, then VL is **truncated** to the
 previous element: the exception is **not** then raised because the
-LD/ST was effectively speculative.
+LD/ST that would otherwise have caused an exception is *required* to be cancelled.
 
 ffirst LD/ST to multiple pages via a Vectorised Index base is considered a security risk due to the abuse of probing multiple pages in rapid succession and getting feedback on which pages would fail.  Therefore Vector Indexed LD/ST is prohibited entirely, and the Mode bit instead used for element-strided LD/ST.  See <https://bugs.libre-soc.org/show_bug.cgi?id=561>
 
-- 
2.30.2