# SV Load and Store
+<!-- hide -->
Links:
* <https://bugs.libre-soc.org/show_bug.cgi?id=561>
* <https://llvm.org/devmtg/2016-11/Slides/Emerson-ScalableVectorizationinLLVMIR.pdf>
* <https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc#vector-loads-and-stores>
* [[ldst/discussion]]
+<!-- show -->
## Rationale
Vectorised Load and Store also presents an extra dimension (literally)
which creates scenarios unique to Vector applications, that a Scalar (and
-even a SIMD) ISA simply never encounters. SVP64 endeavours to add the
+even a SIMD) ISA simply never encounters: not even the complex Addressing
+Modes of the 68,000 or S/360 resemble Vector Load/Store.
+SVP64 endeavours to add the
modes typically found in *all* Scalable Vector ISAs, without changing the
behaviour of the underlying Base (Scalar) v3.0B operations in any way.
(The sole apparent exception is Post-Increment Mode on LD/ST-update
* **fixed aka "unit" stride** - contiguous sequence with no gaps
* **element strided** - sequential but regularly offset, with gaps
* **vector indexed** - vector of base addresses and vector of offsets
-* **Speculative fail-first** - where it makes sense to do so
+* **Speculative Fault-first** - where it makes sense to do so
* **Structure Packing** - covered in SV by [[sv/remap]] and Pack/Unpack Mode.
*Despite being constructed from Scalar LD/ST none of these Modes exist
-or make sense in any Scalar ISA. They **only** exist in Vector ISAs*
+or make sense in any Scalar ISA. They **only** exist in Vector ISAs
+and are a critical part of its value*.
Also included in SVP64 LD/ST is both signed and unsigned Saturation,
as well as Element-width overrides and Twin-Predication.
-Note also that Indexed [[sv/remap]] mode may be applied to both v3.0
-LD/ST Immediate instructions *and* v3.0 LD/ST Indexed instructions.
+Note also that Indexed [[sv/remap]] mode may be applied to both Scalar
+LD/ST Immediate Defined Words *and* LD/ST Indexed Defined Words.
LD/ST-Indexed should not be conflated with Indexed REMAP mode:
clarification is provided below.
A minor complication (caused by the retro-fitting of modern Vector
features to a Scalar ISA) is that certain features do not exactly make
-sense or are considered a security risk. Fail-first on Vector Indexed
+sense or are considered a security risk. Fault-first on Vector Indexed
would allow attackers to probe large numbers of pages from userspace,
-where strided fail-first (by creating contiguous sequential LDs) does not.
+where strided Fault-first (by creating contiguous sequential LDs) does not.
In addition, reduce mode makes no sense. Realistically we need an
alternative table definition for [[sv/svp64]] `RM.MODE`. The following
modes make sense:
-* saturation
-* predicate-result would be useful but is lower priority than Data-Dependent Fail-First
* simple (no augmentation)
-* fail-first (where Vector Indexed is banned)
+* Fault-first (where Vector Indexed is banned)
+* Data-dependent Fail-First (extremely useful for Linked-List pointer-chasing)
* Signed Effective Address computation (Vector Indexed only)
More than that however it is necessary to fit the usual Vector ISA
* **zz**: both sz and dz are set equal to this flag.
* **inv CR bit** just as in branches (BO) these bits allow testing of
a CR bit and whether it is set (inv=0) or unset (inv=1)
-* **N** sets signed/unsigned saturation.
* **RC1** as if Rc=1, stores CRs *but not the result*
* **SEA** - Signed Effective Address, if enabled performs sign-extension on
registers that have been reduced due to elwidth overrides
| 0 | 1 | 2 | 3 4 | description |
|---|---| --- |---------|--------------------------- |
-| 0 | 0 | 0 | zz els | simple mode |
-| 0 | 0 | 1 | PI LF | post-increment and Fault-First |
-| 1 | 0 | N | zz els | sat mode: N=0/1 u/s |
-|VLi| 1 | inv | CR-bit | Rc=1: ffirst CR sel |
-|VLi| 1 | inv | els RC1 | Rc=0: ffirst z/nonz |
+|els| 0 | PI | zz LF | post-increment and Fault-First |
+|VLi| 1 | inv | CR-bit | ffirst CR sel |
The `els` bit is only relevant when `RA.isvec` is clear: this indicates
whether stride is unit or element:
| 0 | 1 | 2 | 3 4 | description |
|---|---| --- |---------|--------------------------- |
|els| 0 | SEA | dz sz | simple mode |
-|VLi| 1 | inv | CR-bit | Rc=1: ffirst CR sel |
-|VLi| 1 | inv | els RC1 | Rc=0: ffirst z/nonz |
+|VLi| 1 | inv | CR-bit | ffirst CR sel |
Vector Indexed Strided Mode is qualified as follows:
Signed Effective Address computation is only relevant for Vector Indexed
Mode, when elwidth overrides are applied. The source override applies to
RB, and before adding to RA in order to calculate the Effective Address,
-if SEA is set RB is sign-extended from elwidth bits to the full 64 bits.
-For other Modes (ffirst, saturate), all EA computation with elwidth
-overrides is unsigned.
+if SEA is set then RB is sign-extended from elwidth bits to the full 64 bits.
+For other Modes (ffirst), all EA computation with elwidth
+overrides is unsigned. RA is *never* altered (not truncated)
+by element-width overrides.
Note that cache-inhibited LD/ST when VSPLAT is activated will perform
**multiple** LD/ST operations, sequentially. Even with scalar src
Thus it can be seen that the use of Indexed REMAP saves copying
and manual reordering of the Vector of RB offsets.
-## LD/ST ffirst
+## LD/ST ffirst (Fault-First)
LD/ST ffirst treats the first LD/ST in a vector (element 0 if REMAP
-is not active) as an ordinary one, with all behaviour with respect to
+is not active and predication is not applied)
+as an ordinary one, with all behaviour with respect to
Interrupts Exceptions Page Faults Memory Management being identical
in every regard to Scalar v3.0 Power ISA LD/ST. However for elements
1 and above, if an exception would occur, then VL is **truncated**
pages in rapid succession and getting speculative feedback on which
pages would fail. Therefore Vector Indexed LD/ST is prohibited
entirely, and the Mode bit instead used for element-strided LD/ST.
+<!-- hide -->
See <https://bugs.libre-soc.org/show_bug.cgi?id=561>
+<!-- show -->
```
for(i = 0; i < VL; i++)
or beginning of a Virtual Memory page. Likewise, to reduce workloads or
balance resources.
+When Predication is used, the "first" element is considered to be the first
+non-predicated element rather than specifically `srcstep=0`.
+
Vertical-First Mode is slightly strange in that only one element at a time
is ever executed anyway. Given that programmers may legitimately choose
to alter srcstep and dststep in non-sequential order as part of explicit
## Data-Dependent Fail-First (not Fail/Fault-First)
Not to be confused with Fail/Fault First, Data-Fail-First performs an
-additional check on the data into a Condition Register Field and if a test
-on the CR Field fails then VL is truncated and further looping terminates.
+additional check on the data, and if the test
+fails then VL is truncated and further looping terminates.
This is precisely the same as Arithmetic Data-Dependent Fail-First,
-the only difference being that the result comes from the LD/ST.
+the only difference being that the result comes from the LD/ST
+rather than from an Arithmetic operation.
+
+Also a crucial difference between Arithmetic and LD/ST Data-Dependent Fail-First:
+except for Store-Conditional a 4-bit Condition Register Field test is created
+for testing purposes
+*but not stored* (thus there is no RC1 Mode as there is in Arithmetic).
+The reason why a CR Field is not stored is because Load/Store, particularly
+the Update instructions, is already expensive in register terms,
+and adding an extra Vector write would be too costly in hardware.
+
+*Programmer's note: Programmers
+may use Data-Dependent Load with a test to truncate VL, and may then
+follow up with a `sv.cmpi` or other operation. The important aspect is
+that the Vector Load truncated on finding a NULL pointer, for example.*
+
+*Programmer's note: Load-with-Update may be used to update
+the register used in Effective Address computation of th
+next element. This may be used to perform single-linked-list
+walking, where Data-Dependent Fail-First terminates and
+truncates the Vector at the first NULL.*
+
+**Load/Store Data-Dependent Fail-First, VLi=0**
In the case of Store operations there is a quirk when VLi (VL inclusive
is "Valid") is clear. Bear in mind the criteria is that the truncated
Vector of results, when VLi is clear, must all pass the "test", but when
VLi is set the *current failed test* is permitted to be included. Thus,
the actual update (store) to Memory is **not permitted to take place**
-should the test fail. Therefore, on testing the value to be stored,
-and after updating the corresponding CR Field Element, when VLi=0 and
-finding that the test fails the Memory store must **not** occur.
+should the test fail.
-Additionally, when VLi=0 and a test fails then RA does **not** receive a
+Additionally in any Load/Store with Update instruction,
+when VLi=0 and a test fails then RA does **not** receive a
copy of the Effective Address. Hardware implementations with Out-of-Order
Micro-Architectures should use speculative Shadow-Hold and Cancellation
-when the test fails.
+(or other Transactional Rollback mechanism) when the test fails.
+
+* **Load, VLi=0**: perform the Memory Load, do not put the result into the regfile yet (or EA into RA). Test the Loaded data: if fail do not store the Load in the register file (or EA into RA). Otherwise proceed with updating regfiles. VL is truncated to "only elements that passed the test"
+* **Store, VLi=0**: even before the Store takes place, perform the test on the data to *be* stored. If fail do not proceed with the Store at all. VL is truncated to "only elements that passed the test"
+
+**Load/Store Data-Dependent Fail-First, VLi=1**
-By contrast if VLi=1 and the test fails, Store may proceed *and then*
-looping terminates. In this way, when non-Inclusive, the Vector of
-Truncated results contains only Stores that passed the test (and RA=EA
-updates if any), and when Inclusive the Vector of Truncated results
-contains the first-failed data.
+By contrast if VLi=1 and the test fails, the Store may proceed *and then*
+looping terminates. In this way, when Inclusive the Vector of Truncated results
+contains the first-failed data (including RA on Updates)
+
+* **Load, VLi=1**: perform the Memory Load, complete it in full (including EA into RA). Test the Loaded data: if fail then VL is truncated to "elements tested".
+* **Store, VLi=0**: same as Load. Perform the Store in full and after-the-fact carry out the test of the original data requested to be stored. If fail then VL is truncated to "elements tested".
Below is an example of loading the starting addresses of Linked-List
nodes. If VLi=1 it will load the NULL pointer into the Vector of results.
If however VLi=0 it will *exclude* the NULL pointer by truncating VL to
-one Element earlier.
+one Element earlier (only loading non-NULL data into registers).
*Programmer's Note: by also setting the RC1 qualifier as well as setting
VLi=1 it is possible to establish a Predicate Mask such that the first
RA=0 # vec - first one is valid, contains ptr
imm = 8 # offset_of(ptr->next)
for i in range(VL):
+ # this part is the Scalar Defined Word (standard scalar ld operation)
EA = GPR(RA+i) + imm # ptr + offset(next)
data = MEM(EA, 8) # 64-bit address of ptr->next
- GPR(RT+i) = data # happens to be read on next loop!
# was a normal vector-ld up to this point. now the Data-Fail-First
- CR.field(i) = conditions(data)
- if CR.field(i).EQ == testbit: # check if zero
- if VLI then VL = i+1 # update VL, inclusive
- else VL = i # update VL
- break # stop looping
+ cr_test = conditions(data)
+ if Rc=1 or RC1: CR.field(i) = cr_test # only store if Rc=1/RC1
+ action_load = True
+ if cr_test.EQ == testbit: # check if zero
+ if VLI then
+ VL = i+1 # update VL, inclusive
+ else
+ VL = i # update VL, exclusive current
+ action_load = False # current load excluded
+ stop = True # stop looping
+ if action_load:
+ GPR(RT+i) = data # happens to be read on next loop!
+ if stop: break
```
-**Data-Dependent Fault-First on Store-Conditional (Rc=1)**
+**Data-Dependent Fail-First on Store-Conditional (Rc=1)**
There are very few instructions that allow Rc=1 for Load/Store:
one of those is the `stdcx.` and other Atomic Store-Conditional
instructions. With Simple-V being a loop around Scalar instructions
-strictly obeying Scalar Program Order a Fail-First loop on an
-Atomic Store-Conditional will always fail the second and all other
-Store-Conditional instructions in Horizontal-First Mode because
+strictly obeying Scalar Program Order a Horizontal-First Fail-First loop
+on an Atomic Store-Conditional will always fail the second and all other
+Store-Conditional instructions because
Load-Reservation and Store-Conditional are required to be executed
in pairs.
By contrast, in Vertical-First Mode it is in fact possible to issue
the pairs, and consequently allowing Vectorised Data-Dependent Fail-First is
-useful. Care should be taken however when VL is truncated in Vertical-First
-Mode.
+useful.
+
+Programmer's note: Care should be taken when VL is truncated in
+Vertical-First Mode.
+
+**Future potential**
+
+Although Rc=1 on LD/ST is a rare occurrence at present, future versions
+of Power ISA *might* conceivably have Rc=1 LD/ST Scalar instructions, and
+with the SVP64 Vectorisation Prefixing being itself a RISC-paradigm that
+is itself fully-independent of the Scalar Suffix Defined Words, prohibiting
+the possibility of Rc=1 Data-Dependent Mode on future potential LD/ST
+operations is not strategically sound.
## LOAD/STORE Elwidths <a name="elwidth"></a>
but (on Indexed Load) allow srcwidth overrides on RB
* Load at the operation width (lb/lh/lw/ld) as usual
* byte-reversal as usual
-* Non-saturated mode:
- - zero-extension or truncation from operation width to dest elwidth
- - place result in destination at dest elwidth
-* Saturated mode:
- - Sign-extension or truncation from operation width to dest width
- - signed/unsigned saturation down to dest elwidth
+* zero-extension or truncation from operation width to dest elwidth
+* place result in destination at dest elwidth
In order to respect Power v3.0B Scalar behaviour the memory side
is treated effectively as completely separate and distinct from SV
Immediate and Indexed LD/ST, does not have element-width overriding
applied to it.
-Note that predication, predication-zeroing, and other modes except
-saturation have all been removed, for clarity and simplicity:
+Note that predication, predication-zeroing, and other modes
+have all been removed, for clarity and simplicity:
```
# LD not VLD!
# read the underlying memory
memread <= MEM(srcbase + imm_offs, op_width)
- # check saturation.
- if svpctx.saturation_mode:
- # ... saturation adjustment...
- memread = clamp(memread, op_width, svctx.dest_elwidth)
- else:
- # truncate/extend to over-ridden dest width.
- memread = adjust_wid(memread, op_width, svctx.dest_elwidth)
+ # truncate/extend to over-ridden dest width.
+ memread = adjust_wid(memread, op_width, svctx.dest_elwidth)
# takes care of inserting memory-read (now correctly byteswapped)
# into regfile underlying LE-defined order, into the right place
For LD/Indexed, the key is that in the calculation of the Effective Address,
RA has no elwidth override but RB does. Pseudocode below is simplified
-for clarity: predication and all modes except saturation are removed:
+for clarity: predication and all modes are removed:
```
# LD not VLD! ld*rx if brev else ld*
if (bytereverse):
memread = byteswap(memread, op_width)
- if svpctx.saturation_mode:
- # ... saturation adjustment...
- memread = clamp(memread, op_width, svctx.dest_elwidth)
- else:
- # truncate/extend to over-ridden dest width.
- memread = adjust_wid(memread, op_width, svctx.dest_elwidth)
+ # truncate/extend to over-ridden dest width.
+ memread = adjust_wid(memread, op_width, svctx.dest_elwidth)
# takes care of inserting memory-read (now correctly byteswapped)
# into regfile underlying LE-defined order, into the right place
Structure Packing, at the vec2/vec3/vec4 granularity level. Beyond that,
REMAP will need to be used.
+**Parallel Reduction REMAP**
+
+No REMAP Schedule is prohibited in SVP64 because the RISC-paradigm Prefix
+is completely separate from the RISC-paradigm Scalar Defined Words. Although
+obscure there does exist the outside possibility that a potential use for
+Parallel Reduction Schedules on LD/ST would find a use in Computer Science.
+Readers are invited to contact the authors of this document if one is ever
+found.
+
--------
[[!tag standards]]
projects / libreriscv.git / blobdiff