(no commit message)

[libreriscv.git] / openpower / sv / svp64_quirks.mdwn

# The Rules

[[!toc]]

SVP64 is designed around fundamental and inviolate RISC principles.
This gives a uniformity and regularity to the ISA, making implementation
straightforward, which was why RISC
as a concept became popular.

1. There are no actual Vector instructions: Scalar instructions
   are the sole exclusive bedrock.
2. No scalar instruction ever deviates in its encoding or meaning
   just because it is prefixed (semantic caveats below)
3. A hardware-level for-loop makes vector elements 100% synonymous
   with scalar instructions (the suffix)

How can a Vector ISA even exist when no actual Vector instructions
are permitted to be added? It comes down to the strict RISC abstraction.
First you start from a **scalar** instruction (32-bit). Second, the
Prefixing is applied *in the abstract* to give the *appearance*
and ultimately the same effect as if an explicit Vector instruction
had also been added.  Looking at the pseudocode of any Vector ISA
(RVV, NEC SX Aurora, Cray)
they always comprise (a) a for-loop around (b) element-based operations.
It is perfectly reasonable and rational to separate (a) from (b)
then find a powerful pre-existing
Supercomputing-class ISA that qualifies for (b).

There are a few exceptional places where these rules get
bent, and others where the rules take some explaining,
and this page tracks them all.

The modification caveat in (2) above semantically
exempts element width overrides,
which still do not actually modify the meaning of the instruction:
an add remains an add, even if its override makes it an 8-bit add rather than
a 64-bit add. Even add-with-carry remains an add-with-carry: it's just
that when elwidth=8 in the Prefix it's an *8-bit* add-with-carry
where the 9th bit becomes Carry-out (not the 65th bit).
In other words, elwidth overrides **definitely** do not fundamentally
alter the actual
Scalar v3.0 ISA encoding itself.  Consequently we can still, in
the strictest semantic sense, not be breaking rule (2).

Likewise, other "modifications" such as saturation or Data-dependent
Fail-First likewise are actually post-augmentation or post-analysis, and do
not fundamentally change an add operation into a subtract
for example, and under absolutely no circumstances do the actual 32-bit
Scalar v3.0 operand field bits change or the number of operands change.

In an early Draft of SVP64,
an experiment was attempted, to modify LD-immediate instructions
to include a
third RC register i.e. reinterpret the normal
v3.0 32-bit instruction as a completely
different encoding if SVP64-prefixed. It did not go well.
The complexity that resulted
in the decode phase was too great. The lesson was learned, the
hard way: it would be infinitely preferable
to add a 32-bit Scalar Load-with-Shift
instruction *first*, which then inherently becomes Vectorised.
Perhaps a future Power ISA spec will have this Load-with-Shift instruction:
both ARM and x86 have it, because it saves greatly on instruction count in
hot-loops.

The other reason for not adding an SVP64-Prefixed instruction without
also having it as a Scalar un-prefixed instruction is that if the
32-bit encoding is ever allocated in a future revision
of the Power ISA
to a completely unrelated operation
then how can a Vectorised version of that new instruction ever be added?
The uniformity and RISC Abstraction is irreparably damaged.
Bottom line here is that the fundamental RISC Principle is strictly adhered
to, even though these are Advanced 64-bit Vector instructions.
Advocates of the RISC Principle will appreciate the uniformity of
SVP64 and the level of systematic abstraction kept between Prefix and Suffix.

# Instruction Groups

The basic principle of SVP64 is the prefix, which contains mode
as well as register augmentation and predicates.  When thinking of
instructions and Vectorising them, it is natural for arithmetic
operations (ADD, OR) to be the first to spring to mind.
Arithmetic instructions have registers, therefore augmentation
applies, end of story, right?

Except, Load and Store deals also with Memory, not just registers.
Power ISA has Condition Register Fields: how can element widths
apply there? And branches: how can you have Saturation on something
that does not return an arithmetic result? In short: there are actually
four different categories (five including those for which Vectorisation
makes no sense at all, such as `sc` or `mtmsr`). The categories are:

* arithmetic/logical including floating-point
* Load/Store
* Condition Register Field operations
* branch

**Arithmetic**

Arithmetic (known as "normal" mode) is where Scalar and Parallel
Reduction can be done: Saturation as well, and two new innovative
modes for Vector ISAs: data-dependent fail-first and predicate result.
Reduction and Saturation are common to see in Vector ISAs: it is just
that they are usually added as explicit instructions,
and NEC SX Aurora has even more iterative instructions. In SVP64 these
concepts are applied in the abstract general form, which takes some
getting used to.

Reduction may, when applied to non-commutative
instructions incorrectly, result in invalid results, but ultimately
it is critical to think in terms of the "rules", that everything is
Scalar instructions in strict Program Order.  Reduction on non-commutative
Scalar Operations is not *prohibited*: the strict Program Order allows
the programmer to think through what would happen and thus potentially
actually come up with legitimate use.

**Branches**

Branch is the one and only place where the Scalar
(non-prefixed) operations differ from the Vector (element)
instructions, as explained in a separate section.
The
RM bits can be used for other purposes because the Arithmetic modes
make no sense at all for a Branch.
Almost the entire
SVP64 RM Field is interpreted differently from other Modes, in
order to support a wide range of parallel boolean condition options
which are expected of a Vector / GPU ISA. These save a considerable
number of instructions in tight inner loop situations.

**CR Field Ops**

Condition Register Fields are 4-bit wide and consequently element-width
overrides make absolutely no sense whatsoever. Therefore the elwidth
override field bits can be used for other purposes when Vectorising
CR Field instructions.  Moreover, Rc=1 is completely invalid for
CR operations such as `crand`: Rc=1 is for arithmetic operations, producing
a "co-result" that goes into CR0 or CR1. Thus, the Arithmetic modes
such as predicate-result make no sense, and neither does Saturation.
All of these differences, which require quite a lot of logical
reasoning and deduction, help explain why there is an entirely different
CR ops Vectorisation Category.

A particularly strange quirk of CR-based Vector Operations is that the
Scalar Power ISA CR Register is 32-bits, but actually comprises eight
CR Fields, CR0-CR7. With each CR Field being four bits (EQ, LT, GT, SO)
this makes up 32 bits, and therefore a CR operand referring to one bit
of the CR will be 5 bits in length (BA, BT).
*However*, some instructions refer
to a *CR Field* (CR0-CR7) and consequently these operands
(BF, BFA etc) are only 3-bits.

(*It helps here to think of the top 3 bits of BA as referring
to a CR Field, like BFA does, and the bottom 2 bits of BA
referring to
EQ/LT/GT/SO within that Field*)

With SVP64 extending the number of CR *Fields* to 128, the number of
32-bit CR *Registers* extends to 16, in order to hold all 128 CR *Fields*
(8 per CR Register). Then, it gets even more strange, when it comes
to Vectorisation, which applies to the CR Field *numbers*.  The
hardware-for-loop for Rc=1 for example starts at CR0 for element 0,
and moves to CR1 for element 1, and so on.  The reason here is quite
simple: each element result has to have its own CR Field co-result.

In other words, the
element is the 4-bit CR *Field*, not the bits *of* the 32-bit
CR Register, and not the CR *Register* (of which there are now 16).
All quite logical, but a little mind-bending.

**Load/Store**

LOAD/STORE is another area that has different needs: this time it is
down to limitations in Scalar LD/ST. Vector ISAs have Load/Store modes
which simply make no sense in a RISC Scalar ISA: element-stride and
unit-stride and the entire concept of a stride itself (a spacing
between elements) has no place at all in a Scalar ISA. The problems
come when trying to *retrofit* the concept of "Vector Elements" onto
a Scalar ISA. Consequently it required a couple of bits (Modes) in the SVP64
RM Prefix to convey the stride mode, changing the Effective Address
computation as a result. Interestingly, worth noting for Hardware
designers: it did turn out to be possible to perform pre-multiplication
of the D/DS Immediate by the stride amount, making it possible to avoid
actually modifying the LD/ST Pipeline itself.

Other areas where LD/ST went quirky: element-width overrides especially
when combined with Saturation, given that LD/ST operations have byte,
halfword, word, dword and quad variants. The interaction between these
widths as part of the actual operation, and the source and destination
elwidth overrides, was particularly obtuse and hard to derive: some care
and attention is advised, here, when reading the specification,
especially on arithmetic loads (lbarx, lharx etc.)

**Non-vectorised**

The concept of a Vectorised halt (`attn`) makes no sense. There are never
going to be a Vector of global MSRs (Machine Status Register). `mtcr`
on the other hand is a grey area: `mtspr` is clearly Vectoriseable.
Even `td` and `tdi` makes a strange type of sense to permit it to be
Vectorised, because a sequence of comparisons could be Vectorised.
Vectorised System Calls (`sc`) or `tlbie` and other Cache or Virtual
Nemory Management
instructions, these make no sense to Vectorise.

However, it is really quite important to not be tempted to conclude that
just because these instructions are un-vectoriseable, the opcode space
must be free for reiterpretation and use for other purposes. This would
be a serious mistake because a future revision of the specification
might *retire* the Scalar instruction, replace it with another.
Again this comes down to being quite strict about the rules: only Scalar
instructions get Vectorised: there are *no* actual explicit Vector
instructions.

**Summary**

Where a traditional Vector ISA effectively duplicates the entirety
of a Scalar ISA and then adds additional instructions which only
make sense in a Vector Context, such as Vector Shuffle, SVP64 goes to
considerable lengths to keep strictly to augmentation and embedding
of an entire Scalar ISA's instructions into an abstract Vectorisation
Context. That abstraction subdivides down into Categories appropriate
for the type of operation (Branch, CRs, Memory, Arithmetic),
and each Category has its own relevant but
ultimately rational quirks.

# Abstraction between Prefix and Suffix

In the introduction paragraph, a great fuss was made emphasising that
the Prefix is kept separate from the Suffix.  The whole idea there is
that a Multi-issue Decoder and subsequent pipelines would in no way have
"back-propagation" of state that can only be determined far too late.
This *has* been preserved, however there is a hiccup.

Examining the Power ISA 3.1 a 64-bit Prefix was introduced, EXT001.
The encoding of the prefix has 6 bits that are dedicated to letting
the hardware know what the remainder of the Prefix bits mean: how they
are formatted, even without having to examine the Suffix to which
they are applied.

SVP64 has such pressure on its 24-bit encoding that it was simply
not possible to perform the same trick used by Power ISA 3.1 Prefixing.
Therefore, rather unfortunately, it becomes necessary to perform
a *partial decoding* of the v3.0 Suffix before the 24-bit SVP64 RM
Fields may be identified.  Fortunately this is straightforward, and
does not rely on any outside state, and even more fortunately
for a Multi-Issue Execution decoder, the length 32/64 is also
easy to identify by looking for the EXT001 pattern. Once identified
the 32/64 bits may be passed independently to multiple Decoders in
parallel.

# Predication

Predication is entirely missing from the Power ISA.
Adding it would be a costly mistake because it cannot be retrofitted
to an ISA without literally duplicating all instructions. Prefixing
is about the only sane way to go.

CR Fields as predicate masks could be spread across multiple register
file entries, making them costly to read in one hit. Therefore the
possibility exists that an instruction element writing to a CR Field
could *overwrite* the Predicate mask CR Vector during the middle of
a for-loop.

Clearly this is bad, so don't do it.  If there are potential issues
they can be avoided by using the crweird instructions to get CR Field
bits into an Integer GPR (r3, r10 or r30) and use that GPR as a
Predicate mask instead.

Even in Vertical-First Mode, which is a single Scalar instruction executed
with "offset" registers (in effect), the rule still applies: don't write
to the same register being used as the predicate, it's `UNDEFINED`
behaviour.

## Single Predication

So named because there is a Twin Predication concept as well, Single
Predication is also unlike other Vector ISAs because it allows zeroing
on both the source and destination.  This takes some explaining.

In Vector ISAs, there is a Predicate Mask, it applies to the
destination only, and there
is a choice of actions when a Predicate Mask bit
is zero:

* set the destination element to zero 
* skip that element operation entirely, leaving the destination unmodified

The problem comes if the underlying register file SRAM is say 64-bit wide
write granularity but the Vector elements are say 8-bit wide.
Some Vector ISAs strongly advocate Zeroing because to leave one single
element at a small bitwidth in amongst other elements where the register
file does not have the prerequisite access granularity is very expensive,
requiring a Read-Modify-Write cycle to preserve the untouched elements.
Putting zero into the destination avoids that Read.

This is technically very easy to solve: use a Register File that does
in fact have the smallest element-level write-enable granularity.
If the elements are 8 bit then allow 8-bit writes!

With that technical issue solved there is nothing in the way of choosing
to support both zeroing and non-zeroing (skipping) at the ISA level:
SV chooses to further support both *on both the source and destination*.
This can result in the source and destination
element indices getting "out-of-sync" even though the Predicate Mask
is the same because the behaviour is different when zeros in the
Predicate are encountered.

## Twin Predication

Twin Predication is an entirely new concept not present in any commercial
Vector ISA of the past forty years.  To explain how normal Single-predication
is applied in a standard Vector ISA:

* Predication on the **source** of a LOAD instruction creates something
  called "Vector Compressed Load" (VCOMPRESS).
* Predication on the **destination** of a STORE instruction creates something
  called "Vector Expanded Store" (VEXPAND).
* SVP64 allows the two to be put back-to-back: one on source, one on
  destination.

The above allows a reader familiar with VCOMPRESS and VEXPAND to
conceptualise what the effect of Twin Predication is, but it actually
goes much further: in *any* twin-predicated instruction (extsw, fmv)
it is possible to apply one predicate to the source register (compressing
the source element array) and another *completely separate* predicate
to the destination register, not just on Load/Stores but on *arithmetic*
operations.

No other Vector ISA in the world has this back-to-back
capability.  All true Vector
ISAs have Predicate Masks: it is an absolutely essential characteristic.
However none of them have abstracted dual predicates out to the extent
where this VCOMPRESS-VEXPAND effect is applicable *in general* to a
wide range of arithmetic
instructions, as well as Load/Store.

It is however important to note that not all instructions can be Twin
Predicated (2P): some remain only Single Predicated (1P), as is normally found
in other Vector ISAs. Arithmetic operations with
four registers (3-in, 1-out, VA-Form for example) are Single. The reason
is that there just wasn't enough space in the 24-bits of the SVP64 Prefix.
Consequently, when using a given instruction, it is necessary to look
up in the ISA Tables whether it is 1P or 2P. caveat emptor!

Also worth a special mention: all Load/Store operations are Twin-Predicated.
The underlying key to understanding:

* one Predicate effectively applies to the Array of Memory *Addresses*,
* the other Predicate effectively applies to the Array of Memory *Data*.

# CR weird instructions

[[sv/cr_int_predication]] is by far the biggest violator of the SVP64
rules, for good reasons.  Transfers between Vectors of CR Fields and Integers
for use as predicates is very awkward without them.

Normally, element width overrides allow the element width to be specified
as 8, 16, 32 or default (64) bit. With CR weird instructions producing or
consuming either 1 bit or 4 bit elements (in effect) some adaptation was
required.  When this perspective is taken (that results or sources are
1 or 4 bits) the weirdness starts to make sense, because the "elements",
such as they are, are still packed sequentially.

From a hardware implementation perspective however they will need special
handling as far as Hazard Dependencies are concerned, due to nonconformance
(bit-level management)

# mv.x (vector permute)

[[sv/mv.x]] aka `GPR(RT) = GPR(GPR(RA))` is so horrendous in
terms of Register Hazard Management that its addition to any Scalar
ISA is anathematic. In a Traditional Vector ISA however, where the
indices are isolated behind a single Vector Hazard, there is no
problem at all.  `sv.mv.x` is also fraught, precisely because it
sits on top of a Standard Scalar register paradigm, not a Vector
ISA with separate and distinct Vector registers.

To help partly solve this, `sv.mv.x` would have had to have
been made relative:

```
for i in range(VL):
    GPR(RT+i) = GPR(RT+MIN(GPR(RA+i), VL))
```

The reason for doing so is that MAXVL or VL may be used to limit
the number of Register Hazards that need to be raised to a fixed
quantity, at Issue time.

`mv.x` itself would still have to be added as a Scalar instruction,
but the behaviour of `sv.mv.x` would have to be different from that
Scalar version.

Normally, Scalar Instructions have a good justification for being
added as Scalar instructions on their own merit. `mv.x` is the
polar opposite, and in the end, the idea was thrown out, and Indexed
REMAP added in its place.  Indexed REMAP comes with its own quirks,
solving the Hazard problem, described in a later section.

# Branch-Conditional

[[sv/branches]] are a very special exception to the rule that there
shall be no deviation from the corresponding
Scalar instruction.  This because of the tight
integration with looping and the application of Boolean Logic
manipulation needed for Parallel operations (predicate mask usage).
This results in an extremely important observation that `scalar identity
behaviour` is violated: the SV Prefixed variant of branch is **not** the same
operation as the unprefixed 32-bit scalar version.

One key difference is that LR is only updated if certain additional
conditions are met, whereas Scalar `bclrl` for example unconditionally
overwrites LR.

Well over 500 Vectorised branch instructions exist in SVP64 due to the
number of options available: close integration and interaction with
the base Scalar Branch was unavoidable in order to create Conditional
Branching suitable for parallel 3D / CUDA GPU workloads.

# Saturation

The application of Saturation as a retro-fit to a Scalar ISA is challenging.
It does help that within the SFFS Compliancy subset there are no Saturated
operations at all: they are only added in VSX.

Saturation does not inherently change the instruction itself: it does however
come with some fundamental implications, when applied. For example:
a Floating-Point operation that would normally raise an exception will
no longer do so, instead setting the CR1.SO Flag.  Another quirky
example: signed operations which produce a negative result will be
truncated to zero if Unsigned Saturation is requested.

One very important aspect for implementors is that the operation in
effect has to be considered to be performed at infinite precision,
followed by saturation detection. In practice this does not actually
require infinite precision hardware! Two 8-bit integers being
added can only ever overflow into a 9-bit result.

Overall some care and consideration needs to be applied.

# Fail-First

Fail-First (both the Load/Store and Data-Dependent variants)
is worthy of a special mention in its own right. Where VL is
normally forward-looking and may be part of a pre-decode phase
in a (simplified) pipelined architecture with no Read-after-Write Hazards,
Fail-First changes that because at any point during the execution
of the element-level instructions, one of those elements may not only
terminate further continuation of the hardware-for-looping but also
effect a change of VL:

```
for i in range(VL):
    result = element_operation(GPR(RA+i), GPR(RB+i))
    if test(result):
        VL = i
        break
```

This is not exactly a violation of SVP64 Rules, more of a breakage
of user expectations, particularly for LD/ST where exceptions
would normally be expected to be raised, Fail-First provides for
avoidance of those exceptions.

For Hardware implementers, a standard Out-of-Order micro-architecture
allows for Cancellation of speculatively-executed elements that extended
beyond the Vector Truncation point.  In-order systems will have a slightly
harder time and may choose to execute one element only at a time, reducing
performance as a result.

# OE=1

The hardware cost of Sticky Overflow in a parallel environment is immense.
The SFFS Compliancy Level is permitted optionally to support XER.SO.
Therefore the decision is made to make it mandatory **not** to
support XER.SO. However, CR.SO *is* supported such that when Rc=1
is set the CR.SO flag will contain only the overflow of
the current instruction, rather than being actually "sticky".
Hardware Out-of-Order designers will recognise and appreciate
that the Hazards are
reduced to Read-After-Write (RAW) and that the WAR Hazard is removed.

This is sort-of a quirk and sort-of not, because the option to support
XER.SO is already optional from the SFFS Compliancy Level.

# Indexed REMAP and CR Field Predication Hazards

Normal Vector ISAs and those Packed SIMD ISAs inspired by them have
Vector "Permute" or "Shuffle" instructions. These provide a Vector of
indices whereby another Vector is reordered (permuted, shuffled) according
to the indices.  Register Hazard Managent here is trivial because there
are three registers: indices source vector, elements source vector to
be shuffled, result vector.

For SVP64 which is based on top of a Scalar Register File paradigm,
combined with the hard requirement to respect full Register Hazard
Management as if element instructions were actual Scalar instructions,
the addition of a Vector permute instruction under these strict
conditions would result in a catastrophic
reduction in performance, due to having to consider Read-after-Write
and Write-after-Read Hazards *at the element level*.

A little leniency and rule-bending is therefore required.

Rather than add explicit Vector permute instructions, the "Indexing"
has been separated out into a REMAP Schedule.  When an Indexed
REMAP is requested, it is assumed (required, of software) that
subsequent instructions intending to use those indices *will not*
attempt to modify the indices. It is *Software* that must consider them
to be read-only.

This simple relaxation of the rules releases Hardware from having the
horrendous job of dynamically detecting Write-after-Read Hazards on a
huge range of registers.

A similar Hazard problem exists for CR Field Predicates, in Vertical-First
Mode.  Instructions could modify CR Fields currently being used as Predicate
Masks: detecting this is so horrendous for hardware resource utilisation
and hardware complexity that, again, the decision is made to relax these
constraints and for Software to take that into account.

# Floating-Point "Single" becomes "Half"

In several places in the Power ISA there are operations that are on
32-bit quantities in 64-bit registers.  The best example is FP which
has 64-bit operations (`fadd`) and 32-bit operations (`fadds` or
FP Add "single").  Element-width overrides it would seem to
be unnecessary, under these circunstances.

However, it is not possible for `fadds` to fit two elements into
64-bit: that breaks the simplicity of SVP64.
Bear in mind that the FP32 bits are spread out across a 64
bit register in FP64 format.  The solution here was to consider the
"s" at the end of each instruction
to mean "half of the element's width". Thus, `sv.fadds/ew=32`
actually stores an FP16 spread out across the 32 bits of an
element, in FP32 format, where `sv.fadd/ew=32` stores a full
FP32 result into the full 32 bits.

Where this breaks down is when attempting to do half-width on
BF16 or FP16 operations: there does not exist a BF8 or an IEE754 FP8
format, so these should be avoided.

# Vertical-First and Subvectors

Documented in the [[sv/setvl]] page, Vertical-First goes through
elements second instructions first and requires an explicit
[[sv/svstep]] instruction to move to the next element,
(whereas Horizontal-First
loops through elements in full first before moving on to
the next instruction): *Subvectors are considered "elements"*
in Vertical-First Mode.

This is conceptually quite easy to keep in mind that a Vertical-First
instruction does one element at a time, and when SUBVL is set,
that "element" in essence becomes a vec2/3/4.

# Swizzle and Pack/Unpack

These are both so weird it's best to just read the pages in full
and pay attention: [[sv/mv.swizzle]] and [[sv/mv.vec]].
Swizzle Moves only engage with vec2/3/4, *reordering* the copying
of the sub-vector elements (including allowing repeats and skips)
based on an immediate supplied by the instruction.  The fun
comes when Pack/Unpack are enabled, and it is really important
to be aware how the Arrays of vec2/3/4 become re-ordered
*and swizzled at the same time*.

Pack/Unpack applies to 
[[sv/mv.vec]] as well however the uniform relationship and
the fact that the source and destination subvector length
must be the same (vec2/3/4) makes things slightly easier to
understand.  The main thing to keep in mind about Pack/Unpack
is that it engages a swap of the ordering of the VL-SUBVL
nested for-loops, in exactly the same way that Matrix REMAP
can do.  When Pack or Unpack is enabled it is the SUBVL for-loop
thar becomes outermost.