5 * <https://bugs.libre-soc.org/show_bug.cgi?id=574> Saturation
6 * <https://bugs.libre-soc.org/show_bug.cgi?id=558#c47> Parallel Prefix
7 * <https://bugs.libre-soc.org/show_bug.cgi?id=697> Reduce Modes
8 * <https://bugs.libre-soc.org/show_bug.cgi?id=864> parallel prefix simulator
9 * <https://bugs.libre-soc.org/show_bug.cgi?id=809> OV sv.addex discussion
10 * ARM SVE Fault-first <https://alastairreid.github.io/papers/sve-ieee-micro-2017.pdf>
12 This is the appendix to [[sv/svp64]], providing explanations of modes
13 etc. leaving the main svp64 page's primary purpose as outlining the
20 # Partial Implementations
22 It is perfectly legal to implement subsets of SVP64 as long as illegal
23 instruction traps are always raised on unimplemented features,
24 so that soft-emulation is possible,
25 even for future revisions of SVP64. With SVP64 being partly controlled
26 through contextual SPRs, a little care has to be taken.
29 not implemented including reserved ones for future use must raise an illegal
30 instruction trap if read or written. This allows software the
31 opportunity to emulate the context created by the given SPR.
33 See [[sv/compliancy_levels]] for full details.
35 # XER, SO and other global flags
37 Vector systems are expected to be high performance. This is achieved
38 through parallelism, which requires that elements in the vector be
39 independent. XER SO/OV and other global "accumulation" flags (CR.SO) cause
40 Read-Write Hazards on single-bit global resources, having a significant
43 Consequently in SV, XER.SO behaviour is disregarded (including
44 in `cmp` instructions). XER.SO is not read, but XER.OV may be written,
45 breaking the Read-Modify-Write Hazard Chain that complicates
46 microarchitectural implementations.
47 This includes when `scalar identity behaviour` occurs. If precise
48 OpenPOWER v3.0/1 scalar behaviour is desired then OpenPOWER v3.0/1
49 instructions should be used without an SV Prefix.
51 TODO jacob add about OV https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-large-integer-arithmetic-paper.pdf
53 Of note here is that XER.SO and OV may already be disregarded in the
54 Power ISA v3.0/1 SFFS (Scalar Fixed and Floating) Compliancy Subset.
55 SVP64 simply makes it mandatory to disregard XER.SO even for other Subsets,
56 but only for SVP64 Prefixed Operations.
58 XER.CA/CA32 on the other hand is expected and required to be implemented
59 according to standard Power ISA Scalar behaviour. Interestingly, due
60 to SVP64 being in effect a hardware for-loop around Scalar instructions
61 executing in precise Program Order, a little thought shows that a Vectorised
62 Carry-In-Out add is in effect a Big Integer Add, taking a single bit Carry In
63 and producing, at the end, a single bit Carry out. High performance
64 implementations may exploit this observation to deploy efficient
65 Parallel Carry Lookahead.
67 # assume VL=4, this results in 4 sequential ops (below)
68 sv.adde r0.v, r4.v, r8.v
70 # instructions that get executed in backend hardware:
71 adde r0, r4, r8 # takes carry-in, produces carry-out
72 adde r1, r5, r9 # takes carry from previous
74 adde r3, r7, r11 # likewise
76 It can clearly be seen that the carry chains from one
77 64 bit add to the next, the end result being that a
78 256-bit "Big Integer Add with Carry" has been performed, and that
79 CA contains the 257th bit. A one-instruction 512-bit Add-with-Carry
80 may be performed by setting VL=8, and a one-instruction
81 1024-bit Add-with-Carry by setting VL=16, and so on. More on
82 this in [[openpower/sv/biginteger]]
84 # v3.0B/v3.1 relevant instructions
86 SV is primarily designed for use as an efficient hybrid 3D GPU / VPU /
89 Vectorisation of the VSX Packed SIMD system makes no sense whatsoever,
90 the sole exceptions potentially being any operations with 128-bit
91 operands such as `vrlq` (Rotate Quad Word) and `xsaddqp` (Scalar
93 SV effectively *replaces* the majority of VSX, requiring far less
94 instructions, and provides, at the very minimum, predication
95 (which VSX was designed without).
97 Likewise, Load/Store Multiple make no sense to
98 have because they are not only provided by SV, the SV alternatives may
99 be predicated as well, making them far better suited to use in function
100 calls and context-switching.
102 Additionally, some v3.0/1 instructions simply make no sense at all in a
103 Vector context: `rfid` falls into this category,
104 as well as `sc` and `scv`. Here there is simply no point
105 trying to Vectorise them: the standard OpenPOWER v3.0/1 instructions
106 should be called instead.
108 Fortuitously this leaves several Major Opcodes free for use by SV
109 to fit alternative future instructions. In a 3D context this means
110 Vector Product, Vector Normalise, [[sv/mv.swizzle]], Texture LD/ST
111 operations, and others critical to an efficient, effective 3D GPU and
112 VPU ISA. With such instructions being included as standard in other
113 commercially-successful GPU ISAs it is likewise critical that a 3D
114 GPU/VPU based on svp64 also have such instructions.
116 Note however that svp64 is stand-alone and is in no way
117 critically dependent on the existence or provision of 3D GPU or VPU
118 instructions. These should be considered entirely separate
119 extensions, and their discussion
120 and specification is out of scope for this document.
122 ## Major opcode map (v3.0B)
124 This table is taken from v3.0B.
125 Table 9: Primary Opcode Map (opcode bits 0:5)
128 | 000 | 001 | 010 | 011 | 100 | 101 | 110 | 111
129 000 | | | tdi | twi | EXT04 | | | mulli | 000
130 001 | subfic | | cmpli | cmpi | addic | addic. | addi | addis | 001
131 010 | bc/l/a | EXT17 | b/l/a | EXT19 | rlwimi| rlwinm | | rlwnm | 010
132 011 | ori | oris | xori | xoris | andi. | andis. | EXT30 | EXT31 | 011
133 100 | lwz | lwzu | lbz | lbzu | stw | stwu | stb | stbu | 100
134 101 | lhz | lhzu | lha | lhau | sth | sthu | lmw | stmw | 101
135 110 | lfs | lfsu | lfd | lfdu | stfs | stfsu | stfd | stfdu | 110
136 111 | lq | EXT57 | EXT58 | EXT59 | EXT60 | EXT61 | EXT62 | EXT63 | 111
137 | 000 | 001 | 010 | 011 | 100 | 101 | 110 | 111
140 It is important to note that having a different v3.0B Scalar opcode
141 that is different from an SVP64 one is highly undesirable: the complexity
142 in the decoder is greatly increased, through breaking of the RISC paradigm.
144 # EXTRA Field Mapping
146 The purpose of the 9-bit EXTRA field mapping is to mark individual
147 registers (RT, RA, BFA) as either scalar or vector, and to extend
148 their numbering from 0..31 in Power ISA v3.0 to 0..127 in SVP64.
149 Three of the 9 bits may also be used up for a 2nd Predicate (Twin
150 Predication) leaving a mere 6 bits for qualifying registers. As can
151 be seen there is significant pressure on these (and in fact all) SVP64 bits.
153 In Power ISA v3.1 prefixing there are bits which describe and classify
154 the prefix in a fashion that is independent of the suffix. MLSS for
155 example. For SVP64 there is insufficient space to make the SVP64 Prefix
156 "self-describing", and consequently every single Scalar instruction
157 had to be individually analysed, by rote, to craft an EXTRA Field Mapping.
158 This process was semi-automated and is described in this section.
159 The final results, which are part of the SVP64 Specification, are here:
160 [[openpower/opcode_regs_deduped]]
162 * Firstly, every instruction's mnemonic (`add RT, RA, RB`) was analysed
163 from reading the markdown formatted version of the Scalar pseudocode
164 which is machine-readable and found in [[openpower/isatables]]. The
165 analysis gives, by instruction, a "Register Profile". `add RT, RA, RB`
166 for example is given a designation `RM-2R-1W` because it requires
167 two GPR reads and one GPR write.
168 * Secondly, the total number of registers was added up (2R-1W is 3 registers)
169 and if less than or equal to three then that instruction could be given an
170 EXTRA3 designation. Four or more is given an EXTRA2 designation because
171 there are only 9 bits available.
172 * Thirdly, the instruction was analysed to see if Twin or Single
173 Predication was suitable. As a general rule this was if there
174 was only a single operand and a single result (`extw` and LD/ST)
175 however it was found that some 2 or 3 operand instructions also
176 qualify. Given that 3 of the 9 bits of EXTRA had to be sacrificed for use
177 in Twin Predication, some compromises were made, here. LDST is
178 Twin but also has 3 operands in some operations, so only EXTRA2 can be used.
179 * Fourthly, a packing format was decided: for 2R-1W an EXTRA3 indexing
180 could have been decided
181 that RA would be indexed 0 (EXTRA bits 0-2), RB indexed 1 (EXTRA bits 3-5)
182 and RT indexed 2 (EXTRA bits 6-8). In some cases (LD/ST with update)
183 RA-as-a-source is given a **different** EXTRA index from RA-as-a-result
184 (because it is possible to do, and perceived to be useful). Rc=1
185 co-results (CR0, CR1) are always given the same EXTRA index as their
186 main result (RT, FRT).
187 * Fifthly, in an automated process the results of the analysis
188 were outputted in CSV Format for use in machine-readable form
189 by sv_analysis.py <https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/sv/sv_analysis.py;hb=HEAD>
191 This process was laborious but logical, and, crucially, once a
192 decision is made (and ratified) cannot be reversed.
193 Qualifying future Power ISA Scalar instructions for SVP64
194 is **strongly** advised to utilise this same process and the same
195 sv_analysis.py program as a canonical method of maintaining the
196 relationships. Alterations to that same program which
197 change the Designation is **prohibited** once finalised (ratified
198 through the Power ISA WG Process). It would
199 be similar to deciding that `add` should be changed from X-Form
202 # Single Predication <a name="1p"> </a>
204 This is a standard mode normally found in Vector ISAs. every element in every source Vector and in the destination uses the same bit of one single predicate mask.
206 In SVSTATE, for Single-predication, implementors MUST increment both srcstep and dststep, but depending on whether sz and/or dz are set, srcstep and
207 dststep can still potentially become different indices. Only when sz=dz
208 is srcstep guaranteed to equal dststep at all times.
210 Note that in some Mode Formats there is only one flag (zz). This indicates
211 that *both* sz *and* dz are set to the same.
219 The following schedule for srcstep and dststep will occur:
221 | srcstep | dststep | comment |
222 | ---- | ----- | -------- |
223 | 0 | 0 | both mask[src=0] and mask[dst=0] are 1 |
224 | 1 | 2 | sz=1 but dz=0: dst skips mask[1], src soes not |
225 | 2 | 3 | mask[src=2] and mask[dst=3] are 1 |
226 | end | end | loop has ended because dst reached VL-1 |
234 The following schedule for srcstep and dststep will occur:
236 | srcstep | dststep | comment |
237 | ---- | ----- | -------- |
238 | 0 | 0 | both mask[src=0] and mask[dst=0] are 1 |
239 | 2 | 1 | sz=0 but dz=1: src skips mask[1], dst does not |
240 | 3 | 2 | mask[src=3] and mask[dst=2] are 1 |
241 | end | end | loop has ended because src reached VL-1 |
243 In both these examples it is crucial to note that despite there being
244 a single predicate mask, with sz and dz being different, srcstep and
245 dststep are being requested to react differently.
253 The following schedule for srcstep and dststep will occur:
255 | srcstep | dststep | comment |
256 | ---- | ----- | -------- |
257 | 0 | 0 | both mask[src=0] and mask[dst=0] are 1 |
258 | 2 | 2 | sz=0 and dz=0: both src and dst skip mask[1] |
259 | 3 | 3 | mask[src=3] and mask[dst=3] are 1 |
260 | end | end | loop has ended because src and dst reached VL-1 |
262 Here, both srcstep and dststep remain in lockstep because sz=dz=1
264 # Twin Predication <a name="2p"> </a>
266 This is a novel concept that allows predication to be applied to a single
267 source and a single dest register. The following types of traditional
268 Vector operations may be encoded with it, *without requiring explicit
271 * VSPLAT (a single scalar distributed across a vector)
272 * VEXTRACT (like LLVM IR [`extractelement`](https://releases.llvm.org/11.0.0/docs/LangRef.html#extractelement-instruction))
273 * VINSERT (like LLVM IR [`insertelement`](https://releases.llvm.org/11.0.0/docs/LangRef.html#insertelement-instruction))
274 * VCOMPRESS (like LLVM IR [`llvm.masked.compressstore.*`](https://releases.llvm.org/11.0.0/docs/LangRef.html#llvm-masked-compressstore-intrinsics))
275 * VEXPAND (like LLVM IR [`llvm.masked.expandload.*`](https://releases.llvm.org/11.0.0/docs/LangRef.html#llvm-masked-expandload-intrinsics))
277 Those patterns (and more) may be applied to:
279 * mv (the usual way that V\* ISA operations are created)
280 * exts\* sign-extension
281 * rwlinm and other RS-RA shift operations (**note**: excluding
282 those that take RA as both a src and dest. These are not
283 1-src 1-dest, they are 2-src, 1-dest)
284 * LD and ST (treating AGEN as one source)
285 * FP fclass, fsgn, fneg, fabs, fcvt, frecip, fsqrt etc.
286 * Condition Register ops mfcr, mtcr and other similar
288 This is a huge list that creates extremely powerful combinations,
289 particularly given that one of the predicate options is `(1<<r3)`
291 Additional unusual capabilities of Twin Predication include a back-to-back
292 version of VCOMPRESS-VEXPAND which is effectively the ability to do
293 sequentially ordered multiple VINSERTs. The source predicate selects a
294 sequentially ordered subset of elements to be inserted; the destination
295 predicate specifies the sequentially ordered recipient locations.
296 This is equivalent to
297 `llvm.masked.compressstore.*`
299 `llvm.masked.expandload.*`
300 with a single instruction.
302 This extreme power and flexibility comes down to the fact that SVP64
303 is not actually a Vector ISA: it is a loop-abstraction-concept that
304 is applied *in general* to Scalar operations, just like the x86
305 `REP` instruction (if put on steroids).
309 The pack/unpack concept of VSX `vpack` is abstracted out as Sub-Vector
311 Two bits in the `SVSHAPE` [[sv/spr]]
312 enable either "packing" or "unpacking"
313 on the subvectors vec2/3/4.
316 "normal" SVP64 operation with `SUBVL!=1:` (assuming no elwidth overrides),
317 note that the VL loop is outer and the SUBVL loop inner:
321 for j in range(SUBVL):
327 For pack/unpack (again, no elwidth overrides), note that now there is the
328 option to swap the SUBVL and VL loop orders.
329 In effect the Pack/Unpack performs a Transpose of the subvector elements:
331 # yield an outer-SUBVL or inner VL loop with SUBVL
334 for j in range(SUBVL): # subvl is outer
335 for i in range(VL): # vl is inner
338 for i in range(VL): # vl is outer
339 for j in range(SUBVL): # subvl is inner
342 # walk through both source and dest indices simultaneously
343 for src_idx, dst_idx in zip(index_p(PACK), index_p(UNPACK)):
344 move_operation(RT+dst_idx, RA+src_idx)
346 "yield" from python is used here for simplicity and clarity.
347 The two Finite State Machines for the generation of the source
348 and destination element offsets progress incrementally in
351 Example VL=2, SUBVL=3, PACK_en=1 - elements grouped by
352 vec3 will be redistributed such that Sub-elements 0 are
353 packed together, Sub-elements 1 are packed together, as
359 dststep=0 dststep=1 dststep=2
362 Setting of both `PACK` and `UNPACK` is neither prohibited nor
363 `UNDEFINED` because the reordering is fully deterministic, and
364 additional REMAP reordering may be applied. Combined with
365 Matrix REMAP this would
366 give potentially up to 4 Dimensions of reordering.
368 Pack/Unpack has quirky interactions on
369 [[sv/mv.swizzle]] because it can set a different subvector length for
370 destination, and has a slightly different pseudocode algorithm
371 for Vertical-First Mode.
373 Pack/Unpack is enabled (set up) through [[sv/setvl]].
377 Reduction in SVP64 is deterministic and somewhat of a misnomer. A normal
378 Vector ISA would have explicit Reduce opcodes with defined characteristics
379 per operation: in SX Aurora there is even an additional scalar argument
380 containing the initial reduction value, and the default is either 0
381 or 1 depending on the specifics of the explicit opcode.
382 SVP64 fundamentally has to
383 utilise *existing* Scalar Power ISA v3.0B operations, which presents some
386 The solution turns out to be to simply define reduction as permitting
387 deterministic element-based schedules to be issued using the base Scalar
388 operations, and to rely on the underlying microarchitecture to resolve
389 Register Hazards at the element level. This goes back to
390 the fundamental principle that SV is nothing more than a Sub-Program-Counter
391 sitting between Decode and Issue phases.
393 For Scalar Reduction,
394 Microarchitectures *may* take opportunities to parallelise the reduction
395 but only if in doing so they preserve strict Program Order at the Element Level.
396 Opportunities where this is possible include an `OR` operation
397 or a MIN/MAX operation: it may be possible to parallelise the reduction,
398 but for Floating Point it is not permitted due to different results
399 being obtained if the reduction is not executed in strict Program-Sequential
402 In essence it becomes the programmer's responsibility to leverage the
403 pre-determined schedules to desired effect.
405 ## Scalar result reduction and iteration
407 Scalar Reduction per se does not exist, instead is implemented in SVP64
408 as a simple and natural relaxation of the usual restriction on the Vector
409 Looping which would terminate if the destination was marked as a Scalar.
410 Scalar Reduction by contrast *keeps issuing Vector Element Operations*
411 even though the destination register is marked as scalar.
412 Thus it is up to the programmer to be aware of this, observe some
413 conventions, and thus end up achieving the desired outcome of scalar
416 It is also important to appreciate that there is no
417 actual imposition or restriction on how this mode is utilised: there
418 will therefore be several valuable uses (including Vector Iteration
420 and it is up to the programmer to make best use of the
421 (strictly deterministic) capability
424 In this mode, which is suited to operations involving carry or overflow,
425 one register must be assigned, by convention by the programmer to be the
426 "accumulator". Scalar reduction is thus categorised by:
428 * One of the sources is a Vector
429 * the destination is a scalar
430 * optionally but most usefully when one source scalar register is
431 also the scalar destination (which may be informally termed
433 * That the source register type is the same as the destination register
434 type identified as the "accumulator". Scalar reduction on `cmp`,
435 `setb` or `isel` makes no sense for example because of the mixture
436 between CRs and GPRs.
438 *Note that issuing instructions in Scalar reduce mode such as `setb`
439 are neither `UNDEFINED` nor prohibited, despite them not making much
440 sense at first glance.
441 Scalar reduce is strictly defined behaviour, and the cost in
442 hardware terms of prohibition of seemingly non-sensical operations is too great.
443 Therefore it is permitted and required to be executed successfully.
444 Implementors **MAY** choose to optimise such instructions in instances
445 where their use results in "extraneous execution", i.e. where it is clear
446 that the sequence of operations, comprising multiple overwrites to
447 a scalar destination **without** cumulative, iterative, or reductive
448 behaviour (no "accumulator"), may discard all but the last element
449 operation. Identification
450 of such is trivial to do for `setb` and `cmp`: the source register type is
451 a completely different register file from the destination.
452 Likewise Scalar reduction when the destination is a Vector
453 is as if the Reduction Mode was not requested. However it would clearly
454 be unacceptable to perform such optimisations on cache-inhibited LD/ST,
455 so some considerable care needs to be taken.*
457 Typical applications include simple operations such as `ADD r3, r10.v,
458 r3` where, clearly, r3 is being used to accumulate the addition of all
459 elements of the vector starting at r10.
461 # add RT, RA,RB but when RT==RA
463 iregs[RA] += iregs[RB+i] # RT==RA
465 However, *unless* the operation is marked as "mapreduce" (`sv.add/mr`)
467 **terminates** at the first scalar operation. Only by marking the
468 operation as "mapreduce" will it continue to issue multiple sub-looped
469 (element) instructions in `Program Order`.
471 To perform the loop in reverse order, the ```RG``` (reverse gear) bit must be set. This may be useful in situations where the results may be different
472 (floating-point) if executed in a different order. Given that there is
473 no actual prohibition on Reduce Mode being applied when the destination
474 is a Vector, the "Reverse Gear" bit turns out to be a way to apply Iterative
475 or Cumulative Vector operations in reverse. `sv.add/rg r3.v, r4.v, r4.v`
476 for example will start at the opposite end of the Vector and push
477 a cumulative series of overlapping add operations into the Execution units of
478 the underlying hardware.
480 Other examples include shift-mask operations where a Vector of inserts
481 into a single destination register is required (see [[sv/bitmanip]], bmset),
482 as a way to construct
483 a value quickly from multiple arbitrary bit-ranges and bit-offsets.
484 Using the same register as both the source and destination, with Vectors
485 of different offsets masks and values to be inserted has multiple
486 applications including Video, cryptography and JIT compilation.
489 # * Vector of shift-offsets contained in RC (r12.v)
490 # * Vector of masks contained in RB (r8.v)
491 # * Vector of values to be masked-in in RA (r4.v)
492 # * Scalar destination RT (r0) to receive all mask-offset values
493 sv.bmset/mr r0, r4.v, r8.v, r12.v
495 Due to the Deterministic Scheduling,
496 Subtract and Divide are still permitted to be executed in this mode,
497 although from an algorithmic perspective it is strongly discouraged.
498 It would be better to use addition followed by one final subtract,
499 or in the case of divide, to get better accuracy, to perform a multiply
500 cascade followed by a final divide.
502 Note that single-operand or three-operand scalar-dest reduce is perfectly
503 well permitted: the programmer may still declare one register, used as
504 both a Vector source and Scalar destination, to be utilised as
505 the "accumulator". In the case of `sv.fmadds` and `sv.maddhw` etc
506 this naturally fits well with the normal expected usage of these
509 If an interrupt or exception occurs in the middle of the scalar mapreduce,
510 the scalar destination register **MUST** be updated with the current
511 (intermediate) result, because this is how ```Program Order``` is
512 preserved (Vector Loops are to be considered to be just another way of issuing instructions
513 in Program Order). In this way, after return from interrupt,
514 the scalar mapreduce may continue where it left off. This provides
515 "precise" exception behaviour.
517 Note that hardware is perfectly permitted to perform multi-issue
518 parallel optimisation of the scalar reduce operation: it's just that
519 as far as the user is concerned, all exceptions and interrupts **MUST**
523 # Fail-on-first <a name="fail-first"> </a>
525 Data-dependent fail-on-first has two distinct variants: one for LD/ST
527 the other for arithmetic operations (actually, CR-driven)
528 [[sv/normal]] and CR operations [[sv/cr_ops]].
530 case the assumption is that vector elements are required appear to be
531 executed in sequential Program Order, element 0 being the first.
533 * LD/ST ffirst treats the first LD/ST in a vector (element 0) as an
534 ordinary one. Exceptions occur "as normal". However for elements 1
535 and above, if an exception would occur, then VL is **truncated** to the
537 * Data-driven (CR-driven) fail-on-first activates when Rc=1 or other
538 CR-creating operation produces a result (including cmp). Similar to
539 branch, an analysis of the CR is performed and if the test fails, the
540 vector operation terminates and discards all element operations
541 above the current one (and the current one if VLi is not set),
542 and VL is truncated to either
543 the *previous* element or the current one, depending on whether
544 VLi (VL "inclusive") is set.
546 Thus the new VL comprises a contiguous vector of results,
547 all of which pass the testing criteria (equal to zero, less than zero).
549 The CR-based data-driven fail-on-first is new and not found in ARM
550 SVE or RVV. At the same time it is also "old" because it is a generalisation
552 [Block compare](https://rvbelzen.tripod.com/z80prgtemp/z80prg04.htm)
553 instructions, especially
554 [CPIR](http://z80-heaven.wikidot.com/instructions-set:cpir)
555 which is based on CP (compare) as the ultimate "element" (suffix)
556 operation to which the repeat (prefix) is applied.
557 It is extremely useful for reducing instruction count,
558 however requires speculative execution involving modifications of VL
559 to get high performance implementations. An additional mode (RC1=1)
560 effectively turns what would otherwise be an arithmetic operation
561 into a type of `cmp`. The CR is stored (and the CR.eq bit tested
562 against the `inv` field).
563 If the CR.eq bit is equal to `inv` then the Vector is truncated and
565 Note that when RC1=1 the result elements are never stored, only the CRs.
567 VLi is only available as an option when `Rc=0` (or for instructions
568 which do not have Rc). When set, the current element is always
569 also included in the count (the new length that VL will be set to).
570 This may be useful in combination with "inv" to truncate the Vector
571 to *exclude* elements that fail a test, or, in the case of implementations
572 of strncpy, to include the terminating zero.
574 In CR-based data-driven fail-on-first there is only the option to select
575 and test one bit of each CR (just as with branch BO). For more complex
576 tests this may be insufficient. If that is the case, a vectorised crops
577 (crand, cror) may be used, and ffirst applied to the crop instead of to
578 the arithmetic vector.
580 One extremely important aspect of ffirst is:
582 * LDST ffirst may never set VL equal to zero. This because on the first
583 element an exception must be raised "as normal".
584 * CR-based data-dependent ffirst on the other hand **can** set VL equal
585 to zero. This is the only means in the entirety of SV that VL may be set
586 to zero (with the exception of via the SV.STATE SPR). When VL is set
587 zero due to the first element failing the CR bit-test, all subsequent
588 vectorised operations are effectively `nops` which is
589 *precisely the desired and intended behaviour*.
591 Another aspect is that for ffirst LD/STs, VL may be truncated arbitrarily
592 to a nonzero value for any implementation-specific reason. For example:
593 it is perfectly reasonable for implementations to alter VL when ffirst
594 LD or ST operations are initiated on a nonaligned boundary, such that
595 within a loop the subsequent iteration of that loop begins subsequent
596 ffirst LD/ST operations on an aligned boundary. Likewise, to reduce
597 workloads or balance resources.
599 CR-based data-dependent first on the other hand MUST not truncate VL
600 arbitrarily to a length decided by the hardware: VL MUST only be
601 truncated based explicitly on whether a test fails.
602 This because it is a precise test on which algorithms
605 *Note: there is no reverse-direction for Data-dependent Fail-First.
606 REMAP will need to be activated to invert the ordering of element
609 ## Data-dependent fail-first on CR operations (crand etc)
611 Operations that actually produce or alter CR Field as a result
612 do not also in turn have an Rc=1 mode. However it makes no
613 sense to try to test the 4 bits of a CR Field for being equal
614 or not equal to zero. Moreover, the result is already in the
615 form that is desired: it is a CR field. Therefore,
616 CR-based operations have their own SVP64 Mode, described
619 There are two primary different types of CR operations:
621 * Those which have a 3-bit operand field (referring to a CR Field)
622 * Those which have a 5-bit operand (referring to a bit within the
625 More details can be found in [[sv/cr_ops]].
629 Pred-result mode may not be applied on CR-based operations.
631 Although CR operations (mtcr, crand, cror) may be Vectorised,
632 predicated, pred-result mode applies to operations that have
633 an Rc=1 mode, or make sense to add an RC1 option.
635 Predicate-result merges common CR testing with predication, saving on
636 instruction count. In essence, a Condition Register Field test
637 is performed, and if it fails it is considered to have been
638 *as if* the destination predicate bit was zero. Given that
639 there are no CR-based operations that produce Rc=1 co-results,
640 there can be no pred-result mode for mtcr and other CR-based instructions
642 Arithmetic and Logical Pred-result, which does have Rc=1 or for which
643 RC1 Mode makes sense, is covered in [[sv/normal]]
647 CRs are slightly more involved than INT or FP registers due to the
648 possibility for indexing individual bits (crops BA/BB/BT). Again however
649 the access pattern needs to be understandable in relation to v3.0B / v3.1B
650 numbering, with a clear linear relationship and mapping existing when
653 ## CR EXTRA mapping table and algorithm <a name="cr_extra"></a>
655 Numbering relationships for CR fields are already complex due to being
656 in BE format (*the relationship is not clearly explained in the v3.0B
657 or v3.1 specification*). However with some care and consideration
658 the exact same mapping used for INT and FP regfiles may be applied,
659 just to the upper bits, as explained below. Firstly and most
660 importantly a new notation
661 `CR{field number}` is used to indicate access to a particular
662 Condition Register Field (as opposed to the notation `CR[bit]`
663 which accesses one bit of the 32 bit Power ISA v3.0B
666 `CR{n}` refers to `CR0` when `n=0` and consequently, for CR0-7, is defined, in v3.0B pseudocode, as:
668 CR{7-n} = CR[32+n*4:35+n*4]
670 For SVP64 the relationship for the sequential
671 numbering of elements is to the CR **fields** within
672 the CR Register, not to individual bits within the CR register.
674 The `CR{n}` notation is designed to give *linear sequential
675 numbering* in the Vector domain on a straight sequential Vector Loop.
676 Without it, there is the risk of massive confusion as CR Fields
677 could be accessed in order `CR7 CR6 ... CR0 CR15 CR14 .. CR8 CR23..`
680 In OpenPOWER v3.0/1, BF/BT/BA/BB are all 5 bits. The top 3 bits (0:2)
681 select one of the 8 CRs; the bottom 2 bits (3:4) select one of 4 bits
682 *in* that CR (EQ/LT/GT/SO). The numbering was determined (after 4 months of
683 analysis and research) to be as follows:
685 CR_index = 7-(BA>>2) # top 3 bits but de-MSB0'd
686 bit_index = 3-(BA & 0b11) # low 2 bits but de-MSB0'd
687 CR_reg = CR{CR_index} # get the CR
688 # finally get the bit from the CR.
689 CR_bit = (CR_reg & (1<<bit_index)) != 0
691 When it comes to applying SV, it is the *CR Field* number `CR_reg`
693 applies, **not** the `CR_bit` portion (bits 3-4):
698 spec = EXTRA2<<1 | 0b0
700 # vector constructs "BA[0:2] spec[1:2] 00 BA[3:4]"
701 return ((BA >> 2)<<6) | # hi 3 bits shifted up
702 (spec[1:2]<<4) | # to make room for these
703 (BA & 0b11) # CR_bit on the end
705 # scalar constructs "00 spec[1:2] BA[0:4]"
706 return (spec[1:2] << 5) | BA
708 Thus, for example, to access a given bit for a CR in SV mode, the v3.0B
709 algorithm to determine CR\_reg is modified to as follows:
711 CR_index = 7-(BA>>2) # top 3 bits but BE
713 # vector mode, 0-124 increments of 4
714 CR_index = (CR_index<<4) | (spec[1:2] << 2)
716 # scalar mode, 0-32 increments of 1
717 CR_index = (spec[1:2]<<3) | CR_index
718 # same as for v3.0/v3.1 from this point onwards
719 bit_index = 3-(BA & 0b11) # low 2 bits but BE
720 CR_reg = CR{CR_index} # get the CR
721 # finally get the bit from the CR.
722 CR_bit = (CR_reg & (1<<bit_index)) != 0
724 Note here that the decoding pattern to determine CR\_bit does not change.
726 Note: high-performance implementations may read/write Vectors of CRs in
727 batches of aligned 32-bit chunks (CR0-7, CR7-15). This is to greatly
728 simplify internal design. If instructions are issued where CR Vectors
729 do not start on a 32-bit aligned boundary, performance may be affected.
731 ## CR fields as inputs/outputs of vector operations
733 CRs (or, the arithmetic operations associated with them)
734 may be marked as Vectorised or Scalar. When Rc=1 in arithmetic operations that have no explicit EXTRA to cover the CR, the CR is Vectorised if the destination is Vectorised. Likewise if the destination is scalar then so is the CR.
736 When vectorized, the CR inputs/outputs are sequentially read/written
737 to 4-bit CR fields. Vectorised Integer results, when Rc=1, will begin
738 writing to CR8 (TBD evaluate) and increase sequentially from there.
741 * implementations may rely on the Vector CRs being aligned to 8. This
742 means that CRs may be read or written in aligned batches of 32 bits
743 (8 CRs per batch), for high performance implementations.
744 * scalar Rc=1 operation (CR0, CR1) and callee-saved CRs (CR2-4) are not
745 overwritten by vector Rc=1 operations except for very large VL
746 * CR-based predication, from CR32, is also not interfered with
747 (except by large VL).
749 However when the SV result (destination) is marked as a scalar by the
750 EXTRA field the *standard* v3.0B behaviour applies: the accompanying
751 CR when Rc=1 is written to. This is CR0 for integer operations and CR1
754 Note that yes, the CR Fields are genuinely Vectorised. Unlike in SIMD VSX which
755 has a single CR (CR6) for a given SIMD result, SV Vectorised OpenPOWER
756 v3.0B scalar operations produce a **tuple** of element results: the
757 result of the operation as one part of that element *and a corresponding
758 CR element*. Greatly simplified pseudocode:
761 # calculate the vector result of an add
762 iregs[RT+i] = iregs[RA+i] + iregs[RB+i]
763 # now calculate CR bits
764 CRs{8+i}.eq = iregs[RT+i] == 0
765 CRs{8+i}.gt = iregs[RT+i] > 0
768 If a "cumulated" CR based analysis of results is desired (a la VSX CR6)
769 then a followup instruction must be performed, setting "reduce" mode on
770 the Vector of CRs, using cr ops (crand, crnor) to do so. This provides far
771 more flexibility in analysing vectors than standard Vector ISAs. Normal
772 Vector ISAs are typically restricted to "were all results nonzero" and
773 "were some results nonzero". The application of mapreduce to Vectorised
774 cr operations allows far more sophisticated analysis, particularly in
775 conjunction with the new crweird operations see [[sv/cr_int_predication]].
777 Note in particular that the use of a separate instruction in this way
778 ensures that high performance multi-issue OoO inplementations do not
779 have the computation of the cumulative analysis CR as a bottleneck and
780 hindrance, regardless of the length of VL.
783 SVP64 [[sv/branches]] may be used, even when the branch itself is to
784 the following instruction. The combined side-effects of CTR reduction
785 and VL truncation provide several benefits.
787 (see [[discussion]]. some alternative schemes are described there)
789 ## Rc=1 when SUBVL!=1
791 sub-vectors are effectively a form of Packed SIMD (length 2 to 4). Only 1 bit of
792 predicate is allocated per subvector; likewise only one CR is allocated
795 This leaves a conundrum as to how to apply CR computation per subvector,
796 when normally Rc=1 is exclusively applied to scalar elements. A solution
797 is to perform a bitwise OR or AND of the subvector tests. Given that
798 OE is ignored in SVP64, this field may (when available) be used to select OR or
801 ### Table of CR fields
803 CRn is the notation used by the OpenPower spec to refer to CR field #i,
804 so FP instructions with Rc=1 write to CR1 (n=1).
806 CRs are not stored in SPRs: they are registers in their own right.
807 Therefore context-switching the full set of CRs involves a Vectorised
808 mfcr or mtcr, using VL=8 to do so. This is exactly as how
809 scalar OpenPOWER context-switches CRs: it is just that there are now
812 The 64 SV CRs are arranged similarly to the way the 128 integer registers
813 are arranged. TODO a python program that auto-generates a CSV file
814 which can be included in a table, which is in a new page (so as not to
815 overwhelm this one). [[svp64/cr_names]]
819 Instructions are broken down by Register Profiles as listed in the
820 following auto-generated page: [[opcode_regs_deduped]]. These tables,
821 despite being auto-generated, are part of the Specification.
823 # SV pseudocode illustration
825 ## Single-predicated Instruction
827 illustration of normal mode add operation: zeroing not included, elwidth
828 overrides not included. if there is no predicate, it is set to all 1s
830 function op_add(rd, rs1, rs2) # add not VADD!
831 int i, id=0, irs1=0, irs2=0;
832 predval = get_pred_val(FALSE, rd);
833 for (i = 0; i < VL; i++)
834 STATE.srcoffs = i # save context
835 if (predval & 1<<i) # predication uses intregs
836 ireg[rd+id] <= ireg[rs1+irs1] + ireg[rs2+irs2];
837 if (!int_vec[rd].isvec) break;
838 if (rd.isvec) { id += 1; }
839 if (rs1.isvec) { irs1 += 1; }
840 if (rs2.isvec) { irs2 += 1; }
841 if (id == VL or irs1 == VL or irs2 == VL) {
842 # end VL hardware loop
843 STATE.srcoffs = 0; # reset
847 This has several modes:
850 * RT.v = RA.v RB.s (and RA.s RB.v)
853 * RT.s = RA.v RB.s (and RA.s RB.v)
856 All of these may be predicated. Vector-Vector is straightfoward.
857 When one of source is a Vector and the other a Scalar, it is clear that
858 each element of the Vector source should be added to the Scalar source,
859 each result placed into the Vector (or, if the destination is a scalar,
860 only the first nonpredicated result).
862 The one that is not obvious is RT=vector but both RA/RB=scalar.
863 Here this acts as a "splat scalar result", copying the same result into
864 all nonpredicated result elements. If a fixed destination scalar was
865 intended, then an all-Scalar operation should be used.
867 See <https://bugs.libre-soc.org/show_bug.cgi?id=552>
869 # Assembly Annotation
871 Assembly code annotation is required for SV to be able to successfully
872 mark instructions as "prefixed".
874 A reasonable (prototype) starting point:
880 * ew=8/16/32 - element width
881 * sew=8/16/32 - source element width
883 * mode=mr/satu/sats/crpred
884 * pred=1\<\<3/r3/~r3/r10/~r10/r30/~r30/lt/gt/le/ge/eq/ne
886 similar to x86 "rex" prefix.
888 For actual assembler:
890 sv.asmcode/mode.vec{N}.ew=8,sw=16,m={pred},sm={pred} reg.v, src.s
894 * m={pred}: predicate mask mode
895 * sm={pred}: source-predicate mask mode (only allowed in Twin-predication)
896 * vec{N}: vec2 OR vec3 OR vec4 - sets SUBVL=2/3/4
897 * ew={N}: ew=8/16/32 - sets elwidth override
898 * sw={N}: sw=8/16/32 - sets source elwidth override
899 * ff={xx}: see fail-first mode
900 * pr={xx}: see predicate-result mode
901 * sat{x}: satu / sats - see saturation mode
902 * mr: see map-reduce mode
903 * mrr: map-reduce, reverse-gear (VL-1 downto 0)
904 * mr.svm see map-reduce with sub-vector mode
905 * crm: see map-reduce CR mode
906 * crm.svm see map-reduce CR with sub-vector mode
907 * sz: predication with source-zeroing
908 * dz: predication with dest-zeroing
913 - pm=lt/gt/le/ge/eq/ne/so/ns
916 - ff=lt/gt/le/ge/eq/ne/so/ns
922 - mr OR crm: "normal" map-reduce mode or CR-mode.
923 - mr.svm OR crm.svm: when vec2/3/4 set, sub-vector mapreduce is enabled
925 # Parallel-reduction algorithm
927 The principle of SVP64 is that SVP64 is a fully-independent
928 Abstraction of hardware-looping in between issue and execute phases
929 that has no relation to the operation it issues.
930 Additional state cannot be saved on context-switching beyond that
931 of SVSTATE, making things slightly tricky.
933 Executable demo pseudocode, full version
934 [here](https://git.libre-soc.org/?p=libreriscv.git;a=blob;f=openpower/sv/test_preduce.py;hb=HEAD)
937 [[!inline pages="openpower/sv/preduce.py" raw="yes" ]]
940 This algorithm works by noting when data remains in-place rather than
941 being reduced, and referring to that alternative position on subsequent
942 layers of reduction. It is re-entrant. If however interrupted and
943 restored, some implementations may take longer to re-establish the
946 Its application by default is that:
948 * RA, FRA or BFA is the first register as the first operand
949 (ci index offset in the above pseudocode)
950 * RB, FRB or BFB is the second (co index offset)
951 * RT (result) also uses ci **if RA==RT**
953 For more complex applications a REMAP Schedule must be used
956 if passed a predicate mask with only one bit set, this algorithm
957 takes no action, similar to when a predicate mask is all zero.*
959 *Implementor's Note: many SIMD-based Parallel Reduction Algorithms are
960 implemented in hardware with MVs that ensure lane-crossing is minimised.
961 The mistake which would be catastrophic to SVP64 to make is to then
962 limit the Reduction Sequence for all implementors
963 based solely and exclusively on what one
964 specific internal microarchitecture does.
965 In SIMD ISAs the internal SIMD Architectural design is exposed and imposed on the programmer. Cray-style Vector ISAs on the other hand provide convenient,
966 compact and efficient encodings of abstract concepts.*
967 **It is the Implementor's responsibility to produce a design
968 that complies with the above algorithm,
969 utilising internal Micro-coding and other techniques to transparently
970 insert micro-architectural lane-crossing Move operations
971 if necessary or desired, to give the level of efficiency or performance
974 # Element-width overrides <a name="elwidth"> </>
976 Element-width overrides are best illustrated with a packed structure
977 union in the c programming language. The following should be taken
978 literally, and assume always a little-endian layout:
985 uint8_t actual_bytes[8];
988 elreg_t int_regfile[128];
990 get_polymorphed_reg(reg, bitwidth, offset):
992 res.l = 0; // TODO: going to need sign-extending / zero-extending
994 reg.b = int_regfile[reg].b[offset]
996 reg.s = int_regfile[reg].s[offset]
998 reg.i = int_regfile[reg].i[offset]
1000 reg.l = int_regfile[reg].l[offset]
1003 set_polymorphed_reg(reg, bitwidth, offset, val):
1005 # not a vector: first element only, overwrites high bits
1006 int_regfile[reg].l[0] = val
1008 int_regfile[reg].b[offset] = val
1009 elif bitwidth == 16:
1010 int_regfile[reg].s[offset] = val
1011 elif bitwidth == 32:
1012 int_regfile[reg].i[offset] = val
1013 elif bitwidth == 64:
1014 int_regfile[reg].l[offset] = val
1016 In effect the GPR registers r0 to r127 (and corresponding FPRs fp0
1017 to fp127) are reinterpreted to be "starting points" in a byte-addressable
1018 memory. Vectors - which become just a virtual naming construct - effectively
1021 It is extremely important for implementors to note that the only circumstance
1022 where upper portions of an underlying 64-bit register are zero'd out is
1023 when the destination is a scalar. The ideal register file has byte-level
1024 write-enable lines, just like most SRAMs, in order to avoid READ-MODIFY-WRITE.
1026 An example ADD operation with predication and element width overrides:
1028 for (i = 0; i < VL; i++)
1029 if (predval & 1<<i) # predication
1030 src1 = get_polymorphed_reg(RA, srcwid, irs1)
1031 src2 = get_polymorphed_reg(RB, srcwid, irs2)
1032 result = src1 + src2 # actual add here
1033 set_polymorphed_reg(RT, destwid, ird, result)
1034 if (!RT.isvec) break
1035 if (RT.isvec) { id += 1; }
1036 if (RA.isvec) { irs1 += 1; }
1037 if (RB.isvec) { irs2 += 1; }
1039 Thus it can be clearly seen that elements are packed by their
1040 element width, and the packing starts from the source (or destination)
1041 specified by the instruction.
1043 # Twin (implicit) result operations
1045 Some operations in the Power ISA already target two 64-bit scalar
1046 registers: `lq` for example, and LD with update.
1047 Some mathematical algorithms are more
1048 efficient when there are two outputs rather than one, providing
1049 feedback loops between elements (the most well-known being add with
1050 carry). 64-bit multiply
1051 for example actually internally produces a 128 bit result, which clearly
1052 cannot be stored in a single 64 bit register. Some ISAs recommend
1053 "macro op fusion": the practice of setting a convention whereby if
1054 two commonly used instructions (mullo, mulhi) use the same ALU but
1055 one selects the low part of an identical operation and the other
1056 selects the high part, then optimised micro-architectures may
1057 "fuse" those two instructions together, using Micro-coding techniques,
1060 The practice and convention of macro-op fusion however is not compatible
1061 with SVP64 Horizontal-First, because Horizontal Mode may only
1062 be applied to a single instruction at a time, and SVP64 is based on
1063 the principle of strict Program Order even at the element
1064 level. Thus it becomes
1065 necessary to add explicit more complex single instructions with
1066 more operands than would normally be seen in the average RISC ISA
1067 (3-in, 2-out, in some cases). If it
1068 was not for Power ISA already having LD/ST with update as well as
1069 Condition Codes and `lq` this would be hard to justify.
1071 With limited space in the `EXTRA` Field, and Power ISA opcodes
1072 being only 32 bit, 5 operands is quite an ask. `lq` however sets
1073 a precedent: `RTp` stands for "RT pair". In other words the result
1074 is stored in RT and RT+1. For Scalar operations, following this
1075 precedent is perfectly reasonable. In Scalar mode,
1076 `madded` therefore stores the two halves of the 128-bit multiply
1079 What, then, of `sv.madded`? If the destination is hard-coded to
1080 RT and RT+1 the instruction is not useful when Vectorised because
1081 the output will be overwritten on the next element. To solve this
1082 is easy: define the destination registers as RT and RT+MAXVL
1083 respectively. This makes it easy for compilers to statically allocate
1084 registers even when VL changes dynamically.
1086 Bear in mind that both RT and RT+MAXVL are starting points for Vectors,
1087 and bear in mind that element-width overrides still have to be taken
1088 into consideration, the starting point for the implicit destination
1089 is best illustrated in pseudocode:
1092 for (i = 0; i < VL; i++)
1093 if (predval & 1<<i) # predication
1094 src1 = get_polymorphed_reg(RA, srcwid, irs1)
1095 src2 = get_polymorphed_reg(RB, srcwid, irs2)
1096 src2 = get_polymorphed_reg(RC, srcwid, irs3)
1097 result = src1*src2 + src2
1098 destmask = (2<<destwid)-1
1099 # store two halves of result, both start from RT.
1100 set_polymorphed_reg(RT, destwid, ird , result&destmask)
1101 set_polymorphed_reg(RT, destwid, ird+MAXVL, result>>destwid)
1102 if (!RT.isvec) break
1103 if (RT.isvec) { id += 1; }
1104 if (RA.isvec) { irs1 += 1; }
1105 if (RB.isvec) { irs2 += 1; }
1106 if (RC.isvec) { irs3 += 1; }
1108 The significant part here is that the second half is stored
1109 starting not from RT+MAXVL at all: it is the *element* index
1110 that is offset by MAXVL, both halves actually starting from RT.
1111 If VL is 3, MAXVL is 5, RT is 1, and dest elwidth is 32 then the elements
1112 RT0 to RT2 are stored:
1115 r0 unchanged unchanged
1120 r5 unchanged unchanged
1122 Note that all of the LO halves start from r1, but that the HI halves
1123 start from half-way into r3. The reason is that with MAXVL bring
1124 5 and elwidth being 32, this is the 5th element
1125 offset (in 32 bit quantities) counting from r1.
1127 *Programmer's note: accessing registers that have been placed
1128 starting on a non-contiguous boundary (half-way along a scalar
1129 register) can be inconvenient: REMAP can provide an offset but
1130 it requires extra instructions to set up. A simple solution
1131 is to ensure that MAXVL is rounded up such that the Vector
1132 ends cleanly on a contiguous register boundary. MAXVL=6 in
1133 the above example would achieve that*
1135 Additional DRAFT Scalar instructions in 3-in 2-out form
1136 with an implicit 2nd destination:
1138 * [[isa/svfixedarith]]