1 # Variable-width Variable-packed SIMD / Simple-V / Parallelism Extension Proposal
5 This proposal exists so as to be able to satisfy several disparate
6 requirements: power-conscious, area-conscious, and performance-conscious
7 designs all pull an ISA and its implementation in different conflicting
8 directions, as do the specific intended uses for any given implementation.
10 Additionally, the existing P (SIMD) proposal and the V (Vector) proposals,
11 whilst each extremely powerful in their own right and clearly desirable,
14 * Clearly independent in their origins (Cray and AndeStar v3 respectively)
15 so need work to adapt to the RISC-V ethos and paradigm
16 * Are sufficiently large so as to make adoption (and exploration for
17 analysis and review purposes) prohibitively expensive
18 * Both contain partial duplication of pre-existing RISC-V instructions
19 (an undesirable characteristic)
20 * Both have independent and disparate methods for introducing parallelism
21 at the instruction level.
22 * Both require that their respective parallelism paradigm be implemented
23 along-side and integral to their respective functionality *or not at all*.
24 * Both independently have methods for introducing parallelism that
25 could, if separated, benefit
26 *other areas of RISC-V not just DSP or Floating-point respectively*.
28 Therefore it makes a huge amount of sense to have a means and method
29 of introducing instruction parallelism in a flexible way that provides
30 implementors with the option to choose exactly where they wish to offer
31 performance improvements and where they wish to optimise for power
32 and/or area (and if that can be offered even on a per-operation basis that
33 would provide even more flexibility).
35 Additionally it makes sense to *split out* the parallelism inherent within
36 each of P and V, and to see if each of P and V then, in *combination* with
37 a "best-of-both" parallelism extension, could be added on *on top* of
38 this proposal, to topologically provide the exact same functionality of
41 Furthermore, an additional goal of this proposal is to reduce the number
42 of opcodes utilised by each of P and V as they currently stand, leveraging
43 existing RISC-V opcodes where possible, and also potentially allowing
44 P and V to make use of Compressed Instructions as a result.
46 **TODO**: reword this to better suit this document:
48 Having looked at both P and V as they stand, they're _both_ very much
49 "separate engines" that, despite both their respective merits and
50 extremely powerful features, don't really cleanly fit into the RV design
51 ethos (or the flexible extensibility) and, as such, are both in danger
52 of not being widely adopted. I'm inclined towards recommending:
54 * splitting out the DSP aspects of P-SIMD to create a single-issue DSP
55 * splitting out the polymorphism, esoteric data types (GF, complex
56 numbers) and unusual operations of V to create a single-issue "Esoteric
57 Floating-Point" extension
58 * splitting out the loop-aspects, vector aspects and data-width aspects
59 of both P and V to a *new* "P-SIMD / Simple-V" and requiring that they
60 apply across *all* Extensions, whether those be DSP, M, Base, V, P -
63 **TODO**: propose overflow registers be actually one of the integer regs
64 (flowing to multiple regs).
66 **TODO**: propose "mask" (predication) registers likewise. combination with
67 standard RV instructions and overflow registers extremely powerful
69 ## CSRs marking registers as Vector
71 A 32-bit CSR would be needed (1 bit per integer register) to indicate
72 whether a register was, if referred to, implicitly to be treated as
75 A second 32-bit CSR would be needed (1 bit per floating-point register)
76 to indicate whether a floating-point register was to be treated as a
79 In this way any standard (current or future) operation involving
80 register operands may detect if the operation is to be vector-vector,
81 vector-scalar or scalar-scalar (standard) simply through a single
84 ## CSR vector-length and CSR SIMD packed-bitwidth
86 **TODO** analyse each of these:
88 * splitting out the loop-aspects, vector aspects and data-width aspects
89 * integer reg 0 *and* fp reg0 share CSR vlen 0 *and* CSR packed-bitwidth 0
90 * integer reg 1 *and* fp reg1 share CSR vlen 1 *and* CSR packed-bitwidth 1
96 * CSR vlen 0 *and* CSR packed-bitwidth 0 register contain extra bits
97 specifying an *INDEX* of WHICH int/fp register they refer to
98 * CSR vlen 1 *and* CSR packed-bitwidth 1 register contain extra bits
99 specifying an *INDEX* of WHICH int/fp register they refer to
103 Have to be very *very* careful about not implementing too few of those
104 (or too many). Assess implementation impact on decode latency. Is it
107 Implementation of the latter:
109 Operation involving (referring to) register M:
111 bitwidth = default # default for opcode?
112 vectorlen = 1 # scalar
114 for (o = 0, o < 2, o++)
115 if (CSR-Vector_registernum[o] == M)
116 bitwidth = CSR-Vector_bitwidth[o]
117 vectorlen = CSR-Vector_len[o]
120 and for the former it would simply be:
122 bitwidth = CSR-Vector_bitwidth[M]
123 vectorlen = CSR-Vector_len[M]
127 * One single "global" vector-length CSR
131 **TODO**: propose two LOAD/STORE offset CSRs, which mark a particular
132 register as being "if you use this reg in LOAD/STORE, use the offset
133 amount CSRoffsN (N=0,1) instead of treating LOAD/STORE as contiguous".
134 can be used for matrix spanning.
136 > For LOAD/STORE, could a better option be to interpret the offset in the
137 > opcode as a stride instead, so "LOAD t3, 12(t2)" would, if t3 is
138 > configured as a length-4 vector base, result in t3 = *t2, t4 = *(t2+12),
139 > t5 = *(t2+24), t6 = *(t2+32)? Perhaps include a bit in the
140 > vector-control CSRs to select between offset-as-stride and unit-stride
143 So there would be an instruction like this:
145 | SETOFF | On=rN | OBank={float|int} | Smode={offs|unit} | OFFn=rM |
146 | opcode | 5 bit | 1 bit | 1 bit | 5 bit, OFFn=XLEN |
151 * CSR-Offset register n <= (float|int) register number N
152 * CSR-Offset Stride-mode = offset or unit
153 * CSR-Offset amount register n = contents of register M
155 LOAD rN, ldoffs(rM) would then be (assuming packed bit-width not set):
159 vector-len = CSR-Vector-length register N
161 for (o = 0, o < 2, o++)
162 if (CSR-Offset register o == M)
163 offs = CSR-Offset amount register o
164 if CSR-Offset Stride-mode == offset:
168 for (i = 0, i < vector-len; i++)
169 r[N+i] = mem[(offs*i + r[M+i])*stride]
171 # Analysis and discussion of Vector vs SIMD
173 There are four combined areas between the two proposals that help with
174 parallelism without over-burdening the ISA with a huge proliferation of
177 * Fixed vs variable parallelism (fixed or variable "M" in SIMD)
178 * Implicit vs fixed instruction bit-width (integral to instruction or not)
179 * Implicit vs explicit type-conversion (compounded on bit-width)
180 * Implicit vs explicit inner loops.
181 * Masks / tagging (selecting/preventing certain indexed elements from execution)
183 The pros and cons of each are discussed and analysed below.
185 ## Fixed vs variable parallelism length
187 In David Patterson and Andrew Waterman's analysis of SIMD and Vector
188 ISAs, the analysis comes out clearly in favour of (effectively) variable
189 length SIMD. As SIMD is a fixed width, typically 4, 8 or in extreme cases
190 16 or 32 simultaneous operations, the setup, teardown and corner-cases of SIMD
191 are extremely burdensome except for applications whose requirements
192 *specifically* match the *precise and exact* depth of the SIMD engine.
194 Thus, SIMD, no matter what width is chosen, is never going to be acceptable
195 for general-purpose computation, and in the context of developing a
196 general-purpose ISA, is never going to satisfy 100 percent of implementors.
198 That basically leaves "variable-length vector" as the clear *general-purpose*
199 winner, at least in terms of greatly simplifying the instruction set,
200 reducing the number of instructions required for any given task, and thus
201 reducing power consumption for the same.
203 ## Implicit vs fixed instruction bit-width
205 SIMD again has a severe disadvantage here, over Vector: huge proliferation
206 of specialist instructions that target 8-bit, 16-bit, 32-bit, 64-bit, and
207 have to then have operations *for each and between each*. It gets very
210 The V-Extension on the other hand proposes to set the bit-width of
211 future instructions on a per-register basis, such that subsequent instructions
212 involving that register are *implicitly* of that particular bit-width until
213 otherwise changed or reset.
215 This has some extremely useful properties, without being particularly
216 burdensome to implementations, given that instruction decode already has
217 to direct the operation to a correctly-sized width ALU engine, anyway.
219 Not least: in places where an ISA was previously constrained (due for
220 whatever reason, including limitations of the available operand spcace),
221 implicit bit-width allows the meaning of certain operations to be
222 type-overloaded *without* pollution or alteration of frozen and immutable
223 instructions, in a fully backwards-compatible fashion.
225 ## Implicit and explicit type-conversion
227 The Draft 2.3 V-extension proposal has (deprecated) polymorphism to help
228 deal with over-population of instructions, such that type-casting from
229 integer (and floating point) of various sizes is automatically inferred
230 due to "type tagging" that is set with a special instruction. A register
231 will be *specifically* marked as "16-bit Floating-Point" and, if added
232 to an operand that is specifically tagged as "32-bit Integer" an implicit
233 type-conversion will take placce *without* requiring that type-conversion
234 to be explicitly done with its own separate instruction.
236 However, implicit type-conversion is not only quite burdensome to
237 implement (explosion of inferred type-to-type conversion) but also is
238 never really going to be complete. It gets even worse when bit-widths
239 also have to be taken into consideration.
241 Overall, type-conversion is generally best to leave to explicit
242 type-conversion instructions, or in definite specific use-cases left to
243 be part of an actual instruction (DSP or FP)
245 ## Zero-overhead loops vs explicit loops
247 The initial Draft P-SIMD Proposal by Chuanhua Chang of Andes Technology
248 contains an extremely interesting feature: zero-overhead loops. This
249 proposal would basically allow an inner loop of instructions to be
250 repeated indefinitely, a fixed number of times.
252 Its specific advantage over explicit loops is that the pipeline in a
253 DSP can potentially be kept completely full *even in an in-order
254 implementation*. Normally, it requires a superscalar architecture and
255 out-of-order execution capabilities to "pre-process" instructions in order
256 to keep ALU pipelines 100% occupied.
258 This very simple proposal offers a way to increase pipeline activity in the
259 one key area which really matters: the inner loop.
261 ## Mask and Tagging (Predication)
263 Tagging (aka Masks aka Predication) is a pseudo-method of implementing
264 simplistic branching in a parallel fashion, by allowing execution on
265 elements of a vector to be switched on or off depending on the results
266 of prior operations in the same array position.
268 The reason for considering this is simple: by *definition* it
269 is not possible to perform individual parallel branches in a SIMD
270 (Single-Instruction, **Multiple**-Data) context. Branches (modifying
271 of the Program Counter) will result in *all* parallel data having
272 a different instruction executed on it: that's just the definition of
273 SIMD, and it is simply unavoidable.
275 So these are the ways in which conditional execution may be implemented:
277 * explicit compare and branch: BNE x, y -> offs would jump offs
278 instructions if x was not equal to y
279 * explicit store of tag condition: CMP x, y -> tagbit
280 * implicit (condition-code) ADD results in a carry, carry bit implicitly
281 (or sometimes explicitly) goes into a "tag" (mask) register
283 The first of these is a "normal" branch method, which is flat-out impossible
284 to parallelise without look-ahead and effectively rewriting instructions.
285 This would defeat the purpose of RISC.
287 The latter two are where parallelism becomes easy to do without complexity:
288 every operation is modified to be "conditionally executed" (in an explicit
289 way directly in the instruction format *or* implicitly).
291 RVV (Vector-Extension) proposes to have *explicit* storing of the compare
292 in a tag/mask register, and to *explicitly* have every vector operation
293 *require* that its operation be "predicated" on the bits within an
294 explicitly-named tag/mask register.
296 SIMD (P-Extension) has not yet published precise documentation on what its
297 schema is to be: there is however verbal indication at the time of writing
300 > The "compare" instructions in the DSP/SIMD ISA proposed by Andes will
301 > be executed using the same compare ALU logic for the base ISA with some
302 > minor modifications to handle smaller data types. The function will not
305 This is an *implicit* form of predication as the base RV ISA does not have
306 condition-codes or predication. By adding a CSR it becomes possible
307 to also tag certain registers as "predicated if referenced as a destination".
310 // in future operations if r0 is the destination use r5 as
311 // the PREDICATION register
312 IMPLICICSRPREDICATE r0, r5
313 // store the compares in r5 as the PREDICATION register
315 // r0 is used here. ah ha! that means it's predicated using r5!
318 With enough registers (and there are enough registers) some fairly
319 complex predication can be set up and yet still execute without significant
320 stalling, even in a simple non-superscalar architecture.
322 ### Retro-fitting Predication into branch-explicit ISA
324 One of the goals of this parallelism proposal is to avoid instruction
325 duplication. However, with the base ISA having been designed explictly
326 to *avoid* condition-codes entirely, shoe-horning predication into it
327 bcomes quite challenging.
329 However what if all branch instructions, if referencing a vectorised
330 register, were instead given *completely new analogous meanings* that
331 resulted in a parallel bit-wise predication register being set? This
332 would have to be done for both C.BEQZ and C.BNEZ, as well as BEQ, BNE,
335 We might imagine that FEQ, FLT and FLT would also need to be converted,
336 however these are effectively *already* in the precise form needed and
337 do not need to be converted *at all*! The difference is that FEQ, FLT
338 and FLE *specifically* write a 1 to an integer register if the condition
339 holds, and 0 if not. All that needs to be done here is to say, "if
340 the integer register is tagged with a bit that says it is a predication
341 register, the **bit** in the integer register is set based on the
342 current vector index" instead.
344 There is, in the standard Conditional Branch instruction, more than
345 adequate space to interpret it in a similar fashion:
348 31 |30 ..... 25 |24 ... 20 | 19 ... 15 | 14 ...... 12 | 11 ....... 8 | 7 | 6 ....... 0 |
349 imm[12] | imm[10:5] | rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
350 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
351 offset[12,10:5] || src2 | src1 | BEQ | offset[11,4:1] || BRANCH |
357 31 |30 ..... 25 |24 ... 20 | 19 ... 15 | 14 ...... 12 | 11 ....... 8 | 7 | 6 ....... 0 |
358 imm[12] | imm[10:5] | rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
359 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
360 reserved || src2 | src1 | BEQ | predicate rs3 || BRANCH |
363 Similarly the C.BEQZ and C.BNEZ instruction format may be retro-fitted,
364 with the interesting side-effect that there is space within what is presently
365 the "immediate offset" field to reinterpret that to add in not only a bit
366 field to distinguish between floating-point compare and integer compare,
367 not only to add in a second source register, but also use some of the bits as
368 a predication target as well.
371 15 ...... 13 | 12 ........... 10 | 9..... 7 | 6 ................. 2 | 1 .. 0 |
372 funct3 | imm | rs10 | imm | op |
374 C.BEQZ | offset[8,4:3] | src | offset[7:6,2:1,5] | C1 |
377 Now uses the CS format:
380 15 ...... 13 | 12 ........... 10 | 9..... 7 | 6 .. 5 | 4......... 2 | 1 .. 0 |
381 funct3 | imm | rs10 | imm | | op |
382 3 | 3 | 3 | 2 | 3 | 2 |
383 C.BEQZ | predicate rs3 | src1 | I/F B | src2 | C1 |
386 Bit 6 would be decoded as "operation refers to Integer or Float" including
387 interpreting src1 and src2 accordingly as outlined in Table 12.2 of the
388 "C" Standard, version 2.0,
389 whilst Bit 5 would allow the operation to be extended, in combination with
390 funct3 = 110 or 111: a combination of four distinct comparison operators.
394 In the above sections the five different ways where parallel instruction
395 execution has closely and loosely inter-related implications for the ISA and
396 for implementors, were outlined. The pluses and minuses came out as
399 * Fixed vs variable parallelism: <b>variable</b>
400 * Implicit (indirect) vs fixed (integral) instruction bit-width: <b>indirect</b>
401 * Implicit vs explicit type-conversion: <b>explicit</b>
402 * Implicit vs explicit inner loops: <b>implicit</b>
403 * Tag or no-tag: <b>Complex and needs further thought</b>
405 In particular: variable-length vectors came out on top because of the
406 high setup, teardown and corner-cases associated with the fixed width
407 of SIMD. Implicit bit-width helps to extend the ISA to escape from
408 former limitations and restrictions (in a backwards-compatible fashion),
409 and implicit (zero-overhead) loops provide a means to keep pipelines
410 potentially 100% occupied *without* requiring a super-scalar or out-of-order
413 Constructing a SIMD/Simple-Vector proposal based around even only these four
414 (five?) requirements would therefore seem to be a logical thing to do.
418 **TODO** *basically borrow from both P and V, which should be quite simple
419 to do, with the exception of Tag/no-tag, which needs a bit more
420 thought. V's Section 17.19 of Draft V2.3 spec is reminiscent of B's BGS
421 gather-scatterer, and, if implemented, could actually be a really useful
422 way to span 8-bit up to 64-bit groups of data, where BGS as it stands
423 and described by Clifford does **bits** of up to 16 width. Lots to
424 look at and investigate!*
426 # Note on implementation of parallelism
428 One extremely important aspect of this proposal is to respect and support
429 implementors desire to focus on power, area or performance. In that regard,
430 it is proposed that implementors be free to choose whether to implement
431 the Vector (or variable-width SIMD) parallelism as sequential operations
432 with a single ALU, fully parallel (if practical) with multiple ALUs, or
433 a hybrid combination of both.
435 In Broadcom's Videocore-IV, they chose hybrid, and called it "Virtual
436 Parallelism". They achieve a 16-way SIMD at an **instruction** level
437 by providing a combination of a 4-way parallel ALU *and* an externally
438 transparent loop that feeds 4 sequential sets of data into each of the
441 Also in the same core, it is worth noting that particularly uncommon
442 but essential operations (Reciprocal-Square-Root for example) are
443 *not* part of the 4-way parallel ALU but instead have a *single* ALU.
444 Under the proposed Vector (varible-width SIMD) implementors would
445 be free to do precisely that: i.e. free to choose *on a per operation
446 basis* whether and how much "Virtual Parallelism" to deploy.
448 It is absolutely critical to note that it is proposed that such choices MUST
449 be **entirely transparent** to the end-user and the compiler. Whilst
450 a Vector (varible-width SIM) may not precisely match the width of the
451 parallelism within the implementation, the end-user **should not care**
452 and in this way the performance benefits are gained but the ISA remains
453 straightforward. All that happens at the end of an instruction run is: some
454 parallel units (if there are any) would remain offline, completely
455 transparently to the ISA, the program, and the compiler.
457 The "SIMD considered harmful" trap of having huge complexity and extra
458 instructions to deal with corner-cases is thus avoided, and implementors
459 get to choose precisely where to focus and target the benefits of their
460 implementation efforts, without "extra baggage".
462 # Example of vector / vector, vector / scalar, scalar / scalar => vector add
464 register CSRvectorlen[XLEN][4]; # not quite decided yet about this one...
465 register CSRpredicate[XLEN][4]; # 2^4 is max vector length
466 register CSRreg_is_vectorised[XLEN]; # just for fun support scalars as well
467 register x[32][XLEN];
469 function op_add(rd, rs1, rs2, predr)
471 /* note that this is ADD, not PADD */
472 int i, id, irs1, irs2;
473 # checks CSRvectorlen[rd] == CSRvectorlen[rs] etc. ignored
474 # also destination makes no sense as a scalar but what the hell...
475 for (i = 0, id=0, irs1=0, irs2=0; i<CSRvectorlen[rd]; i++)
476 if (CSRpredicate[predr][i]) # i *think* this is right...
477 x[rd+id] <= x[rs1+irs1] + x[rs2+irs2];
478 # now increment the idxs
479 if (CSRreg_is_vectorised[rd]) # bitfield check rd, scalar/vector?
481 if (CSRreg_is_vectorised[rs1]) # bitfield check rs1, scalar/vector?
483 if (CSRreg_is_vectorised[rs2]) # bitfield check rs2, scalar/vector?
487 # V-Extension to Simple-V Comparative Analysis
489 This section covers the ways in which Simple-V is comparable
490 to, or more flexible than, V-Extension (V2.3-draft). Also covered is
491 one major weak-point (register files are fixed size, where V is
492 arbitrary length), and how best to deal with that, should V be adapted
493 to be on top of Simple-V.
495 The first stages of this section go over each of the sections of V2.3-draft V
498 ## 17.3 Shape Encoding
500 Simple-V's proposed means of expressing whether a register (from the
501 standard integer or the standard floating-point file) is a scalar or
502 a vector is to simply set the vector length to 1. The instruction
503 would however have to specify which register file (integer or FP) that
504 the vector-length was to be applied to.
506 Extended shapes (2-D etc) would not be part of Simple-V at all.
508 ## 17.4 Representation Encoding
510 Simple-V would not have representation-encoding. This is part of
511 polymorphism, which is considered too complex to implement (TODO: confirm?)
513 ## 17.5 Element Bitwidth
515 This is directly equivalent to Simple-V's "Packed", and implies that
516 integer (or floating-point) are divided down into vector-indexable
517 chunks of size Bitwidth.
519 In this way it becomes possible to have ADD effectively and implicitly
520 turn into ADDb (8-bit add), ADDw (16-bit add) and so on, and where
521 vector-length has been set to greater than 1, it becomes a "Packed"
524 It remains to be decided what should be done when RV32 / RV64 ADD (sized)
525 opcodes are used. One useful idea would be, on an RV64 system where
526 a 32-bit-sized ADD was performed, to simply use the least significant
527 32-bits of the register (exactly as is currently done) but at the same
528 time to *respect the packed bitwidth as well*.
530 The extended encoding (Table 17.6) would not be part of Simple-V.
532 ## 17.6 Base Vector Extension Supported Types
534 TODO: analyse. probably exactly the same.
536 ## 17.7 Maximum Vector Element Width
538 No equivalent in Simple-V
540 ## 17.8 Vector Configuration Registers
544 ## 17.9 Legal Vector Unit Configurations
548 ## 17.10 Vector Unit CSRs
552 > Ok so this is an aspect of Simple-V that I hadn't thought through,
553 > yet (proposal / idea only a few days old!). in V2.3-Draft ISA Section
554 > 17.10 the CSRs are listed. I note that there's some general-purpose
555 > CSRs (including a global/active vector-length) and 16 vcfgN CSRs. i
556 > don't precisely know what those are for.
558 > In the Simple-V proposal, *every* register in both the integer
559 > register-file *and* the floating-point register-file would have at
560 > least a 2-bit "data-width" CSR and probably something like an 8-bit
561 > "vector-length" CSR (less in RV32E, by exactly one bit).
563 > What I *don't* know is whether that would be considered perfectly
564 > reasonable or completely insane. If it turns out that the proposed
565 > Simple-V CSRs can indeed be stored in SRAM then I would imagine that
566 > adding somewhere in the region of 10 bits per register would be... okay?
567 > I really don't honestly know.
569 > Would these proposed 10-or-so-bit per-register Simple-V CSRs need to
570 > be multi-ported? No I don't believe they would.
572 ## 17.11 Maximum Vector Length (MVL)
574 Basically implicitly this is set to the maximum size of the register
575 file multiplied by the number of 8-bit packed ints that can fit into
576 a register (4 for RV32, 8 for RV64 and 16 for RV128).
578 ## !7.12 Vector Instruction Formats
580 No equivalent in Simple-V because *all* instructions of *all* Extensions
581 are implicitly parallelised (and packed).
583 ## 17.13 Polymorphic Vector Instructions
585 Polymorphism (implicit type-casting) is deliberately not supported
588 ## 17.14 Rapid Configuration Instructions
590 TODO: analyse if this is useful to have an equivalent in Simple-V
592 ## 17.15 Vector-Type-Change Instructions
594 TODO: analyse if this is useful to have an equivalent in Simple-V
596 ## 17.16 Vector Length
598 Has a direct corresponding equivalent.
600 ## 17.17 Predicated Execution
602 Predicated Execution is another name for "masking" or "tagging". Masked
603 (or tagged) implies that there is a bit field which is indexed, and each
604 bit associated with the corresponding indexed offset register within
605 the "Vector". If the tag / mask bit is 1, when a parallel operation is
606 issued, the indexed element of the vector has the operation carried out.
607 However if the tag / mask bit is *zero*, that particular indexed element
608 of the vector does *not* have the requested operation carried out.
610 In V2.3-draft V, there is a significant (not recommended) difference:
611 the zero-tagged elements are *set to zero*. This loses a *significant*
612 advantage of mask / tagging, particularly if the entire mask register
613 is itself a general-purpose register, as that general-purpose register
614 can be inverted, shifted, and'ed, or'ed and so on. In other words
615 it becomes possible, especially if Carry/Overflow from each vector
616 operation is also accessible, to do conditional (step-by-step) vector
617 operations including things like turn vectors into 1024-bit or greater
618 operands with very few instructions, by treating the "carry" from
619 one instruction as a way to do "Conditional add of 1 to the register
620 next door". If V2.3-draft V sets zero-tagged elements to zero, such
621 extremely powerful techniques are simply not possible.
623 It is noted that there is no mention of an equivalent to BEXT (element
624 skipping) which would be particularly fascinating and powerful to have.
625 In this mode, the "mask" would skip elements where its mask bit was zero
626 in either the source or the destination operand.
628 Lots to be discussed.
630 ## 17.18 Vector Load/Store Instructions
632 The Vector Load/Store instructions as proposed in V are extremely powerful
633 and can be used for reordering and regular restructuring.
637 if (unit-strided) stride = elsize;
638 else stride = areg[as2]; // constant-strided
639 for (int i=0; i<vl; ++i)
641 for (int j=0; j<seglen+1; j++)
642 vreg[vd+j][i] = mem[areg[as1] + (i*(seglen+1)+j)*stride];
646 if (unit-strided) stride = elsize;
647 else stride = areg[as2]; // constant-strided
648 for (int i=0; i<vl; ++i)
650 for (int j=0; j<seglen+1; j++)
651 mem[areg[base] + (i*(seglen+1)+j)*stride] = vreg[vd+j][i];
655 for (int i=0; i<vl; ++i)
657 for (int j=0; j<seglen+1; j++)
658 vreg[vd+j][i] = mem[sreg[base] + vreg[vs2][i] + j*elsize];
662 for (int i=0; i<vl; ++i)
664 for (int j=0; j<seglen+1; j++)
665 mem[sreg[base] + vreg[vs2][i] + j*elsize] = vreg[vd+j][i];
667 Keeping these instructions as-is for Simple-V is highly recommended.
668 However: one of the goals of this Extension is to retro-fit (re-use)
669 existing RV Load/Store:
672 31 20 | 19 15 | 14 12 | 11 7 | 6 0 |
673 imm[11:0] | rs1 | funct3 | rd | opcode |
675 offset[11:0] | base | width | dest | LOAD |
679 31 25 | 24 20 | 19 15 | 14 12 | 11 7 | 6 0 |
680 imm[11:5] | rs2 | rs1 | funct3 | imm[4:0] | opcode |
681 7 | 5 | 5 | 3 | 5 | 7 |
682 offset[11:5] | src | base | width | offset[4:0] | STORE |
686 ## 17.19 Vector Register Gather
692 > However, there are also several features that go beyond simply attaching VL
693 > to a scalar operation and are crucial to being able to vectorize a lot of
694 > code. To name a few:
695 > - Conditional execution (i.e., predicated operations)
696 > - Inter-lane data movement (e.g. SLIDE, SELECT)
697 > - Reductions (e.g., VADD with a scalar destination)
699 Ok so the Conditional and also the Reductions is one of the reasons
700 why as part of SimpleV / variable-SIMD / parallelism (gah gotta think
701 of a decent name) i proposed that it be implemented as "if you say r0
702 is to be a vector / SIMD that means operations actually take place on
705 Consequently any parallel operation could be paused (or... more
706 specifically: vectors disabled by resetting it back to a default /
707 scalar / vector-length=1) yet the results would actually be in the
708 *main register file* (integer or float) and so anything that wasn't
709 possible to easily do in "simple" parallel terms could be done *out*
710 of parallel "mode" instead.
712 I do appreciate that the above does imply that there is a limit to the
713 length that SimpleV (whatever) can be parallelised, namely that you
714 run out of registers! my thought there was, "leave space for the main
715 V-Ext proposal to extend it to the length that V currently supports".
716 Honestly i had not thought through precisely how that would work.
718 Inter-lane (SELECT) i saw 17.19 in V2.3-Draft p117, I liked that,
719 it reminds me of the discussion with Clifford on bit-manipulation
720 (gather-scatter except not Bit Gather Scatter, *data* gather scatter): if
721 applied "globally and outside of V and P" SLIDE and SELECT might become
722 an extremely powerful way to do fast memory copy and reordering [2[.
724 However I haven't quite got my head round how that would work: i am
725 used to the concept of register "tags" (the modern term is "masks")
726 and i *think* if "masks" were applied to a Simple-V-enhanced LOAD /
727 STORE you would get the exact same thing as SELECT.
729 SLIDE you could do simply by setting say r0 vector-length to say 16
730 (meaning that if referred to in any operation it would be an implicit
731 parallel operation on *all* registers r0 through r15), and temporarily
732 set say.... r7 vector-length to say... 5. Do a LOAD on r7 and it would
733 implicitly mean "load from memory into r7 through r11". Then you go
734 back and do an operation on r0 and ta-daa, you're actually doing an
735 operation on a SLID {SLIDED?) vector.
737 The advantage of Simple-V (whatever) over V would be that you could
738 actually do *operations* in the middle of vectors (not just SLIDEs)
739 simply by (as above) setting r0 vector-length to 16 and r7 vector-length
740 to 5. There would be nothing preventing you from doing an ADD on r0
741 (which meant do an ADD on r0 through r15) followed *immediately in the
742 next instruction with no setup cost* a MUL on r7 (which actually meant
743 "do a parallel MUL on r7 through r11").
745 btw it's worth mentioning that you'd get scalar-vector and vector-scalar
746 implicitly by having one of the source register be vector-length 1
747 (the default) and one being N > 1. but without having special opcodes
748 to do it. i *believe* (or more like "logically infer or deduce" as
749 i haven't got access to the spec) that that would result in a further
750 opcode reduction when comparing [draft] V-Ext to [proposed] Simple-V.
752 Also, Reduction *might* be possible by specifying that the destination be
753 a scalar (vector-length=1) whilst the source be a vector. However... it
754 would be an awful lot of work to go through *every single instruction*
755 in *every* Extension, working out which ones could be parallelised (ADD,
756 MUL, XOR) and those that definitely could not (DIV, SUB). Is that worth
757 the effort? maybe. Would it result in huge complexity? probably.
758 Could an implementor just go "I ain't doing *that* as parallel!
759 let's make it virtual-parallelism (sequential reduction) instead"?
760 absolutely. So, now that I think it through, Simple-V (whatever)
761 covers Reduction as well. huh, that's a surprise.
764 > - Vector-length speculation (making it possible to vectorize some loops with
765 > unknown trip count) - I don't think this part of the proposal is written
768 Now that _is_ an interesting concept. A little scary, i imagine, with
769 the possibility of putting a processor into a hard infinite execution
773 > Also, note the vector ISA consumes relatively little opcode space (all the
774 > arithmetic fits in 7/8ths of a major opcode). This is mainly because data
775 > type and size is a function of runtime configuration, rather than of opcode.
777 yes. i love that aspect of V, i am a huge fan of polymorphism [1]
778 which is why i am keen to advocate that the same runtime principle be
779 extended to the rest of the RISC-V ISA [3]
781 Yikes that's a lot. I'm going to need to pull this into the wiki to
782 make sure it's not lost.
784 [1] inherent data type conversion: 25 years ago i designed a hypothetical
785 hyper-hyper-hyper-escape-code-sequencing ISA based around 2-bit
786 (escape-extended) opcodes and 2-bit (escape-extended) operands that
787 only required a fixed 8-bit instruction length. that relied heavily
788 on polymorphism and runtime size configurations as well. At the time
789 I thought it would have meant one HELL of a lot of CSRs... but then I
790 met RISC-V and was cured instantly of that delusion^Wmisapprehension :)
792 [2] Interestingly if you then also add in the other aspect of Simple-V
793 (the data-size, which is effectively functionally orthogonal / identical
794 to "Packed" of Packed-SIMD), masked and packed *and* vectored LOAD / STORE
795 operations become byte / half-word / word augmenters of B-Ext's proposed
796 "BGS" i.e. where B-Ext's BGS dealt with bits, masked-packed-vectored
797 LOAD / STORE would deal with 8 / 16 / 32 bits at a time. Where it
798 would get really REALLY interesting would be masked-packed-vectored
799 B-Ext BGS instructions. I can't even get my head fully round that,
800 which is a good sign that the combination would be *really* powerful :)
802 [3] ok sadly maybe not the polymorphism, it's too complicated and I
803 think would be much too hard for implementors to easily "slide in" to an
804 existing non-Simple-V implementation. i say that despite really *really*
805 wanting IEEE 704 FP Half-precision to end up somewhere in RISC-V in some
806 fashion, for optimising 3D Graphics. *sigh*.
808 ## TODO: analyse, auto-increment on unit-stride and constant-stride
810 so i thought about that for a day or so, and wondered if it would be
811 possible to propose a variant of zero-overhead loop that included
812 auto-incrementing the two address registers a2 and a3, as well as
813 providing a means to interact between the zero-overhead loop and the
814 vsetvl instruction. a sort-of pseudo-assembly of that would look like:
816 # a2 to be auto-incremented by t0 times 4
817 zero-overhead-set-auto-increment a2, t0, 4
818 # a2 to be auto-incremented by t0 times 4
819 zero-overhead-set-auto-increment a3, t0, 4
820 zero-overhead-set-loop-terminator-condition a0 zero
821 zero-overhead-set-start-end stripmine, stripmine+endoffset
831 the question is: would something like this even be desirable? it's a
832 variant of auto-increment [1]. last time i saw any hint of auto-increment
833 register opcodes was in the 1980s... 68000 if i recall correctly... yep
836 [1] http://fourier.eng.hmc.edu/e85_old/lectures/instruction/node6.html
840 Another option for auto-increment is for vector-memory-access instructions
841 to support post-increment addressing for unit-stride and constant-stride
842 modes. This can be implemented by the scalar unit passing the operation
843 to the vector unit while itself executing an appropriate multiply-and-add
844 to produce the incremented address. This does *not* require additional
845 ports on the scalar register file, unlike scalar post-increment addressing
848 ## TODO: instructions (based on Hwacha) V-Ext duplication analysis
850 This is partly speculative due to lack of access to an up-to-date
851 V-Ext Spec (V2.3-draft RVV 0.4-Draft at the time of writing). However
852 basin an analysis instead on Hwacha, a cursory examination shows over
853 an **85%** duplication of V-Ext operand-related instructions when
854 compared to Simple-V on a standard RG64G base. Even Vector Fetch
855 is analogous to "zero-overhead loop".
859 * Vector Indexed Memory Instructions (non-contiguous)
860 * Vector Atomic Memory Instructions.
861 * Some of the Vector Misc ops: VEIDX, VFIRST, VCLASS, VPOPC
862 and potentially more.
865 Table of RV32V Instructions
867 | RV32V | RV Equivalent (FP) | RV Equivalent (Int) | Notes |
869 | VADD | FADD | ADD | |
870 | VSUB | FSUB | SUB | |
876 | VSEQ | FEQ | BEQ | {1} |
877 | VSNE | !FEQ | BNE | {1} |
878 | VSLT | FLT | BLT | {1} |
879 | VSGE | !FLE | BGE | {1} |
889 | VDIV | FDIV | DIV | |
891 | VMUL | FMUL | MUL | |
895 | VSGNJ | FSGNJ | | |
896 | VSGNJN | FSGNJN | | |
897 | VSGNJX | FSNGJX | | |
898 | VSQRT | FSQRT | | |
908 | VMADD | FMADD | | |
909 | VMSUB | FMSUB | | |
910 | VNMADD | FNMSUB | | |
911 | VNMSUB | FNMADD | | |
918 | VAMOSWAP | | AMOSWAP | |
919 | VAMOADD | | AMOADD | |
920 | VAMOAND | | AMOAND | |
921 | VAMOOR | | AMOOR | |
922 | VAMOXOR | | AMOXOR | |
923 | VAMOMIN | | AMOMIN | |
924 | VAMOMAX | | AMOMAX | |
928 * {1} retro-fit predication variants into branch instructions (base and C),
929 decoding triggered by CSR bit marking register as "Vector type".
933 > I suspect that the "hardware loop" in question is actually a zero-overhead
934 > loop unit that diverts execution from address X to address Y if a certain
937 not quite. The zero-overhead loop unit interestingly would be at
938 an [independent] level above vector-length. The distinctions are
941 * Vector-length issues *virtual* instructions where the register
942 operands are *specifically* altered (to cover a range of registers),
943 whereas zero-overhead loops *specifically* do *NOT* alter the operands
946 * Vector-length-driven "virtual" instructions are driven by *one*
947 and *only* one instruction (whether it be a LOAD, STORE, or pure
948 one/two/three-operand opcode) whereas zero-overhead loop units
949 specifically apply to *multiple* instructions.
951 Where vector-length-driven "virtual" instructions might get conceptually
952 blurred with zero-overhead loops is LOAD / STORE. In the case of LOAD /
953 STORE, to actually be useful, vector-length-driven LOAD / STORE should
954 increment the LOAD / STORE memory address to correspondingly match the
955 increment in the register bank. example:
957 * set vector-length for r0 to 4
958 * issue RV32 LOAD from addr 0x1230 to r0
960 translates effectively to:
962 * RV32 LOAD from addr 0x1230 to r0
965 * RV32 LOAD from addr 0x123B to r3
971 | Mnemonic | 16-bit Instruction | Simple-V Equivalent |
972 | ------------------ | ------------------------- | ------------------- |
973 | ADD16 rt, ra, rb | add | RV ADD (bitwidth=16) |
974 | RADD16 rt, ra, rb | Signed Halving add | |
975 | URADD16 rt, ra, rb | Unsigned Halving add | |
976 | KADD16 rt, ra, rb | Signed Saturating add | |
977 | UKADD16 rt, ra, rb | Unsigned Saturating add | |
978 | SUB16 rt, ra, rb | sub | RV SUB (bitwidth=16) |
979 | RSUB16 rt, ra, rb | Signed Halving sub | |
980 | URSUB16 rt, ra, rb | Unsigned Halving sub | |
981 | KSUB16 rt, ra, rb | Signed Saturating sub | |
982 | UKSUB16 rt, ra, rb | Unsigned Saturating sub | |
983 | CRAS16 rt, ra, rb | Cross Add & Sub | |
984 | RCRAS16 rt, ra, rb | Signed Halving Cross Add & Sub | |
985 | URCRAS16 rt, ra, rb| Unsigned Halving Cross Add & Sub | |
986 | KCRAS16 rt, ra, rb | Signed Saturating Cross Add & Sub | |
987 | UKCRAS16 rt, ra, rb| Unsigned Saturating Cross Add & Sub | |
988 | CRSA16 rt, ra, rb | Cross Sub & Add | |
989 | RCRSA16 rt, ra, rb | Signed Halving Cross Sub & Add | |
990 | URCRSA16 rt, ra, rb| Unsigned Halving Cross Sub & Add | |
991 | KCRSA16 rt, ra, rb | Signed Saturating Cross Sub & Add | |
992 | UKCRSA16 rt, ra, rb| Unsigned Saturating Cross Sub & Add | |
996 | Mnemonic | 16-bit Instruction | Simple-V Equivalent |
997 | ------------------ | ------------------------- | ------------------- |
998 | ADD8 rt, ra, rb | add | RV ADD (bitwidth=8)|
999 | RADD8 rt, ra, rb | Signed Halving add | |
1000 | URADD8 rt, ra, rb | Unsigned Halving add | |
1001 | KADD8 rt, ra, rb | Signed Saturating add | |
1002 | UKADD8 rt, ra, rb | Unsigned Saturating add | |
1003 | SUB8 rt, ra, rb | sub | RV SUB (bitwidth=8)|
1004 | RSUB8 rt, ra, rb | Signed Halving sub | |
1005 | URSUB8 rt, ra, rb | Unsigned Halving sub | |
1009 > What does an ADD of two different-sized vectors do in simple-V?
1011 * if the two source operands are not the same, throw an exception.
1012 * if the destination operand is also a vector, and the source is longer
1013 than the destination, throw an exception.
1015 > And what about instructions like JALR?
1016 > What does jumping to a vector do?
1018 * Throw an exception. Whether that actually results in spawning threads
1019 as part of the trap-handling remains to be seen.
1021 # Impementing V on top of Simple-V
1023 * Number of Offset CSRs extends from 2
1024 * Extra register file: vector-file
1025 * Setup of Vector length and bitwidth CSRs now can specify vector-file
1026 as well as integer or float file.
1029 # Implementing P (renamed to DSP) on top of Simple-V
1031 * Implementors indicate chosen bitwidth support in Vector-bitwidth CSR
1032 (caveat: anything not specified drops through to software-emulation / traps)
1035 # Register reordering <a name="register_reordering"></a>
1052 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
1053 | - | - | - | - | - | - | - | - |
1054 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
1058 | Reg Num | (3..0) |
1069 Virtual Register Reordering:
1071 | Reg Num | Bits (0) | Bits (1) | Bits (2) |
1072 | ------- | -------- | -------- | -------- |
1073 | r0 | (32..0) | (32..0) |
1076 | r4 | (32..0) | (32..0) | (32..0) |
1079 SIMD register file splitting still to consider. For RV64, benefits of doubling
1080 (quadrupling in the case of Half-Precision IEEE754 FP) the apparent
1081 size of the floating point register file to 64 (128 in the case of HP)
1082 seem pretty clear and worth the complexity.
1084 64 virtual 32-bit F.P. registers and given that 32-bit FP operations are
1085 done on 64-bit registers it's not so conceptually difficult. May even
1086 be achieved by *actually* splitting the regfile into 64 virtual 32-bit
1087 registers such that a 64-bit FP scalar operation is dropped into (r0.H
1088 r0.L) tuples. Implementation therefore hidden through register renaming.
1090 # Analysis of CSR decoding on latency <a name="csr_decoding_analysis"></a>
1092 It could indeed have been logically deduced (or expected), that there
1093 would be additional decode latency in this proposal, because if
1094 overloading the opcodes to have different meanings, there is guaranteed
1095 to be some state, some-where, directly related to registers.
1097 There are several cases:
1099 * All operands vector-length=1 (scalars), all operands
1100 packed-bitwidth="default": instructions are passed through direct as if
1101 Simple-V did not exist. Simple-V is, in effect, completely disabled.
1102 * At least one operand vector-length > 1, all operands
1103 packed-bitwidth="default": any parallel vector ALUs placed on "alert",
1104 virtual parallelism looping may be activated.
1105 * All operands vector-length=1 (scalars), at least one
1106 operand packed-bitwidth != default: degenerate case of SIMD,
1107 implementation-specific complexity here (packed decode before ALUs or
1109 * At least one operand vector-length > 1, at least one operand
1110 packed-bitwidth != default: parallel vector ALUs (if any)
1111 placed on "alert", virtual parallelsim looping may be activated,
1112 implementation-specific SIMD complexity kicks in (packed decode before
1115 Bear in mind that the proposal includes that the decision whether
1116 to parallelise in hardware or whether to virtual-parallelise (to
1117 dramatically simplify compilers and also not to run into the SIMD
1118 instruction proliferation nightmare) *or* a transprent combination
1119 of both, be done on a *per-operand basis*, so that implementors can
1120 specifically choose to create an application-optimised implementation
1121 that they believe (or know) will sell extremely well, without having
1122 "Extra Standards-Mandated Baggage" that would otherwise blow their area
1123 or power budget completely out the window.
1125 Additionally, two possible CSR schemes have been proposed, in order to
1126 greatly reduce CSR space:
1128 * per-register CSRs (vector-length and packed-bitwidth)
1129 * a smaller number of CSRs with the same information but with an *INDEX*
1130 specifying WHICH register in one of three regfiles (vector, fp, int)
1131 the length and bitwidth applies to.
1133 (See "CSR vector-length and CSR SIMD packed-bitwidth" section for details)
1135 In addition, LOAD/STORE has its own associated proposed CSRs that
1136 mirror the STRIDE (but not yet STRIDE-SEGMENT?) functionality of
1139 Also bear in mind that, for reasons of simplicity for implementors,
1140 I was coming round to the idea of permitting implementors to choose
1141 exactly which bitwidths they would like to support in hardware and which
1142 to allow to fall through to software-trap emulation.
1144 So the question boils down to:
1146 * whether either (or both) of those two CSR schemes have significant
1147 latency that could even potentially require an extra pipeline decode stage
1148 * whether there are implementations that can be thought of which do *not*
1149 introduce significant latency
1150 * whether it is possible to explicitly (through quite simply
1151 disabling Simple-V-Ext) or implicitly (detect the case all-vlens=1,
1152 all-simd-bitwidths=default) switch OFF any decoding, perhaps even to
1153 the extreme of skipping an entire pipeline stage (if one is needed)
1154 * whether packed bitwidth and associated regfile splitting is so complex
1155 that it should definitely, definitely be made mandatory that implementors
1156 move regfile splitting into the ALU, and what are the implications of that
1157 * whether even if that *is* made mandatory, is software-trapped
1158 "unsupported bitwidths" still desirable, on the basis that SIMD is such
1159 a complete nightmare that *even* having a software implementation is
1160 better, making Simple-V have more in common with a software API than
1163 Whilst the above may seem to be severe minuses, there are some strong
1166 * Significant reduction of V's opcode space: over 85%.
1167 * Smaller reduction of P's opcode space: around 10%.
1168 * The potential to use Compressed instructions in both Vector and SIMD
1169 due to the overloading of register meaning (implicit vectorisation,
1171 * Not only present but also future extensions automatically gain parallelism.
1172 * Already mentioned but worth emphasising: the simplification to compiler
1173 writers and assembly-level writers of having the same consistent ISA
1174 regardless of whether the internal level of parallelism (number of
1175 parallel ALUs) is only equal to one ("virtual" parallelism), or is
1176 greater than one, should not be underestimated.
1181 * SIMD considered harmful <https://www.sigarch.org/simd-instructions-considered-harmful/>
1182 * Link to first proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/GuukrSjgBH8>
1183 * Recommendation by Jacob Bachmeyer to make zero-overhead loop an
1184 "implicit program-counter" <https://groups.google.com/a/groups.riscv.org/d/msg/isa-dev/vYVi95gF2Mo/SHz6a4_lAgAJ>
1185 * Re-continuing P-Extension proposal <https://groups.google.com/a/groups.riscv.org/forum/#!msg/isa-dev/IkLkQn3HvXQ/SEMyC9IlAgAJ>
1186 * First Draft P-SIMD (DSP) proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/vYVi95gF2Mo>
1187 * B-Extension discussion <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/zi_7B15kj6s>
1188 * Broadcom VideoCore-IV <https://docs.broadcom.com/docs/12358545>
1189 Figure 2 P17 and Section 3 on P16.
1190 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-262.html>
1191 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-263.html>
1192 * Vector Workshop <http://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf>
1193 * Predication <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/XoP4BfYSLXA>
1194 * Branch Divergence <https://jbush001.github.io/2014/12/07/branch-divergence-in-parallel-kernels.html>
1195 * Life of Triangles (3D) <https://jbush001.github.io/2016/02/27/life-of-triangle.html>
1196 * Videocore-IV <https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-3d-Graphics-Pipeline>
1197 * Discussion proposing CSRs that change ISA definition
1198 <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/InzQ1wr_3Ak>