1 # Variable-width Variable-packed SIMD / Simple-V / Parallelism Extension Proposal
3 * TODO 23may2018: CSR-CAM-ify regfile tables
4 * TODO 23may2018: zero-mark predication CSR
5 * TODO 28may2018: sort out VSETVL: CSR length to be removed?
7 Key insight: Simple-V is intended as an abstraction layer to provide
8 a consistent "API" to parallelisation of existing *and future* operations.
9 *Actual* internal hardware-level parallelism is *not* required, such
10 that Simple-V may be viewed as providing a "compact" or "consolidated"
11 means of issuing multiple near-identical arithmetic instructions to an
12 instruction queue (FIFO), pending execution.
14 *Actual* parallelism, if added independently of Simple-V in the form
15 of Out-of-order restructuring (including parallel ALU lanes) or VLIW
16 implementations, or SIMD, or anything else, would then benefit *if*
17 Simple-V was added on top.
23 This proposal exists so as to be able to satisfy several disparate
24 requirements: power-conscious, area-conscious, and performance-conscious
25 designs all pull an ISA and its implementation in different conflicting
26 directions, as do the specific intended uses for any given implementation.
28 The existing P (SIMD) proposal and the V (Vector) proposals,
29 whilst each extremely powerful in their own right and clearly desirable,
32 * Clearly independent in their origins (Cray and AndesStar v3 respectively)
33 so need work to adapt to the RISC-V ethos and paradigm
34 * Are sufficiently large so as to make adoption (and exploration for
35 analysis and review purposes) prohibitively expensive
36 * Both contain partial duplication of pre-existing RISC-V instructions
37 (an undesirable characteristic)
38 * Both have independent, incompatible and disparate methods for introducing
39 parallelism at the instruction level
40 * Both require that their respective parallelism paradigm be implemented
41 along-side and integral to their respective functionality *or not at all*.
42 * Both independently have methods for introducing parallelism that
43 could, if separated, benefit
44 *other areas of RISC-V not just DSP or Floating-point respectively*.
46 There are also key differences between Vectorisation and SIMD (full
47 details outlined in the Appendix), the key points being:
49 * SIMD has an extremely seductively compelling ease of implementation argument:
50 each operation is passed to the ALU, which is where the parallelism
51 lies. There is *negligeable* (if any) impact on the rest of the core
52 (with life instead being made hell for compiler writers and applications
53 writers due to extreme ISA proliferation).
54 * By contrast, Vectorisation has quite some complexity (for considerable
55 flexibility, reduction in opcode proliferation and much more).
56 * Vectorisation typically includes much more comprehensive memory load
57 and store schemes (unit stride, constant-stride and indexed), which
58 in turn have ramifications: virtual memory misses (TLB cache misses)
59 and even multiple page-faults... all caused by a *single instruction*,
60 yet with a clear benefit that the regularisation of LOAD/STOREs can
61 be optimised for minimal impact on caches and maximised throughput.
62 * By contrast, SIMD can use "standard" memory load/stores (32-bit aligned
63 to pages), and these load/stores have absolutely nothing to do with the
64 SIMD / ALU engine, no matter how wide the operand. Simplicity but with
65 more impact on instruction and data caches.
67 Overall it makes a huge amount of sense to have a means and method
68 of introducing instruction parallelism in a flexible way that provides
69 implementors with the option to choose exactly where they wish to offer
70 performance improvements and where they wish to optimise for power
71 and/or area (and if that can be offered even on a per-operation basis that
72 would provide even more flexibility).
74 Additionally it makes sense to *split out* the parallelism inherent within
75 each of P and V, and to see if each of P and V then, in *combination* with
76 a "best-of-both" parallelism extension, could be added on *on top* of
77 this proposal, to topologically provide the exact same functionality of
78 each of P and V. Each of P and V then can focus on providing the best
79 operations possible for their respective target areas, without being
80 hugely concerned about the actual parallelism.
82 Furthermore, an additional goal of this proposal is to reduce the number
83 of opcodes utilised by each of P and V as they currently stand, leveraging
84 existing RISC-V opcodes where possible, and also potentially allowing
85 P and V to make use of Compressed Instructions as a result.
87 # Analysis and discussion of Vector vs SIMD
89 There are six combined areas between the two proposals that help with
90 parallelism (increased performance, reduced power / area) without
91 over-burdening the ISA with a huge proliferation of
94 * Fixed vs variable parallelism (fixed or variable "M" in SIMD)
95 * Implicit vs fixed instruction bit-width (integral to instruction or not)
96 * Implicit vs explicit type-conversion (compounded on bit-width)
97 * Implicit vs explicit inner loops.
98 * Single-instruction LOAD/STORE.
99 * Masks / tagging (selecting/preventing certain indexed elements from execution)
101 The pros and cons of each are discussed and analysed below.
103 ## Fixed vs variable parallelism length
105 In David Patterson and Andrew Waterman's analysis of SIMD and Vector
106 ISAs, the analysis comes out clearly in favour of (effectively) variable
107 length SIMD. As SIMD is a fixed width, typically 4, 8 or in extreme cases
108 16 or 32 simultaneous operations, the setup, teardown and corner-cases of SIMD
109 are extremely burdensome except for applications whose requirements
110 *specifically* match the *precise and exact* depth of the SIMD engine.
112 Thus, SIMD, no matter what width is chosen, is never going to be acceptable
113 for general-purpose computation, and in the context of developing a
114 general-purpose ISA, is never going to satisfy 100 percent of implementors.
116 To explain this further: for increased workloads over time, as the
117 performance requirements increase for new target markets, implementors
118 choose to extend the SIMD width (so as to again avoid mixing parallelism
119 into the instruction issue phases: the primary "simplicity" benefit of
120 SIMD in the first place), with the result that the entire opcode space
121 effectively doubles with each new SIMD width that's added to the ISA.
123 That basically leaves "variable-length vector" as the clear *general-purpose*
124 winner, at least in terms of greatly simplifying the instruction set,
125 reducing the number of instructions required for any given task, and thus
126 reducing power consumption for the same.
128 ## Implicit vs fixed instruction bit-width
130 SIMD again has a severe disadvantage here, over Vector: huge proliferation
131 of specialist instructions that target 8-bit, 16-bit, 32-bit, 64-bit, and
132 have to then have operations *for each and between each*. It gets very
135 The V-Extension on the other hand proposes to set the bit-width of
136 future instructions on a per-register basis, such that subsequent instructions
137 involving that register are *implicitly* of that particular bit-width until
138 otherwise changed or reset.
140 This has some extremely useful properties, without being particularly
141 burdensome to implementations, given that instruction decode already has
142 to direct the operation to a correctly-sized width ALU engine, anyway.
144 Not least: in places where an ISA was previously constrained (due for
145 whatever reason, including limitations of the available operand space),
146 implicit bit-width allows the meaning of certain operations to be
147 type-overloaded *without* pollution or alteration of frozen and immutable
148 instructions, in a fully backwards-compatible fashion.
150 ## Implicit and explicit type-conversion
152 The Draft 2.3 V-extension proposal has (deprecated) polymorphism to help
153 deal with over-population of instructions, such that type-casting from
154 integer (and floating point) of various sizes is automatically inferred
155 due to "type tagging" that is set with a special instruction. A register
156 will be *specifically* marked as "16-bit Floating-Point" and, if added
157 to an operand that is specifically tagged as "32-bit Integer" an implicit
158 type-conversion will take place *without* requiring that type-conversion
159 to be explicitly done with its own separate instruction.
161 However, implicit type-conversion is not only quite burdensome to
162 implement (explosion of inferred type-to-type conversion) but also is
163 never really going to be complete. It gets even worse when bit-widths
164 also have to be taken into consideration. Each new type results in
165 an increased O(N^2) conversion space that, as anyone who has examined
166 python's source code (which has built-in polymorphic type-conversion),
167 knows that the task is more complex than it first seems.
169 Overall, type-conversion is generally best to leave to explicit
170 type-conversion instructions, or in definite specific use-cases left to
171 be part of an actual instruction (DSP or FP)
173 ## Zero-overhead loops vs explicit loops
175 The initial Draft P-SIMD Proposal by Chuanhua Chang of Andes Technology
176 contains an extremely interesting feature: zero-overhead loops. This
177 proposal would basically allow an inner loop of instructions to be
178 repeated indefinitely, a fixed number of times.
180 Its specific advantage over explicit loops is that the pipeline in a DSP
181 can potentially be kept completely full *even in an in-order single-issue
182 implementation*. Normally, it requires a superscalar architecture and
183 out-of-order execution capabilities to "pre-process" instructions in
184 order to keep ALU pipelines 100% occupied.
186 By bringing that capability in, this proposal could offer a way to increase
187 pipeline activity even in simpler implementations in the one key area
188 which really matters: the inner loop.
190 However when looking at much more comprehensive schemes
191 "A portable specification of zero-overhead loop control hardware
192 applied to embedded processors" (ZOLC), optimising only the single
193 inner loop seems inadequate, tending to suggest that ZOLC may be
194 better off being proposed as an entirely separate Extension.
196 ## Single-instruction LOAD/STORE
198 In traditional Vector Architectures there are instructions which
199 result in multiple register-memory transfer operations resulting
200 from a single instruction. They're complicated to implement in hardware,
201 yet the benefits are a huge consistent regularisation of memory accesses
202 that can be highly optimised with respect to both actual memory and any
203 L1, L2 or other caches. In Hwacha EECS-2015-263 it is explicitly made
204 clear the consequences of getting this architecturally wrong:
205 L2 cache-thrashing at the very least.
207 Complications arise when Virtual Memory is involved: TLB cache misses
208 need to be dealt with, as do page faults. Some of the tradeoffs are
209 discussed in <http://people.eecs.berkeley.edu/~krste/thesis.pdf>, Section
210 4.6, and an article by Jeff Bush when faced with some of these issues
211 is particularly enlightening
212 <https://jbush001.github.io/2015/11/03/lost-in-translation.html>
214 Interestingly, none of this complexity is faced in SIMD architectures...
215 but then they do not get the opportunity to optimise for highly-streamlined
216 memory accesses either.
218 With the "bang-per-buck" ratio being so high and the indirect improvement
219 in L1 Instruction Cache usage (reduced instruction count), as well as
220 the opportunity to optimise L1 and L2 cache usage, the case for including
221 Vector LOAD/STORE is compelling.
223 ## Mask and Tagging (Predication)
225 Tagging (aka Masks aka Predication) is a pseudo-method of implementing
226 simplistic branching in a parallel fashion, by allowing execution on
227 elements of a vector to be switched on or off depending on the results
228 of prior operations in the same array position.
230 The reason for considering this is simple: by *definition* it
231 is not possible to perform individual parallel branches in a SIMD
232 (Single-Instruction, **Multiple**-Data) context. Branches (modifying
233 of the Program Counter) will result in *all* parallel data having
234 a different instruction executed on it: that's just the definition of
235 SIMD, and it is simply unavoidable.
237 So these are the ways in which conditional execution may be implemented:
239 * explicit compare and branch: BNE x, y -> offs would jump offs
240 instructions if x was not equal to y
241 * explicit store of tag condition: CMP x, y -> tagbit
242 * implicit (condition-code) such as ADD results in a carry, carry bit
243 implicitly (or sometimes explicitly) goes into a "tag" (mask) register
245 The first of these is a "normal" branch method, which is flat-out impossible
246 to parallelise without look-ahead and effectively rewriting instructions.
247 This would defeat the purpose of RISC.
249 The latter two are where parallelism becomes easy to do without complexity:
250 every operation is modified to be "conditionally executed" (in an explicit
251 way directly in the instruction format *or* implicitly).
253 RVV (Vector-Extension) proposes to have *explicit* storing of the compare
254 in a tag/mask register, and to *explicitly* have every vector operation
255 *require* that its operation be "predicated" on the bits within an
256 explicitly-named tag/mask register.
258 SIMD (P-Extension) has not yet published precise documentation on what its
259 schema is to be: there is however verbal indication at the time of writing
262 > The "compare" instructions in the DSP/SIMD ISA proposed by Andes will
263 > be executed using the same compare ALU logic for the base ISA with some
264 > minor modifications to handle smaller data types. The function will not
267 This is an *implicit* form of predication as the base RV ISA does not have
268 condition-codes or predication. By adding a CSR it becomes possible
269 to also tag certain registers as "predicated if referenced as a destination".
272 // in future operations from now on, if r0 is the destination use r5 as
273 // the PREDICATION register
274 SET_IMPLICIT_CSRPREDICATE r0, r5
275 // store the compares in r5 as the PREDICATION register
277 // r0 is used here. ah ha! that means it's predicated using r5!
280 With enough registers (and in RISC-V there are enough registers) some fairly
281 complex predication can be set up and yet still execute without significant
282 stalling, even in a simple non-superscalar architecture.
284 (For details on how Branch Instructions would be retro-fitted to indirectly
285 predicated equivalents, see Appendix)
289 In the above sections the five different ways where parallel instruction
290 execution has closely and loosely inter-related implications for the ISA and
291 for implementors, were outlined. The pluses and minuses came out as
294 * Fixed vs variable parallelism: <b>variable</b>
295 * Implicit (indirect) vs fixed (integral) instruction bit-width: <b>indirect</b>
296 * Implicit vs explicit type-conversion: <b>explicit</b>
297 * Implicit vs explicit inner loops: <b>implicit but best done separately</b>
298 * Single-instruction Vector LOAD/STORE: <b>Complex but highly beneficial</b>
299 * Tag or no-tag: <b>Complex but highly beneficial</b>
303 * variable-length vectors came out on top because of the high setup, teardown
304 and corner-cases associated with the fixed width of SIMD.
305 * Implicit bit-width helps to extend the ISA to escape from
306 former limitations and restrictions (in a backwards-compatible fashion),
307 whilst also leaving implementors free to simmplify implementations
308 by using actual explicit internal parallelism.
309 * Implicit (zero-overhead) loops provide a means to keep pipelines
310 potentially 100% occupied in a single-issue in-order implementation
311 i.e. *without* requiring a super-scalar or out-of-order architecture,
312 but doing a proper, full job (ZOLC) is an entirely different matter.
314 Constructing a SIMD/Simple-Vector proposal based around four of these six
315 requirements would therefore seem to be a logical thing to do.
317 # Note on implementation of parallelism
319 One extremely important aspect of this proposal is to respect and support
320 implementors desire to focus on power, area or performance. In that regard,
321 it is proposed that implementors be free to choose whether to implement
322 the Vector (or variable-width SIMD) parallelism as sequential operations
323 with a single ALU, fully parallel (if practical) with multiple ALUs, or
324 a hybrid combination of both.
326 In Broadcom's Videocore-IV, they chose hybrid, and called it "Virtual
327 Parallelism". They achieve a 16-way SIMD at an **instruction** level
328 by providing a combination of a 4-way parallel ALU *and* an externally
329 transparent loop that feeds 4 sequential sets of data into each of the
332 Also in the same core, it is worth noting that particularly uncommon
333 but essential operations (Reciprocal-Square-Root for example) are
334 *not* part of the 4-way parallel ALU but instead have a *single* ALU.
335 Under the proposed Vector (varible-width SIMD) implementors would
336 be free to do precisely that: i.e. free to choose *on a per operation
337 basis* whether and how much "Virtual Parallelism" to deploy.
339 It is absolutely critical to note that it is proposed that such choices MUST
340 be **entirely transparent** to the end-user and the compiler. Whilst
341 a Vector (varible-width SIMD) may not precisely match the width of the
342 parallelism within the implementation, the end-user **should not care**
343 and in this way the performance benefits are gained but the ISA remains
344 straightforward. All that happens at the end of an instruction run is: some
345 parallel units (if there are any) would remain offline, completely
346 transparently to the ISA, the program, and the compiler.
348 To make that clear: should an implementor choose a particularly wide
349 SIMD-style ALU, each parallel unit *must* have predication so that
350 the parallel SIMD ALU may emulate variable-length parallel operations.
351 Thus the "SIMD considered harmful" trap of having huge complexity and extra
352 instructions to deal with corner-cases is thus avoided, and implementors
353 get to choose precisely where to focus and target the benefits of their
354 implementation efforts, without "extra baggage".
356 In addition, implementors will be free to choose whether to provide an
357 absolute bare minimum level of compliance with the "API" (software-traps
358 when vectorisation is detected), all the way up to full supercomputing
359 level all-hardware parallelism. Options are covered in the Appendix.
361 # CSRs <a name="csrs"></a>
363 There are a number of CSRs needed, which are used at the instruction
364 decode phase to re-interpret RV opcodes (a practice that has
365 precedent in the setting of MISA to enable / disable extensions).
367 * Integer Register N is Vector of length M: r(N) -> r(N..N+M-1)
368 * Integer Register N is of implicit bitwidth M (M=default,8,16,32,64)
369 * Floating-point Register N is Vector of length M: r(N) -> r(N..N+M-1)
370 * Floating-point Register N is of implicit bitwidth M (M=default,8,16,32,64)
371 * Integer Register N is a Predication Register (note: a key-value store)
372 * Vector Length CSR (VSETVL, VGETVL)
374 Also (see Appendix, "Context Switch Example") it may turn out to be important
375 to have a separate (smaller) set of CSRs for M-Mode (and S-Mode) so that
376 Vectorised LOAD / STORE may be used to load and store multiple registers:
377 something that is missing from the Base RV ISA.
381 * for the purposes of LOAD / STORE, Integer Registers which are
382 marked as a Vector will result in a Vector LOAD / STORE.
383 * Vector Lengths are *not* the same as vsetl but are an integral part
385 * Actual vector length is *multipled* by how many blocks of length
386 "bitwidth" may fit into an XLEN-sized register file.
387 * Predication is a key-value store due to the implicit referencing,
388 as opposed to having the predicate register explicitly in the instruction.
389 * Whilst the predication CSR is a key-value store it *generates* easier-to-use
391 * TODO: assess whether the same technique could be applied to the other
392 Vector CSRs, particularly as pointed out in Section 17.8 (Draft RV 0.4,
393 V2.3-Draft ISA Reference) it becomes possible to greatly reduce state
394 needed for context-switches (empty slots need never be stored).
396 ## Predication CSR <a name="predication_csr_table"></a>
398 The Predication CSR is a key-value store indicating whether, if a given
399 destination register (integer or floating-point) is referred to in an
400 instruction, it is to be predicated. The first entry is whether predication
401 is enabled. The second entry is whether the register index refers to a
402 floating-point or an integer register. The third entry is the index
403 of that register which is to be predicated (if referred to). The fourth entry
404 is the integer register that is treated as a bitfield, indexable by the
405 vector element index.
407 | PrCSR | 7 | 6 | 5 | (4..0) | (4..0) |
408 | ----- | - | - | - | ------- | ------- |
409 | 0 | zero0 | inv0 | i/f | regidx | predidx |
410 | 1 | zero1 | inv1 | i/f | regidx | predidx |
411 | .. | zero.. | inv.. | i/f | regidx | predidx |
412 | 15 | zero15 | inv15 | i/f | regidx | predidx |
414 The Predication CSR Table is a key-value store, so implementation-wise
415 it will be faster to turn the table around (maintain topologically
422 int predidx; // redirection: actual int register to use
425 struct pred fp_pred_reg[32];
426 struct pred int_pred_reg[32];
428 for (i = 0; i < 16; i++)
429 idx = CSRpred[i].regidx
430 predidx = CSRpred[i].predidx
431 zero = CSRpred[i].zero
433 if CSRpred[i].type == 0: # integer
434 int_pred_reg[idx].zero = zero
435 int_pred_reg[idx].inv = inv
436 int_pred_reg[idx].predidx = predidx
437 int_pred_reg[idx].enabled = true
439 fp_pred_reg[idx].zero = zero
440 fp_pred_reg[idx].inv = inv
441 fp_pred_reg[idx].predidx = predidx
442 fp_pred_reg[idx].enabled = true
444 So when an operation is to be predicated, it is the internal state that
445 is used. In Section 6.4.2 of Hwacha's Manual (EECS-2015-262) the following
446 pseudo-code for operations is given, where p is the explicit (direct)
447 reference to the predication register to be used:
449 for (int i=0; i<vl; ++i)
451 (d ? vreg[rd][i] : sreg[rd]) =
452 iop(s1 ? vreg[rs1][i] : sreg[rs1],
453 s2 ? vreg[rs2][i] : sreg[rs2]); // for insts with 2 inputs
455 This instead becomes an *indirect* reference using the *internal* state
456 table generated from the Predication CSR key-value store, which is used
457 as follows (Note: d, s1 and s2 are booleans indicating whether destination,
458 source1 and source2 are vector or scalar):
461 preg = int_pred_reg[rd]
463 preg = fp_pred_reg[rd]
465 for (int i=0; i<vl; ++i)
466 if (!preg[rd].enabled)
468 predidx = preg[rd].predidx;
469 predicate = intregfile[rd];
471 predicate = ~predicate;
472 if (predicate && (1<<i))
473 (d ? vreg[rd+i] : sreg[rd]) =
474 iop(s1 ? vreg[rs1+i] : sreg[rs1],
475 s2 ? vreg[rs2+i] : sreg[rs2]); // for insts with 2 inputs
476 else if (preg[rd].zero)
477 // TODO: place zero in dest reg
481 MAXVECTORDEPTH is the same concept as MVL in RVV. However in Simple-V,
482 given that its primary (base, unextended) purpose is for 3D, Video and
483 other purposes (not requiring supercomputing capability), it makes sense
484 to limit MAXVECTORDEPTH to the regfile bitwidth (32 for RV32, 64 for RV64
487 The reason for setting this limit is so that predication registers, when
488 marked as such, may fit into a single register as opposed to fanning out
489 over several registers. This keeps the implementation a little simpler.
490 Note also (as also described in the VSETVL section) that the *minimum*
491 for MAXVECTORDEPTH must be the total number of registers (15 for RV32E
492 and 31 for RV32 or RV64).
494 Note that RVV on top of Simple-V may choose to over-ride this decision.
496 ## Vector-length CSRs
498 Vector lengths are interpreted as meaning "any instruction referring to
499 r(N) generates implicit identical instructions referring to registers
500 r(N+M-1) where M is the Vector Length". Vector Lengths may be set to
501 use up to 16 registers in the register file.
503 One separate CSR table is needed for each of the integer and floating-point
513 An array of 32 4-bit CSRs is needed (4 bits per register) to indicate
514 whether a register was, if referred to in any standard instructions,
515 implicitly to be treated as a vector.
519 * A vector length of 1 indicates that it is to be treated as a scalar.
520 Bitwidths (on the same register) are interpreted and meaningful.
521 * A vector length of 0 indicates that the parallelism is to be switched
522 off for this register (treated as a scalar). When length is 0,
523 the bitwidth CSR for the register is *ignored*.
525 Internally, implementations may choose to use the non-zero vector length
526 to set a bit-field per register, to be used in the instruction decode phase.
527 In this way any standard (current or future) operation involving
528 register operands may detect if the operation is to be vector-vector,
529 vector-scalar or scalar-scalar (standard) simply through a single
532 Note that when using the "vsetl rs1, rs2" instruction (caveat: when the
533 bitwidth is specifically not set) it becomes:
535 CSRvlength = MIN(MIN(CSRvectorlen[rs1], MAXVECTORDEPTH), rs2)
537 This is in contrast to RVV:
539 CSRvlength = MIN(MIN(rs1, MAXVECTORDEPTH), rs2)
541 ## Element (SIMD) bitwidth CSRs
543 Element bitwidths may be specified with a per-register CSR, and indicate
544 how a register (integer or floating-point) is to be subdivided.
553 vew may be one of the following (giving a table "bytestable", used below):
566 Extending this table (with extra bits) is covered in the section
567 "Implementing RVV on top of Simple-V".
569 Note that when using the "vsetl rs1, rs2" instruction, taking bitwidth
570 into account, it becomes:
572 vew = CSRbitwidth[rs1]
574 bytesperreg = (XLEN/8) # or FLEN as appropriate
576 bytesperreg = bytestable[vew] # 1 2 4 8 16
577 simdmult = (XLEN/8) / bytesperreg # or FLEN as appropriate
578 vlen = CSRvectorlen[rs1] * simdmult
579 CSRvlength = MIN(MIN(vlen, MAXVECTORDEPTH), rs2)
581 The reason for multiplying the vector length by the number of SIMD elements
582 (in each individual register) is so that each SIMD element may optionally be
585 An example of how to subdivide the register file when bitwidth != default
586 is given in the section "Bitwidth Virtual Register Reordering".
590 By being a topological remap of RVV concepts, the following RVV instructions
591 remain exactly the same: VMPOP, VMFIRST, VEXTRACT, VINSERT, VMERGE, VSELECT,
592 VSLIDE, VCLASS and VPOPC. Two instructions, VCLIP and VCLIPI, do not
593 have RV Standard equivalents, so are left out of Simple-V.
594 All other instructions from RVV are topologically re-mapped and retain
595 their complete functionality, intact.
597 ## Instruction Format
599 The instruction format for Simple-V does not actually have *any* explicit
600 compare operations, *any* arithmetic, floating point or *any*
602 Instead it *overloads* pre-existing branch operations into predicated
603 variants, and implicitly overloads arithmetic operations and LOAD/STORE
604 depending on CSR configurations for vector length, bitwidth and
605 predication. *This includes Compressed instructions* as well as any
606 future instructions and Custom Extensions.
608 * For analysis of RVV see [[v_comparative_analysis]] which begins to
609 outline topologically-equivalent mappings of instructions
610 * Also see Appendix "Retro-fitting Predication into branch-explicit ISA"
611 for format of Branch opcodes.
613 **TODO**: *analyse and decide whether the implicit nature of predication
614 as proposed is or is not a lot of hassle, and if explicit prefixes are
615 a better idea instead. Parallelism therefore effectively may end up
616 as always being 64-bit opcodes (32 for the prefix, 32 for the instruction)
617 with some opportunities for to use Compressed bringing it down to 48.
618 Also to consider is whether one or both of the last two remaining Compressed
619 instruction codes in Quadrant 1 could be used as a parallelism prefix,
620 bringing parallelised opcodes down to 32-bit (when combined with C)
621 and having the benefit of being explicit.*
625 NOTE TODO: 28may2018: VSETVL may need to be *really* different from RVV,
626 with the instruction format remaining the same.
628 VSETVL is slightly different from RVV in that the minimum vector length
629 is required to be at least the number of registers in the register file,
630 and no more than XLEN. This allows vector LOAD/STORE to be used to switch
631 the entire bank of registers using a single instruction (see Appendix,
632 "Context Switch Example"). The reason for limiting VSETVL to XLEN is
633 down to the fact that predication bits fit into a single register of length
636 The second minor change is that when VSETVL is requested to be stored
637 into x0, it is *ignored* silently.
639 Unlike RVV, implementors *must* provide pseudo-parallelism (using sequential
640 loops in hardware) if actual hardware-parallelism in the ALUs is not deployed.
641 A hybrid is also permitted (as used in Broadcom's VideoCore-IV) however this
642 must be *entirely* transparent to the ISA.
644 ### Under review / discussion: remove CSR vector length, use VSETVL <a name="vsetvl"></a>
646 So the issue is as follows:
648 * CSRs are used to set the "span" of a vector (how many of the standard
649 register file to contiguously use)
650 * VSETVL in RVV works as follows: it sets the vector length (copy of which
651 is placed in a dest register), and if the "required" length is longer
652 than the *available* length, the dest reg is set to the MIN of those
654 * **HOWEVER**... in SV, *EVERY* vector register has its own separate
655 length and thus there is no way (at the time that VSETVL is called) to
656 know what to set the vector length *to*.
657 * At first glance it seems that it would be perfectly fine to just limit
658 the vector operation to the length specified in the destination
659 register's CSR, at the time that each instruction is issued...
660 except that that cannot possibly be guaranteed to match
661 with the value *already loaded into the target register from VSETVL*.
663 Therefore a different approach is needed.
665 Possible options include:
667 * Removing the CSR "Vector Length" and always using the value from
668 VSETVL. "VSETVL destreg, counterreg, #lenimmed" will set VL *and*
669 destreg equal to MIN(counterreg, lenimmed), with register-based
670 variant "VSETVL destreg, counterreg, lenreg" doing the same.
671 * Keeping the CSR "Vector Length" and having the lenreg version have
672 a "twist": "if lengreg is vectorised, read the length from the CSR"
675 The first option (of the ones brainstormed so far) is a lot simpler.
676 It does however mean that the length set in VSETVL will apply across-the-board
677 to all src1, src2 and dest vectorised registers until it is otherwise changed
678 (by another VSETVL call). This is probably desirable behaviour.
680 ## Branch Instruction:
682 Branch operations use standard RV opcodes that are reinterpreted to be
683 "predicate variants" in the instance where either of the two src registers
684 have their corresponding CSRvectorlen[src] entry as non-zero. When this
685 reinterpretation is enabled the predicate target register rs3 is to be
686 treated as a bitfield (up to a maximum of XLEN bits corresponding to a
687 maximum of XLEN elements).
689 If either of src1 or src2 are scalars (CSRvectorlen[src] == 0) the comparison
690 goes ahead as vector-scalar or scalar-vector. Implementors should note that
691 this could require considerable multi-porting of the register file in order
692 to parallelise properly, so may have to involve the use of register cacheing
693 and transparent copying (see Multiple-Banked Register File Architectures
696 In instances where no vectorisation is detected on either src registers
697 the operation is treated as an absolutely standard scalar branch operation.
699 This is the overloaded table for Integer-base Branch operations. Opcode
700 (bits 6..0) is set in all cases to 1100011.
703 31 .. 25 |24 ... 20 | 19 15 | 14 12 | 11 .. 8 | 7 | 6 ... 0 |
704 imm[12,10:5]| rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
705 7 | 5 | 5 | 3 | 4 | 1 | 7 |
706 reserved | src2 | src1 | BPR | predicate rs3 || BRANCH |
707 reserved | src2 | src1 | 000 | predicate rs3 || BEQ |
708 reserved | src2 | src1 | 001 | predicate rs3 || BNE |
709 reserved | src2 | src1 | 010 | predicate rs3 || rsvd |
710 reserved | src2 | src1 | 011 | predicate rs3 || rsvd |
711 reserved | src2 | src1 | 100 | predicate rs3 || BLE |
712 reserved | src2 | src1 | 101 | predicate rs3 || BGE |
713 reserved | src2 | src1 | 110 | predicate rs3 || BLTU |
714 reserved | src2 | src1 | 111 | predicate rs3 || BGEU |
717 Note that just as with the standard (scalar, non-predicated) branch
718 operations, BLT, BGT, BLEU and BTGU may be synthesised by inverting
721 Below is the overloaded table for Floating-point Predication operations.
722 Interestingly no change is needed to the instruction format because
723 FP Compare already stores a 1 or a zero in its "rd" integer register
724 target, i.e. it's not actually a Branch at all: it's a compare.
725 The target needs to simply change to be a predication bitfield (done
729 Standard RVF/D/Q, Opcode (bits 6..0) is set in all cases to 1010011.
730 Likewise Single-precision, fmt bits 26..25) is still set to 00.
731 Double-precision is still set to 01, whilst Quad-precision
732 appears not to have a definition in V2.3-Draft (but should be unaffected).
734 It is however noted that an entry "FNE" (the opposite of FEQ) is missing,
735 and whilst in ordinary branch code this is fine because the standard
736 RVF compare can always be followed up with an integer BEQ or a BNE (or
737 a compressed comparison to zero or non-zero), in predication terms that
738 becomes more of an impact as an explicit (scalar) instruction is needed
739 to invert the predicate bitmask. An additional encoding funct3=011 is
740 therefore proposed to cater for this.
743 31 .. 27| 26 .. 25 |24 ... 20 | 19 15 | 14 12 | 11 .. 7 | 6 ... 0 |
744 funct5 | fmt | rs2 | rs1 | funct3 | rd | opcode |
745 5 | 2 | 5 | 5 | 3 | 4 | 7 |
746 10100 | 00/01/11 | src2 | src1 | 010 | pred rs3 | FEQ |
747 10100 | 00/01/11 | src2 | src1 | **011**| pred rs3 | FNE |
748 10100 | 00/01/11 | src2 | src1 | 001 | pred rs3 | FLT |
749 10100 | 00/01/11 | src2 | src1 | 000 | pred rs3 | FLE |
752 Note (**TBD**): floating-point exceptions will need to be extended
753 to cater for multiple exceptions (and statuses of the same). The
754 usual approach is to have an array of status codes and bit-fields,
755 and one exception, rather than throw separate exceptions for each
758 In Hwacha EECS-2015-262 Section 6.7.2 the following pseudocode is given
759 for predicated compare operations of function "cmp":
761 for (int i=0; i<vl; ++i)
763 preg[pd][i] = cmp(s1 ? vreg[rs1][i] : sreg[rs1],
764 s2 ? vreg[rs2][i] : sreg[rs2]);
766 With associated predication, vector-length adjustments and so on,
767 and temporarily ignoring bitwidth (which makes the comparisons more
768 complex), this becomes:
770 if I/F == INT: # integer type cmp
771 pred_enabled = int_pred_enabled # TODO: exception if not set!
772 preg = int_pred_reg[rd]
775 pred_enabled = fp_pred_enabled # TODO: exception if not set!
776 preg = fp_pred_reg[rd]
779 s1 = CSRvectorlen[src1] > 1;
780 s2 = CSRvectorlen[src2] > 1;
781 for (int i=0; i<vl; ++i)
782 preg[rs3][i] = cmp(s1 ? reg[src1+i] : reg[src1],
783 s2 ? reg[src2+i] : reg[src2]);
787 * Predicated SIMD comparisons would break src1 and src2 further down
788 into bitwidth-sized chunks (see Appendix "Bitwidth Virtual Register
789 Reordering") setting Vector-Length times (number of SIMD elements) bits
790 in Predicate Register rs3 as opposed to just Vector-Length bits.
791 * Predicated Branches do not actually have an adjustment to the Program
792 Counter, so all of bits 25 through 30 in every case are not needed.
793 * There are plenty of reserved opcodes for which bits 25 through 30 could
794 be put to good use if there is a suitable use-case.
795 * FEQ and FNE (and BEQ and BNE) are included in order to save one
796 instruction having to invert the resultant predicate bitfield.
797 FLT and FLE may be inverted to FGT and FGE if needed by swapping
798 src1 and src2 (likewise the integer counterparts).
800 ## Compressed Branch Instruction:
803 15..13 | 12...10 | 9..7 | 6..5 | 4..2 | 1..0 | name |
804 funct3 | imm | rs10 | imm | | op | |
805 3 | 3 | 3 | 2 | 3 | 2 | |
806 C.BPR | pred rs3 | src1 | I/F B | src2 | C1 | |
807 110 | pred rs3 | src1 | I/F 0 | src2 | C1 | P.EQ |
808 111 | pred rs3 | src1 | I/F 0 | src2 | C1 | P.NE |
809 110 | pred rs3 | src1 | I/F 1 | src2 | C1 | P.LT |
810 111 | pred rs3 | src1 | I/F 1 | src2 | C1 | P.LE |
815 * Bits 5 13 14 and 15 make up the comparator type
816 * Bit 6 indicates whether to use integer or floating-point comparisons
817 * In both floating-point and integer cases there are four predication
818 comparators: EQ/NEQ/LT/LE (with GT and GE being synthesised by inverting
821 ## LOAD / STORE Instructions <a name="load_store"></a>
823 For full analysis of topological adaptation of RVV LOAD/STORE
824 see [[v_comparative_analysis]]. All three types (LD, LD.S and LD.X)
825 may be implicitly overloaded into the one base RV LOAD instruction,
826 and likewise for STORE.
831 31 | 30 | 29 25 | 24 20 | 19 15 | 14 12 | 11 7 | 6 0 |
832 imm[11:0] |||| rs1 | funct3 | rd | opcode |
833 1 | 1 | 5 | 5 | 5 | 3 | 5 | 7 |
834 ? | s | rs2 | imm[4:0] | base | width | dest | LOAD |
837 The exact same corresponding adaptation is also carried out on the single,
838 double and quad precision floating-point LOAD-FP and STORE-FP operations,
839 which fit the exact same instruction format. Thus all three types
840 (unit, stride and indexed) may be fitted into FLW, FLD and FLQ,
841 as well as FSW, FSD and FSQ.
845 * LOAD remains functionally (topologically) identical to RVV LOAD
846 (for both integer and floating-point variants).
847 * Predication CSR-marking register is not explicitly shown in instruction, it's
848 implicit based on the CSR predicate state for the rd (destination) register
849 * rs2, the source, may *also be marked as a vector*, which implicitly
850 is taken to indicate "Indexed Load" (LD.X)
851 * Bit 30 indicates "element stride" or "constant-stride" (LD or LD.S)
852 * Bit 31 is reserved (ideas under consideration: auto-increment)
853 * **TODO**: include CSR SIMD bitwidth in the pseudo-code below.
854 * **TODO**: clarify where width maps to elsize
856 Pseudo-code (excludes CSR SIMD bitwidth for simplicity):
858 if (unit-strided) stride = elsize;
859 else stride = areg[as2]; // constant-strided
861 pred_enabled = int_pred_enabled
862 preg = int_pred_reg[rd]
864 for (int i=0; i<vl; ++i)
865 if (preg_enabled[rd] && [!]preg[i])
866 for (int j=0; j<seglen+1; j++)
868 if CSRvectorised[rs2])
871 offs = i*(seglen+1)*stride;
872 vreg[rd+j][i] = mem[sreg[base] + offs + j*stride];
875 Taking CSR (SIMD) bitwidth into account involves using the vector
876 length and register encoding according to the "Bitwidth Virtual Register
877 Reordering" scheme shown in the Appendix (see function "regoffs").
879 A similar instruction exists for STORE, with identical topological
880 translation of all features. **TODO**
882 ## Compressed LOAD / STORE Instructions
884 Compressed LOAD and STORE are of the same format, where bits 2-4 are
885 a src register instead of dest:
888 15 13 | 12 10 | 9 7 | 6 5 | 4 2 | 1 0 |
889 funct3 | imm | rs10 | imm | rd0 | op |
890 3 | 3 | 3 | 2 | 3 | 2 |
891 C.LW | offset[5:3] | base | offset[2|6] | dest | C0 |
894 Unfortunately it is not possible to fit the full functionality
895 of vectorised LOAD / STORE into C.LD / C.ST: the "X" variants (Indexed)
896 require another operand (rs2) in addition to the operand width
897 (which is also missing), offset, base, and src/dest.
899 However a close approximation may be achieved by taking the top bit
900 of the offset in each of the five types of LD (and ST), reducing the
901 offset to 4 bits and utilising the 5th bit to indicate whether "stride"
902 is to be enabled. In this way it is at least possible to introduce
905 (**TODO**: *assess whether the loss of one bit from offset is worth having
906 "stride" capability.*)
908 We also assume (including for the "stride" variant) that the "width"
909 parameter, which is missing, is derived and implicit, just as it is
910 with the standard Compressed LOAD/STORE instructions. For C.LW, C.LD
911 and C.LQ, the width is implicitly 4, 8 and 16 respectively, whilst for
912 C.FLW and C.FLD the width is implicitly 4 and 8 respectively.
914 Interestingly we note that the Vectorised Simple-V variant of
915 LOAD/STORE (Compressed and otherwise), due to it effectively using the
916 standard register file(s), is the direct functional equivalent of
917 standard load-multiple and store-multiple instructions found in other
920 In Section 12.3 riscv-isa manual V2.3-draft it is noted the comments on
921 page 76, "For virtual memory systems some data accesses could be resident
922 in physical memory and some not". The interesting question then arises:
923 how does RVV deal with the exact same scenario?
924 Expired U.S. Patent 5895501 (Filing Date Sep 3 1996) describes a method
925 of detecting early page / segmentation faults and adjusting the TLB
926 in advance, accordingly: other strategies are explored in the Appendix
927 Section "Virtual Memory Page Faults".
931 > What does an ADD of two different-sized vectors do in simple-V?
933 * if the two source operands are not the same, throw an exception.
934 * if the destination operand is also a vector, and the source is longer
935 than the destination, throw an exception.
937 > And what about instructions like JALR?
938 > What does jumping to a vector do?
940 * Throw an exception. Whether that actually results in spawning threads
941 as part of the trap-handling remains to be seen.
943 # Impementing V on top of Simple-V
945 With Simple-V converting the original RVV draft concept-for-concept
946 from explicit opcodes to implicit overloading of existing RV Standard
947 Extensions, certain features were (deliberately) excluded that need
948 to be added back in for RVV to reach its full potential. This is
949 made slightly complicated by the fact that RVV itself has two
950 levels: Base and reserved future functionality.
952 * Representation Encoding is entirely left out of Simple-V in favour of
953 implicitly taking the exact (explicit) meaning from RV Standard Extensions.
954 * VCLIP and VCLIPI do not have corresponding RV Standard Extension
955 opcodes (and are the only such operations).
956 * Extended Element bitwidths (1 through to 24576 bits) were left out
957 of Simple-V as, again, there is no corresponding RV Standard Extension
958 that covers anything even below 32-bit operands.
959 * Polymorphism was entirely left out of Simple-V due to the inherent
960 complexity of automatic type-conversion.
961 * Vector Register files were specifically left out of Simple-V in favour
962 of fitting on top of the integer and floating-point files. An
963 "RVV re-retro-fit" needs to be able to mark (implicitly marked)
964 registers as being actually in a separate *vector* register file.
965 * Fortunately in RVV (Draft 0.4, V2.3-Draft), the "base" vector
966 register file size is 5 bits (32 registers), whilst the "Extended"
967 variant of RVV specifies 8 bits (256 registers) and has yet to
969 * One big difference: Sections 17.12 and 17.17, there are only two possible
970 predication registers in RVV "Base". Through the "indirect" method,
971 Simple-V provides a key-value CSR table that allows (arbitrarily)
972 up to 16 (TBD) of either the floating-point or integer registers to
973 be marked as "predicated" (key), and if so, which integer register to
974 use as the predication mask (value).
978 # Implementing P (renamed to DSP) on top of Simple-V
980 * Implementors indicate chosen bitwidth support in Vector-bitwidth CSR
981 (caveat: anything not specified drops through to software-emulation / traps)
986 ## V-Extension to Simple-V Comparative Analysis
988 This section has been moved to its own page [[v_comparative_analysis]]
992 This section has been moved to its own page [[p_comparative_analysis]]
994 ## Comparison of "Traditional" SIMD, Alt-RVP, Simple-V and RVV Proposals <a name="parallelism_comparisons"></a>
996 This section compares the various parallelism proposals as they stand,
997 including traditional SIMD, in terms of features, ease of implementation,
998 complexity, flexibility, and die area.
1000 ### [[harmonised_rvv_rvp]]
1002 This is an interesting proposal under development to retro-fit the AndesStar
1007 Primary benefit of Alt-RVP is the simplicity with which parallelism
1008 may be introduced (effective multiplication of regfiles and associated ALUs).
1010 * plus: the simplicity of the lanes (combined with the regularity of
1011 allocating identical opcodes multiple independent registers) meaning
1012 that SRAM or 2R1W can be used for entire regfile (potentially).
1013 * minus: a more complex instruction set where the parallelism is much
1014 more explicitly directly specified in the instruction and
1015 * minus: if you *don't* have an explicit instruction (opcode) and you
1016 need one, the only place it can be added is... in the vector unit and
1017 * minus: opcode functions (and associated ALUs) duplicated in Alt-RVP are
1018 not useable or accessible in other Extensions.
1019 * plus-and-minus: Lanes may be utilised for high-speed context-switching
1020 but with the down-side that they're an all-or-nothing part of the Extension.
1021 No Alt-RVP: no fast register-bank switching.
1022 * plus: Lane-switching would mean that complex operations not suited to
1023 parallelisation can be carried out, followed by further parallel Lane-based
1024 work, without moving register contents down to memory (and back)
1025 * minus: Access to registers across multiple lanes is challenging. "Solution"
1026 is to drop data into memory and immediately back in again (like MMX).
1030 Primary benefit of Simple-V is the OO abstraction of parallel principles
1031 from actual (internal) parallel hardware. It's an API in effect that's
1032 designed to be slotted in to an existing implementation (just after
1033 instruction decode) with minimum disruption and effort.
1035 * minus: the complexity (if full parallelism is to be exploited)
1036 of having to use register renames, OoO, VLIW, register file cacheing,
1037 all of which has been done before but is a pain
1038 * plus: transparent re-use of existing opcodes as-is just indirectly
1039 saying "this register's now a vector" which
1040 * plus: means that future instructions also get to be inherently
1041 parallelised because there's no "separate vector opcodes"
1042 * plus: Compressed instructions may also be (indirectly) parallelised
1043 * minus: the indirect nature of Simple-V means that setup (setting
1044 a CSR register to indicate vector length, a separate one to indicate
1045 that it is a predicate register and so on) means a little more setup
1046 time than Alt-RVP or RVV's "direct and within the (longer) instruction"
1048 * plus: shared register file meaning that, like Alt-RVP, complex
1049 operations not suited to parallelisation may be carried out interleaved
1050 between parallelised instructions *without* requiring data to be dropped
1051 down to memory and back (into a separate vectorised register engine).
1052 * plus-and-maybe-minus: re-use of integer and floating-point 32-wide register
1053 files means that huge parallel workloads would use up considerable
1054 chunks of the register file. However in the case of RV64 and 32-bit
1055 operations, that effectively means 64 slots are available for parallel
1057 * plus: inherent parallelism (actual parallel ALUs) doesn't actually need to
1058 be added, yet the instruction opcodes remain unchanged (and still appear
1059 to be parallel). consistent "API" regardless of actual internal parallelism:
1060 even an in-order single-issue implementation with a single ALU would still
1061 appear to have parallel vectoristion.
1062 * hard-to-judge: if actual inherent underlying ALU parallelism is added it's
1063 hard to say if there would be pluses or minuses (on die area). At worse it
1064 would be "no worse" than existing register renaming, OoO, VLIW and register
1065 file cacheing schemes.
1067 ### RVV (as it stands, Draft 0.4 Section 17, RISC-V ISA V2.3-Draft)
1069 RVV is extremely well-designed and has some amazing features, including
1070 2D reorganisation of memory through LOAD/STORE "strides".
1072 * plus: regular predictable workload means that implementations may
1073 streamline effects on L1/L2 Cache.
1074 * plus: regular and clear parallel workload also means that lanes
1075 (similar to Alt-RVP) may be used as an implementation detail,
1076 using either SRAM or 2R1W registers.
1077 * plus: separate engine with no impact on the rest of an implementation
1078 * minus: separate *complex* engine with no RTL (ALUs, Pipeline stages) reuse
1080 * minus: no ISA abstraction or re-use either: additions to other Extensions
1081 do not gain parallelism, resulting in prolific duplication of functionality
1082 inside RVV *and out*.
1083 * minus: when operations require a different approach (scalar operations
1084 using the standard integer or FP regfile) an entire vector must be
1085 transferred out to memory, into standard regfiles, then back to memory,
1086 then back to the vector unit, this to occur potentially multiple times.
1087 * minus: will never fit into Compressed instruction space (as-is. May
1088 be able to do so if "indirect" features of Simple-V are partially adopted).
1089 * plus-and-slight-minus: extended variants may address up to 256
1090 vectorised registers (requires 48/64-bit opcodes to do it).
1091 * minus-and-partial-plus: separate engine plus complexity increases
1092 implementation time and die area, meaning that adoption is likely only
1093 to be in high-performance specialist supercomputing (where it will
1094 be absolutely superb).
1096 ### Traditional SIMD
1098 The only really good things about SIMD are how easy it is to implement and
1099 get good performance. Unfortunately that makes it quite seductive...
1101 * plus: really straightforward, ALU basically does several packed operations
1102 at once. Parallelism is inherent at the ALU, making the addition of
1103 SIMD-style parallelism an easy decision that has zero significant impact
1104 on the rest of any given architectural design and layout.
1105 * plus (continuation): SIMD in simple in-order single-issue designs can
1106 therefore result in superb throughput, easily achieved even with a very
1107 simple execution model.
1108 * minus: ridiculously complex setup and corner-cases that disproportionately
1109 increase instruction count on what would otherwise be a "simple loop",
1110 should the number of elements in an array not happen to exactly match
1111 the SIMD group width.
1112 * minus: getting data usefully out of registers (if separate regfiles
1113 are used) means outputting to memory and back.
1114 * minus: quite a lot of supplementary instructions for bit-level manipulation
1115 are needed in order to efficiently extract (or prepare) SIMD operands.
1116 * minus: MASSIVE proliferation of ISA both in terms of opcodes in one
1117 dimension and parallelism (width): an at least O(N^2) and quite probably
1118 O(N^3) ISA proliferation that often results in several thousand
1119 separate instructions. all requiring separate and distinct corner-case
1121 * minus: EVEN BIGGER proliferation of SIMD ISA if the functionality of
1122 8, 16, 32 or 64-bit reordering is built-in to the SIMD instruction.
1123 For example: add (high|low) 16-bits of r1 to (low|high) of r2 requires
1124 four separate and distinct instructions: one for (r1:low r2:high),
1125 one for (r1:high r2:low), one for (r1:high r2:high) and one for
1126 (r1:low r2:low) *per function*.
1127 * minus: EVEN BIGGER proliferation of SIMD ISA if there is a mismatch
1128 between operand and result bit-widths. In combination with high/low
1129 proliferation the situation is made even worse.
1130 * minor-saving-grace: some implementations *may* have predication masks
1131 that allow control over individual elements within the SIMD block.
1133 ## Comparison *to* Traditional SIMD: Alt-RVP, Simple-V and RVV Proposals <a name="simd_comparison"></a>
1135 This section compares the various parallelism proposals as they stand,
1136 *against* traditional SIMD as opposed to *alongside* SIMD. In other words,
1137 the question is asked "How can each of the proposals effectively implement
1138 (or replace) SIMD, and how effective would they be"?
1142 * Alt-RVP would not actually replace SIMD but would augment it: just as with
1143 a SIMD architecture where the ALU becomes responsible for the parallelism,
1144 Alt-RVP ALUs would likewise be so responsible... with *additional*
1145 (lane-based) parallelism on top.
1146 * Thus at least some of the downsides of SIMD ISA O(N^5) proliferation by
1147 at least one dimension are avoided (architectural upgrades introducing
1148 128-bit then 256-bit then 512-bit variants of the exact same 64-bit
1150 * Thus, unfortunately, Alt-RVP would suffer the same inherent proliferation
1151 of instructions as SIMD, albeit not quite as badly (due to Lanes).
1152 * In the same discussion for Alt-RVP, an additional proposal was made to
1153 be able to subdivide the bits of each register lane (columns) down into
1154 arbitrary bit-lengths (RGB 565 for example).
1155 * A recommendation was given instead to make the subdivisions down to 32-bit,
1156 16-bit or even 8-bit, effectively dividing the registerfile into
1157 Lane0(H), Lane0(L), Lane1(H) ... LaneN(L) or further. If inter-lane
1158 "swapping" instructions were then introduced, some of the disadvantages
1159 of SIMD could be mitigated.
1163 * RVV is designed to replace SIMD with a better paradigm: arbitrary-length
1165 * However whilst SIMD is usually designed for single-issue in-order simple
1166 DSPs with a focus on Multimedia (Audio, Video and Image processing),
1167 RVV's primary focus appears to be on Supercomputing: optimisation of
1168 mathematical operations that fit into the OpenCL space.
1169 * Adding functions (operations) that would normally fit (in parallel)
1170 into a SIMD instruction requires an equivalent to be added to the
1171 RVV Extension, if one does not exist. Given the specialist nature of
1172 some SIMD instructions (8-bit or 16-bit saturated or halving add),
1173 this possibility seems extremely unlikely to occur, even if the
1174 implementation overhead of RVV were acceptable (compared to
1175 normal SIMD/DSP-style single-issue in-order simplicity).
1179 * Simple-V borrows hugely from RVV as it is intended to be easy to
1180 topologically transplant every single instruction from RVV (as
1181 designed) into Simple-V equivalents, with *zero loss of functionality
1183 * With the "parallelism" abstracted out, a hypothetical SIMD-less "DSP"
1184 Extension which contained the basic primitives (non-parallelised
1185 8, 16 or 32-bit SIMD operations) inherently *become* parallel,
1187 * Additionally, standard operations (ADD, MUL) that would normally have
1188 to have special SIMD-parallel opcodes added need no longer have *any*
1189 of the length-dependent variants (2of 32-bit ADDs in a 64-bit register,
1190 4of 32-bit ADDs in a 128-bit register) because Simple-V takes the
1191 *standard* RV opcodes (present and future) and automatically parallelises
1193 * By inheriting the RVV feature of arbitrary vector-length, then just as
1194 with RVV the corner-cases and ISA proliferation of SIMD is avoided.
1195 * Whilst not entirely finalised, registers are expected to be
1196 capable of being subdivided down to an implementor-chosen bitwidth
1197 in the underlying hardware (r1 becomes r1[31..24] r1[23..16] r1[15..8]
1198 and r1[7..0], or just r1[31..16] r1[15..0]) where implementors can
1199 choose to have separate independent 8-bit ALUs or dual-SIMD 16-bit
1200 ALUs that perform twin 8-bit operations as they see fit, or anything
1201 else including no subdivisions at all.
1202 * Even though implementors have that choice even to have full 64-bit
1203 (with RV64) SIMD, they *must* provide predication that transparently
1204 switches off appropriate units on the last loop, thus neatly fitting
1205 underlying SIMD ALU implementations *into* the arbitrary vector-length
1206 RVV paradigm, keeping the uniform consistent API that is a key strategic
1207 feature of Simple-V.
1208 * With Simple-V fitting into the standard register files, certain classes
1209 of SIMD operations such as High/Low arithmetic (r1[31..16] + r2[15..0])
1210 can be done by applying *Parallelised* Bit-manipulation operations
1211 followed by parallelised *straight* versions of element-to-element
1212 arithmetic operations, even if the bit-manipulation operations require
1213 changing the bitwidth of the "vectors" to do so. Predication can
1214 be utilised to skip high words (or low words) in source or destination.
1215 * In essence, the key downside of SIMD - massive duplication of
1216 identical functions over time as an architecture evolves from 32-bit
1217 wide SIMD all the way up to 512-bit, is avoided with Simple-V, through
1218 vector-style parallelism being dropped on top of 8-bit or 16-bit
1219 operations, all the while keeping a consistent ISA-level "API" irrespective
1220 of implementor design choices (or indeed actual implementations).
1222 ### Example Instruction translation: <a name="example_translation"></a>
1224 Instructions "ADD r2 r4 r4" would result in three instructions being
1225 generated and placed into the FIFO:
1231 ## Example of vector / vector, vector / scalar, scalar / scalar => vector add
1233 register CSRvectorlen[XLEN][4]; # not quite decided yet about this one...
1234 register CSRpredicate[XLEN][4]; # 2^4 is max vector length
1235 register CSRreg_is_vectorised[XLEN]; # just for fun support scalars as well
1236 register x[32][XLEN];
1238 function op_add(rd, rs1, rs2, predr)
1240 /* note that this is ADD, not PADD */
1241 int i, id, irs1, irs2;
1242 # checks CSRvectorlen[rd] == CSRvectorlen[rs] etc. ignored
1243 # also destination makes no sense as a scalar but what the hell...
1244 for (i = 0, id=0, irs1=0, irs2=0; i<CSRvectorlen[rd]; i++)
1245 if (CSRpredicate[predr][i]) # i *think* this is right...
1246 x[rd+id] <= x[rs1+irs1] + x[rs2+irs2];
1247 # now increment the idxs
1248 if (CSRreg_is_vectorised[rd]) # bitfield check rd, scalar/vector?
1250 if (CSRreg_is_vectorised[rs1]) # bitfield check rs1, scalar/vector?
1252 if (CSRreg_is_vectorised[rs2]) # bitfield check rs2, scalar/vector?
1256 ## Retro-fitting Predication into branch-explicit ISA <a name="predication_retrofit"></a>
1258 One of the goals of this parallelism proposal is to avoid instruction
1259 duplication. However, with the base ISA having been designed explictly
1260 to *avoid* condition-codes entirely, shoe-horning predication into it
1261 bcomes quite challenging.
1263 However what if all branch instructions, if referencing a vectorised
1264 register, were instead given *completely new analogous meanings* that
1265 resulted in a parallel bit-wise predication register being set? This
1266 would have to be done for both C.BEQZ and C.BNEZ, as well as BEQ, BNE,
1269 We might imagine that FEQ, FLT and FLT would also need to be converted,
1270 however these are effectively *already* in the precise form needed and
1271 do not need to be converted *at all*! The difference is that FEQ, FLT
1272 and FLE *specifically* write a 1 to an integer register if the condition
1273 holds, and 0 if not. All that needs to be done here is to say, "if
1274 the integer register is tagged with a bit that says it is a predication
1275 register, the **bit** in the integer register is set based on the
1276 current vector index" instead.
1278 There is, in the standard Conditional Branch instruction, more than
1279 adequate space to interpret it in a similar fashion:
1282 31 |30 ..... 25 |24..20|19..15| 14...12| 11.....8 | 7 | 6....0 |
1283 imm[12] | imm[10:5] |rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
1284 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
1285 offset[12,10:5] || src2 | src1 | BEQ | offset[11,4:1] || BRANCH |
1291 31 | 30 .. 25 |24 ... 20 | 19 15 | 14 12 | 11 .. 8 | 7 | 6 ... 0 |
1292 imm[12] | imm[10:5]| rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
1293 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
1294 reserved || src2 | src1 | BEQ | predicate rs3 || BRANCH |
1297 Similarly the C.BEQZ and C.BNEZ instruction format may be retro-fitted,
1298 with the interesting side-effect that there is space within what is presently
1299 the "immediate offset" field to reinterpret that to add in not only a bit
1300 field to distinguish between floating-point compare and integer compare,
1301 not only to add in a second source register, but also use some of the bits as
1302 a predication target as well.
1305 15..13 | 12 ....... 10 | 9...7 | 6 ......... 2 | 1 .. 0 |
1306 funct3 | imm | rs10 | imm | op |
1308 C.BEQZ | offset[8,4:3] | src | offset[7:6,2:1,5] | C1 |
1311 Now uses the CS format:
1314 15..13 | 12 . 10 | 9 .. 7 | 6 .. 5 | 4..2 | 1 .. 0 |
1315 funct3 | imm | rs10 | imm | | op |
1316 3 | 3 | 3 | 2 | 3 | 2 |
1317 C.BEQZ | pred rs3 | src1 | I/F B | src2 | C1 |
1320 Bit 6 would be decoded as "operation refers to Integer or Float" including
1321 interpreting src1 and src2 accordingly as outlined in Table 12.2 of the
1322 "C" Standard, version 2.0,
1323 whilst Bit 5 would allow the operation to be extended, in combination with
1324 funct3 = 110 or 111: a combination of four distinct (predicated) comparison
1325 operators. In both floating-point and integer cases those could be
1326 EQ/NEQ/LT/LE (with GT and GE being synthesised by inverting src1 and src2).
1328 ## Register reordering <a name="register_reordering"></a>
1347 May not be an actual CSR: may be generated from Vector Length CSR:
1348 single-bit is less burdensome on instruction decode phase.
1350 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
1351 | - | - | - | - | - | - | - | - |
1352 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
1354 ### Vector Length CSR
1356 | Reg Num | (3..0) |
1367 ### Virtual Register Reordering
1369 This example assumes the above Vector Length CSR table
1371 | Reg Num | Bits (0) | Bits (1) | Bits (2) |
1372 | ------- | -------- | -------- | -------- |
1373 | r0 | (32..0) | (32..0) |
1376 | r4 | (32..0) | (32..0) | (32..0) |
1379 ### Bitwidth Virtual Register Reordering
1381 This example goes a little further and illustrates the effect that a
1382 bitwidth CSR has been set on a register. Preconditions:
1385 * CSRintbitwidth[2] = 010 # integer r2 is 16-bit
1386 * CSRintvlength[2] = 3 # integer r2 is a vector of length 3
1387 * vsetl rs1, 5 # set the vector length to 5
1389 This is interpreted as follows:
1391 * Given that the context is RV32, ELEN=32.
1392 * With ELEN=32 and bitwidth=16, the number of SIMD elements is 2
1393 * Therefore the actual vector length is up to *six* elements
1394 * However vsetl sets a length 5 therefore the last "element" is skipped
1396 So when using an operation that uses r2 as a source (or destination)
1397 the operation is carried out as follows:
1399 * 16-bit operation on r2(15..0) - vector element index 0
1400 * 16-bit operation on r2(31..16) - vector element index 1
1401 * 16-bit operation on r3(15..0) - vector element index 2
1402 * 16-bit operation on r3(31..16) - vector element index 3
1403 * 16-bit operation on r4(15..0) - vector element index 4
1404 * 16-bit operation on r4(31..16) **NOT** carried out due to length being 5
1406 Predication has been left out of the above example for simplicity, however
1407 predication is ANDed with the latter stages (vsetl not equal to maximum
1410 Note also that it is entirely an implementor's choice as to whether to have
1411 actual separate ALUs down to the minimum bitwidth, or whether to have something
1412 more akin to traditional SIMD (at any level of subdivision: 8-bit SIMD
1413 operations carried out 32-bits at a time is perfectly acceptable, as is
1414 8-bit SIMD operations carried out 16-bits at a time requiring two ALUs).
1415 Regardless of the internal parallelism choice, *predication must
1416 still be respected*, making Simple-V in effect the "consistent public API".
1418 vew may be one of the following (giving a table "bytestable", used below):
1420 | vew | bitwidth | bytestable |
1421 | --- | -------- | ---------- |
1422 | 000 | default | XLEN/8 |
1428 | 110 | rsvd | rsvd |
1429 | 111 | rsvd | rsvd |
1431 Pseudocode for vector length taking CSR SIMD-bitwidth into account:
1433 vew = CSRbitwidth[rs1]
1435 bytesperreg = (XLEN/8) # or FLEN as appropriate
1437 bytesperreg = bytestable[vew] # 1 2 4 8 16
1438 simdmult = (XLEN/8) / bytesperreg # or FLEN as appropriate
1439 vlen = CSRvectorlen[rs1] * simdmult
1441 To index an element in a register rnum where the vector element index is i:
1443 function regoffs(rnum, i):
1444 regidx = floor(i / simdmult) # integer-div rounded down
1445 byteidx = i % simdmult # integer-remainder
1446 return rnum + regidx, # actual real register
1448 byteidx * 8 + (vew-1), # high
1452 SIMD register file splitting still to consider. For RV64, benefits of doubling
1453 (quadrupling in the case of Half-Precision IEEE754 FP) the apparent
1454 size of the floating point register file to 64 (128 in the case of HP)
1455 seem pretty clear and worth the complexity.
1457 64 virtual 32-bit F.P. registers and given that 32-bit FP operations are
1458 done on 64-bit registers it's not so conceptually difficult. May even
1459 be achieved by *actually* splitting the regfile into 64 virtual 32-bit
1460 registers such that a 64-bit FP scalar operation is dropped into (r0.H
1461 r0.L) tuples. Implementation therefore hidden through register renaming.
1463 Implementations intending to introduce VLIW, OoO and parallelism
1464 (even without Simple-V) would then find that the instructions are
1465 generated quicker (or in a more compact fashion that is less heavy
1466 on caches). Interestingly we observe then that Simple-V is about
1467 "consolidation of instruction generation", where actual parallelism
1468 of underlying hardware is an implementor-choice that could just as
1469 equally be applied *without* Simple-V even being implemented.
1471 ## Analysis of CSR decoding on latency <a name="csr_decoding_analysis"></a>
1473 It could indeed have been logically deduced (or expected), that there
1474 would be additional decode latency in this proposal, because if
1475 overloading the opcodes to have different meanings, there is guaranteed
1476 to be some state, some-where, directly related to registers.
1478 There are several cases:
1480 * All operands vector-length=1 (scalars), all operands
1481 packed-bitwidth="default": instructions are passed through direct as if
1482 Simple-V did not exist. Simple-V is, in effect, completely disabled.
1483 * At least one operand vector-length > 1, all operands
1484 packed-bitwidth="default": any parallel vector ALUs placed on "alert",
1485 virtual parallelism looping may be activated.
1486 * All operands vector-length=1 (scalars), at least one
1487 operand packed-bitwidth != default: degenerate case of SIMD,
1488 implementation-specific complexity here (packed decode before ALUs or
1490 * At least one operand vector-length > 1, at least one operand
1491 packed-bitwidth != default: parallel vector ALUs (if any)
1492 placed on "alert", virtual parallelsim looping may be activated,
1493 implementation-specific SIMD complexity kicks in (packed decode before
1496 Bear in mind that the proposal includes that the decision whether
1497 to parallelise in hardware or whether to virtual-parallelise (to
1498 dramatically simplify compilers and also not to run into the SIMD
1499 instruction proliferation nightmare) *or* a transprent combination
1500 of both, be done on a *per-operand basis*, so that implementors can
1501 specifically choose to create an application-optimised implementation
1502 that they believe (or know) will sell extremely well, without having
1503 "Extra Standards-Mandated Baggage" that would otherwise blow their area
1504 or power budget completely out the window.
1506 Additionally, two possible CSR schemes have been proposed, in order to
1507 greatly reduce CSR space:
1509 * per-register CSRs (vector-length and packed-bitwidth)
1510 * a smaller number of CSRs with the same information but with an *INDEX*
1511 specifying WHICH register in one of three regfiles (vector, fp, int)
1512 the length and bitwidth applies to.
1514 (See "CSR vector-length and CSR SIMD packed-bitwidth" section for details)
1516 In addition, LOAD/STORE has its own associated proposed CSRs that
1517 mirror the STRIDE (but not yet STRIDE-SEGMENT?) functionality of
1520 Also bear in mind that, for reasons of simplicity for implementors,
1521 I was coming round to the idea of permitting implementors to choose
1522 exactly which bitwidths they would like to support in hardware and which
1523 to allow to fall through to software-trap emulation.
1525 So the question boils down to:
1527 * whether either (or both) of those two CSR schemes have significant
1528 latency that could even potentially require an extra pipeline decode stage
1529 * whether there are implementations that can be thought of which do *not*
1530 introduce significant latency
1531 * whether it is possible to explicitly (through quite simply
1532 disabling Simple-V-Ext) or implicitly (detect the case all-vlens=1,
1533 all-simd-bitwidths=default) switch OFF any decoding, perhaps even to
1534 the extreme of skipping an entire pipeline stage (if one is needed)
1535 * whether packed bitwidth and associated regfile splitting is so complex
1536 that it should definitely, definitely be made mandatory that implementors
1537 move regfile splitting into the ALU, and what are the implications of that
1538 * whether even if that *is* made mandatory, is software-trapped
1539 "unsupported bitwidths" still desirable, on the basis that SIMD is such
1540 a complete nightmare that *even* having a software implementation is
1541 better, making Simple-V have more in common with a software API than
1544 Whilst the above may seem to be severe minuses, there are some strong
1547 * Significant reduction of V's opcode space: over 95%.
1548 * Smaller reduction of P's opcode space: around 10%.
1549 * The potential to use Compressed instructions in both Vector and SIMD
1550 due to the overloading of register meaning (implicit vectorisation,
1552 * Not only present but also future extensions automatically gain parallelism.
1553 * Already mentioned but worth emphasising: the simplification to compiler
1554 writers and assembly-level writers of having the same consistent ISA
1555 regardless of whether the internal level of parallelism (number of
1556 parallel ALUs) is only equal to one ("virtual" parallelism), or is
1557 greater than one, should not be underestimated.
1559 ## Reducing Register Bank porting
1561 This looks quite reasonable.
1562 <https://www.princeton.edu/~rblee/ELE572Papers/MultiBankRegFile_ISCA2000.pdf>
1564 The main details are outlined on page 4. They propose a 2-level register
1565 cache hierarchy, note that registers are typically only read once, that
1566 you never write back from upper to lower cache level but always go in a
1567 cycle lower -> upper -> ALU -> lower, and at the top of page 5 propose
1568 a scheme where you look ahead by only 2 instructions to determine which
1569 registers to bring into the cache.
1571 The nice thing about a vector architecture is that you *know* that
1572 *even more* registers are going to be pulled in: Hwacha uses this fact
1573 to optimise L1/L2 cache-line usage (avoid thrashing), strangely enough
1574 by *introducing* deliberate latency into the execution phase.
1576 ## Overflow registers in combination with predication
1578 **TODO**: propose overflow registers be actually one of the integer regs
1579 (flowing to multiple regs).
1581 **TODO**: propose "mask" (predication) registers likewise. combination with
1582 standard RV instructions and overflow registers extremely powerful, see
1585 When integer overflow is stored in an easily-accessible bit (or another
1586 register), parallelisation turns this into a group of bits which can
1587 potentially be interacted with in predication, in interesting and powerful
1588 ways. For example, by taking the integer-overflow result as a predication
1589 field and shifting it by one, a predicated vectorised "add one" can emulate
1590 "carry" on arbitrary (unlimited) length addition.
1592 However despite RVV having made room for floating-point exceptions, neither
1593 RVV nor base RV have taken integer-overflow (carry) into account, which
1594 makes proposing it quite challenging given that the relevant (Base) RV
1595 sections are frozen. Consequently it makes sense to forgo this feature.
1597 ## Context Switch Example <a name="context_switch"></a>
1599 An unusual side-effect of Simple-V mapping onto the standard register files
1600 is that LOAD-multiple and STORE-multiple are accidentally available, as long
1601 as it is acceptable that the register(s) to be loaded/stored are contiguous
1602 (per instruction). An additional accidental benefit is that Compressed LD/ST
1605 To illustrate how this works, here is some example code from FreeRTOS
1606 (GPLv2 licensed, portasm.S):
1608 /* Macro for saving task context */
1609 .macro portSAVE_CONTEXT
1610 .global pxCurrentTCB
1611 /* make room in stack */
1612 addi sp, sp, -REGBYTES * 32
1616 STORE x2, 1 * REGBYTES(sp)
1617 STORE x3, 2 * REGBYTES(sp)
1620 STORE x30, 29 * REGBYTES(sp)
1621 STORE x31, 30 * REGBYTES(sp)
1623 /* Store current stackpointer in task control block (TCB) */
1624 LOAD t0, pxCurrentTCB //pointer
1628 /* Saves current error program counter (EPC) as task program counter */
1631 STORE t0, 31 * REGBYTES(sp)
1634 /* Saves current return adress (RA) as task program counter */
1636 STORE ra, 31 * REGBYTES(sp)
1639 /* Macro for restoring task context */
1640 .macro portRESTORE_CONTEXT
1642 .global pxCurrentTCB
1643 /* Load stack pointer from the current TCB */
1644 LOAD sp, pxCurrentTCB
1647 /* Load task program counter */
1648 LOAD t0, 31 * REGBYTES(sp)
1651 /* Run in machine mode */
1655 /* Restore registers,
1656 Skip global pointer because that does not change */
1658 LOAD x4, 3 * REGBYTES(sp)
1659 LOAD x5, 4 * REGBYTES(sp)
1662 LOAD x30, 29 * REGBYTES(sp)
1663 LOAD x31, 30 * REGBYTES(sp)
1665 addi sp, sp, REGBYTES * 32
1669 The important bits are the Load / Save context, which may be replaced
1670 with firstly setting up the Vectors and secondly using a *single* STORE
1671 (or LOAD) including using C.ST or C.LD, to indicate that the entire
1672 bank of registers is to be loaded/saved:
1674 /* a few things are assumed here: (a) that when switching to
1675 M-Mode an entirely different set of CSRs is used from that
1676 which is used in U-Mode and (b) that the M-Mode x1 and x4
1677 vectors are also not used anywhere else in M-Mode, consequently
1678 only need to be set up just the once.
1681 MVECTORCSRx1 = 31, defaultlen
1682 MVECTORCSRx4 = 28, defaultlen
1685 SETVL x0, x0, 31 /* x0 ignored silently */
1686 STORE x1, 0x0(sp) // x1 marked as 31-long vector of default bitwidth
1688 /* Restore registers,
1689 Skip global pointer because that does not change */
1691 SETVL x0, x0, 28 /* x0 ignored silently */
1692 LOAD x4, 3 * REGBYTES(sp) // x4 marked as 28-long default bitwidth
1694 Note that although it may just be a bug in portasm.S, x2 and x3 appear not
1695 to be being restored. If however this is a bug and they *do* need to be
1696 restored, then the SETVL call may be moved to *outside* the Save / Restore
1697 Context assembly code, into the macroVectorSetup, as long as vectors are
1698 never used anywhere else (i.e. VL is never altered by M-Mode).
1700 In effect the entire bank of repeated LOAD / STORE instructions is replaced
1701 by one single (compressed if it is available) instruction.
1703 ## Virtual Memory page-faults on LOAD/STORE
1706 ### Notes from conversations
1708 > I was going through the C.LOAD / C.STORE section 12.3 of V2.3-Draft
1709 > riscv-isa-manual in order to work out how to re-map RVV onto the standard
1710 > ISA, and came across an interesting comments at the bottom of pages 75
1713 > " A common mechanism used in other ISAs to further reduce save/restore
1714 > code size is load- multiple and store-multiple instructions. "
1716 > Fascinatingly, due to Simple-V proposing to use the *standard* register
1717 > file, both C.LOAD / C.STORE *and* LOAD / STORE would in effect be exactly
1718 > that: load-multiple and store-multiple instructions. Which brings us
1719 > on to this comment:
1721 > "For virtual memory systems, some data accesses could be resident in
1722 > physical memory and
1723 > some could not, which requires a new restart mechanism for partially
1724 > executed instructions."
1726 > Which then of course brings us to the interesting question: how does RVV
1727 > cope with the scenario when, particularly with LD.X (Indexed / indirect
1728 > loads), part-way through the loading a page fault occurs?
1730 > Has this been noted or discussed before?
1732 For applications-class platforms, the RVV exception model is
1733 element-precise (that is, if an exception occurs on element j of a
1734 vector instruction, elements 0..j-1 have completed execution and elements
1735 j+1..vl-1 have not executed).
1737 Certain classes of embedded platforms where exceptions are always fatal
1738 might choose to offer resumable/swappable interrupts but not precise
1742 > Is RVV designed in any way to be re-entrant?
1747 > What would the implications be for instructions that were in a FIFO at
1748 > the time, in out-of-order and VLIW implementations, where partial decode
1751 The usual bag of tricks for maintaining precise exceptions applies to
1752 vector machines as well. Register renaming makes the job easier, and
1753 it's relatively cheaper for vectors, since the control cost is amortized
1754 over longer registers.
1757 > Would it be reasonable at least to say *bypass* (and freeze) the
1758 > instruction FIFO (drop down to a single-issue execution model temporarily)
1759 > for the purposes of executing the instructions in the interrupt (whilst
1760 > setting up the VM page), then re-continue the instruction with all
1763 This approach has been done successfully, but it's desirable to be
1764 able to swap out the vector unit state to support context switches on
1765 exceptions that result in long-latency I/O.
1768 > Or would it be better to switch to an entirely separate secondary
1769 > hyperthread context?
1771 > Does anyone have any ideas or know if there is any academic literature
1772 > on solutions to this problem?
1774 The Vector VAX offered imprecise but restartable and swappable exceptions:
1775 http://mprc.pku.edu.cn/~liuxianhua/chn/corpus/Notes/articles/isca/1990/VAX%20vector%20architecture.pdf
1777 Sec. 4.6 of Krste's dissertation assesses some of
1778 the tradeoffs and references a bunch of related work:
1779 http://people.eecs.berkeley.edu/~krste/thesis.pdf
1784 Started reading section 4.6 of Krste's thesis, noted the "IEE85 F.P
1785 exceptions" and thought, "hmmm that could go into a CSR, must re-read
1786 the section on FP state CSRs in RVV 0.4-Draft again" then i suddenly
1787 thought, "ah ha! what if the memory exceptions were, instead of having
1788 an immediate exception thrown, were simply stored in a type of predication
1789 bit-field with a flag "error this element failed"?
1791 Then, *after* the vector load (or store, or even operation) was
1792 performed, you could *then* raise an exception, at which point it
1793 would be possible (yes in software... I know....) to go "hmmm, these
1794 indexed operations didn't work, let's get them into memory by triggering
1795 page-loads", then *re-run the entire instruction* but this time with a
1796 "memory-predication CSR" that stops the already-performed operations
1797 (whether they be loads, stores or an arithmetic / FP operation) from
1798 being carried out a second time.
1800 This theoretically could end up being done multiple times in an SMP
1801 environment, and also for LD.X there would be the remote outside annoying
1802 possibility that the indexed memory address could end up being modified.
1804 The advantage would be that the order of execution need not be
1805 sequential, which potentially could have some big advantages.
1806 Am still thinking through the implications as any dependent operations
1807 (particularly ones already decoded and moved into the execution FIFO)
1808 would still be there (and stalled). hmmm.
1812 > > # assume internal parallelism of 8 and MAXVECTORLEN of 8
1817 > > x3[1]: exception
1823 > > what happens to result elements 2-7? those may be *big* results
1825 > > or in the RVV-Extended may be arbitrary bit-widths far greater.
1831 discussion then led to the question of OoO architectures
1833 > The costs of the imprecise-exception model are greater than the benefit.
1834 > Software doesn't want to cope with it. It's hard to debug. You can't
1835 > migrate state between different microarchitectures--unless you force all
1836 > implementations to support the same imprecise-exception model, which would
1837 > greatly limit implementation flexibility. (Less important, but still
1838 > relevant, is that the imprecise model increases the size of the context
1839 > structure, as the microarchitectural guts have to be spilled to memory.)
1841 ## Zero/Non-zero Predication
1843 >> > it just occurred to me that there's another reason why the data
1844 >> > should be left instead of zeroed. if the standard register file is
1845 >> > used, such that vectorised operations are translated to mean "please
1846 >> > insert multiple register-contiguous operations into the instruction
1847 >> > FIFO" and predication is used to *skip* some of those, then if the
1848 >> > next "vector" operation uses the (standard) registers that were masked
1849 >> > *out* of the previous operation it may proceed without blocking.
1851 >> > if however zeroing is made mandatory then that optimisation becomes
1852 >> > flat-out impossible to deploy.
1854 >> > whilst i haven't fully thought through the full implications, i
1855 >> > suspect RVV might also be able to benefit by being able to fit more
1856 >> > overlapping operations into the available SRAM by doing something
1860 > Luke, this is called density time masking. It doesn’t apply to only your
1861 > model with the “standard register file” is used. it applies to any
1862 > architecture that attempts to speed up by skipping computation and writeback
1863 > of masked elements.
1865 > That said, the writing of zeros need not be explicit. It is possible to add
1866 > a “zero bit” per element that, when set, forces a zero to be read from the
1867 > vector (although the underlying storage may have old data). In this case,
1868 > there may be a way to implement DTM as well.
1871 ## Implementation detail for scalar-only op detection <a name="scalar_detection"></a>
1873 Note 1: this idea is a pipeline-bypass concept, which may *or may not* be
1876 Note 2: this is just one possible implementation. Another implementation
1877 may choose to treat *all* operations as vectorised (including treating
1878 scalars as vectors of length 1), choosing to add an extra pipeline stage
1879 dedicated to *all* instructions.
1881 This section *specifically* covers the implementor's freedom to choose
1882 that they wish to minimise disruption to an existing design by detecting
1883 "scalar-only operations", bypassing the vectorisation phase (which may
1884 or may not require an additional pipeline stage)
1886 [[scalardetect.png]]
1888 >> For scalar ops an implementation may choose to compare 2-3 bits through an
1889 >> AND gate: are src & dest scalar? Yep, ok send straight to ALU (or instr
1892 > Those bits cannot be known until after the registers are decoded from the
1893 > instruction and a lookup in the "vector length table" has completed.
1894 > Considering that one of the reasons RISC-V keeps registers in invariant
1895 > positions across all instructions is to simplify register decoding, I expect
1896 > that inserting an SRAM read would lengthen the critical path in most
1901 > briefly: the trick i mentioned about ANDing bits together to check if
1902 > an op was fully-scalar or not was to be read out of a single 32-bit
1903 > 3R1W SRAM (64-bit if FPU exists). the 32/64-bit SRAM contains 1 bit per
1904 > register indicating "is register vectorised yes no". 3R because you need
1905 > to check src1, src2 and dest simultaneously. the entries are *generated*
1906 > from the CSRs and are an optimisation that on slower embedded systems
1907 > would likely not be needed.
1909 > is there anything unreasonable that anyone can foresee about that?
1910 > what are the down-sides?
1913 ## Implementation Paradigms <a name="implementation_paradigms"></a>
1915 TODO: assess various implementation paradigms. These are listed roughly
1916 in order of simplicity (minimum compliance, for ultra-light-weight
1917 embedded systems or to reduce design complexity and the burden of
1918 design implementation and compliance, in non-critical areas), right the
1919 way to high-performance systems.
1921 * Full (or partial) software-emulated (via traps): full support for CSRs
1922 required, however when a register is used that is detected (in hardware)
1923 to be vectorised, an exception is thrown.
1924 * Single-issue In-order, reduced pipeline depth (traditional SIMD / DSP)
1925 * In-order 5+ stage pipelines with instruction FIFOs and mild register-renaming
1926 * Out-of-order with instruction FIFOs and aggressive register-renaming
1929 Also to be taken into consideration:
1931 * "Virtual" vectorisation: single-issue loop, no internal ALU parallelism
1932 * Comphrensive vectorisation: FIFOs and internal parallelism
1933 * Hybrid Parallelism
1935 ### Full or partial software-emulation
1937 The absolute, absolute minimal implementation is to provide the full
1938 set of CSRs and detection logic for when any of the source or destination
1939 registers are vectorised. On detection, a trap is thrown, whether it's
1940 a branch, LOAD, STORE, or an arithmetic operation.
1942 Implementors are entirely free to choose whether to allow absolutely every
1943 single operation to be software-emulated, or whether to provide some emulation
1944 and some hardware support. In particular, for an RV32E implementation
1945 where fast context-switching is a requirement (see "Context Switch Example"),
1946 it makes no sense to allow Vectorised-LOAD/STORE to be implemented as an
1947 exception, as every context-switch will result in double-traps.
1951 > For great floating point DSPs check TI’s C3x, C4X, and C6xx DSPs
1953 Idea: basic simple butterfly swap on a few element indices, primarily targetted
1954 at SIMD / DSP. High-byte low-byte swapping, high-word low-word swapping,
1955 perhaps allow reindexing of permutations up to 4 elements? 8? Reason:
1956 such operations are less costly than a full indexed-shuffle, which requires
1957 a separate instruction cycle.
1959 Predication "all zeros" needs to be "leave alone". Detection of
1960 ADD r1, rs1, rs0 cases result in nop on predication index 0, whereas
1961 ADD r0, rs1, rs2 is actually a desirable copy from r2 into r0.
1962 Destruction of destination indices requires a copy of the entire vector
1963 in advance to avoid.
1965 TBD: floating-point compare and other exception handling
1969 * SIMD considered harmful <https://www.sigarch.org/simd-instructions-considered-harmful/>
1970 * Link to first proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/GuukrSjgBH8>
1971 * Recommendation by Jacob Bachmeyer to make zero-overhead loop an
1972 "implicit program-counter" <https://groups.google.com/a/groups.riscv.org/d/msg/isa-dev/vYVi95gF2Mo/SHz6a4_lAgAJ>
1973 * Re-continuing P-Extension proposal <https://groups.google.com/a/groups.riscv.org/forum/#!msg/isa-dev/IkLkQn3HvXQ/SEMyC9IlAgAJ>
1974 * First Draft P-SIMD (DSP) proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/vYVi95gF2Mo>
1975 * B-Extension discussion <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/zi_7B15kj6s>
1976 * Broadcom VideoCore-IV <https://docs.broadcom.com/docs/12358545>
1977 Figure 2 P17 and Section 3 on P16.
1978 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-262.html>
1979 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-263.html>
1980 * Vector Workshop <http://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf>
1981 * Predication <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/XoP4BfYSLXA>
1982 * Branch Divergence <https://jbush001.github.io/2014/12/07/branch-divergence-in-parallel-kernels.html>
1983 * Life of Triangles (3D) <https://jbush001.github.io/2016/02/27/life-of-triangle.html>
1984 * Videocore-IV <https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-3d-Graphics-Pipeline>
1985 * Discussion proposing CSRs that change ISA definition
1986 <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/InzQ1wr_3Ak>
1987 * Zero-overhead loops <https://pdfs.semanticscholar.org/dbaa/66985cc730d4b44d79f519e96ec9c43ab5b7.pdf>
1988 * Multi-ported VLIW Register File Implementation <https://ce-publications.et.tudelft.nl/publications/1517_multiple_contexts_in_a_multiported_vliw_register_file_impl.pdf>
1989 * Fast context save/restore proposal <https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/57F823FA.6030701%40gmail.com>
1990 * Register File Bank Cacheing <https://www.princeton.edu/~rblee/ELE572Papers/MultiBankRegFile_ISCA2000.pdf>
1991 * Expired Patent on Vector Virtual Memory solutions
1992 <https://patentimages.storage.googleapis.com/fc/f6/e2/2cbee92fcd8743/US5895501.pdf>
1993 * Discussion on RVV "re-entrant" capabilities allowing operations to be
1994 restarted if an exception occurs (VM page-table miss)
1995 <https://groups.google.com/a/groups.riscv.org/d/msg/isa-dev/IuNFitTw9fM/CCKBUlzsAAAJ>
1996 * Dot Product Vector <https://people.eecs.berkeley.edu/~biancolin/papers/arith17.pdf>
1997 * RVV slides 2017 <https://content.riscv.org/wp-content/uploads/2017/12/Wed-1330-RISCVRogerEspasaVEXT-v4.pdf>
1998 * Wavefront skipping using BRAMS <http://www.ece.ubc.ca/~lemieux/publications/severance-fpga2015.pdf>
1999 * Streaming Pipelines <http://www.ece.ubc.ca/~lemieux/publications/severance-fpga2014.pdf>
2000 * Barcelona SIMD Presentation <https://content.riscv.org/wp-content/uploads/2018/05/09.05.2018-9.15-9.30am-RISCV201805-Andes-proposed-P-extension.pdf>
2001 * <http://www.ece.ubc.ca/~lemieux/publications/severance-fpga2015.pdf>