1 # Variable-width Variable-packed SIMD / Simple-V / Parallelism Extension Proposal
3 **Note: this document is out of date and involved early ideas and discussions**
5 Key insight: Simple-V is intended as an abstraction layer to provide
6 a consistent "API" to parallelisation of existing *and future* operations.
7 *Actual* internal hardware-level parallelism is *not* required, such
8 that Simple-V may be viewed as providing a "compact" or "consolidated"
9 means of issuing multiple near-identical arithmetic instructions to an
10 instruction queue (FIFO), pending execution.
12 *Actual* parallelism, if added independently of Simple-V in the form
13 of Out-of-order restructuring (including parallel ALU lanes) or VLIW
14 implementations, or SIMD, or anything else, would then benefit from
15 the uniformity of a consistent API.
17 **No arithmetic operations are added or required to be added.** SV is purely a parallelism API and consequentially is suitable for use even with RV32E.
19 * Talk slides: <http://hands.com/~lkcl/simple_v_chennai_2018.pdf>
20 * Specification: now move to its own page: [[specification]]
26 This proposal exists so as to be able to satisfy several disparate
27 requirements: power-conscious, area-conscious, and performance-conscious
28 designs all pull an ISA and its implementation in different conflicting
29 directions, as do the specific intended uses for any given implementation.
31 The existing P (SIMD) proposal and the V (Vector) proposals,
32 whilst each extremely powerful in their own right and clearly desirable,
35 * Clearly independent in their origins (Cray and AndesStar v3 respectively)
36 so need work to adapt to the RISC-V ethos and paradigm
37 * Are sufficiently large so as to make adoption (and exploration for
38 analysis and review purposes) prohibitively expensive
39 * Both contain partial duplication of pre-existing RISC-V instructions
40 (an undesirable characteristic)
41 * Both have independent, incompatible and disparate methods for introducing
42 parallelism at the instruction level
43 * Both require that their respective parallelism paradigm be implemented
44 along-side and integral to their respective functionality *or not at all*.
45 * Both independently have methods for introducing parallelism that
46 could, if separated, benefit
47 *other areas of RISC-V not just DSP or Floating-point respectively*.
49 There are also key differences between Vectorisation and SIMD (full
50 details outlined in the Appendix), the key points being:
52 * SIMD has an extremely seductively compelling ease of implementation argument:
53 each operation is passed to the ALU, which is where the parallelism
54 lies. There is *negligeable* (if any) impact on the rest of the core
55 (with life instead being made hell for compiler writers and applications
56 writers due to extreme ISA proliferation).
57 * By contrast, Vectorisation has quite some complexity (for considerable
58 flexibility, reduction in opcode proliferation and much more).
59 * Vectorisation typically includes much more comprehensive memory load
60 and store schemes (unit stride, constant-stride and indexed), which
61 in turn have ramifications: virtual memory misses (TLB cache misses)
62 and even multiple page-faults... all caused by a *single instruction*,
63 yet with a clear benefit that the regularisation of LOAD/STOREs can
64 be optimised for minimal impact on caches and maximised throughput.
65 * By contrast, SIMD can use "standard" memory load/stores (32-bit aligned
66 to pages), and these load/stores have absolutely nothing to do with the
67 SIMD / ALU engine, no matter how wide the operand. Simplicity but with
68 more impact on instruction and data caches.
70 Overall it makes a huge amount of sense to have a means and method
71 of introducing instruction parallelism in a flexible way that provides
72 implementors with the option to choose exactly where they wish to offer
73 performance improvements and where they wish to optimise for power
74 and/or area (and if that can be offered even on a per-operation basis that
75 would provide even more flexibility).
77 Additionally it makes sense to *split out* the parallelism inherent within
78 each of P and V, and to see if each of P and V then, in *combination* with
79 a "best-of-both" parallelism extension, could be added on *on top* of
80 this proposal, to topologically provide the exact same functionality of
81 each of P and V. Each of P and V then can focus on providing the best
82 operations possible for their respective target areas, without being
83 hugely concerned about the actual parallelism.
85 Furthermore, an additional goal of this proposal is to reduce the number
86 of opcodes utilised by each of P and V as they currently stand, leveraging
87 existing RISC-V opcodes where possible, and also potentially allowing
88 P and V to make use of Compressed Instructions as a result.
90 # Analysis and discussion of Vector vs SIMD
92 There are six combined areas between the two proposals that help with
93 parallelism (increased performance, reduced power / area) without
94 over-burdening the ISA with a huge proliferation of
97 * Fixed vs variable parallelism (fixed or variable "M" in SIMD)
98 * Implicit vs fixed instruction bit-width (integral to instruction or not)
99 * Implicit vs explicit type-conversion (compounded on bit-width)
100 * Implicit vs explicit inner loops.
101 * Single-instruction LOAD/STORE.
102 * Masks / tagging (selecting/preventing certain indexed elements from execution)
104 The pros and cons of each are discussed and analysed below.
106 ## Fixed vs variable parallelism length
108 In David Patterson and Andrew Waterman's analysis of SIMD and Vector
109 ISAs, the analysis comes out clearly in favour of (effectively) variable
110 length SIMD. As SIMD is a fixed width, typically 4, 8 or in extreme cases
111 16 or 32 simultaneous operations, the setup, teardown and corner-cases of SIMD
112 are extremely burdensome except for applications whose requirements
113 *specifically* match the *precise and exact* depth of the SIMD engine.
115 Thus, SIMD, no matter what width is chosen, is never going to be acceptable
116 for general-purpose computation, and in the context of developing a
117 general-purpose ISA, is never going to satisfy 100 percent of implementors.
119 To explain this further: for increased workloads over time, as the
120 performance requirements increase for new target markets, implementors
121 choose to extend the SIMD width (so as to again avoid mixing parallelism
122 into the instruction issue phases: the primary "simplicity" benefit of
123 SIMD in the first place), with the result that the entire opcode space
124 effectively doubles with each new SIMD width that's added to the ISA.
126 That basically leaves "variable-length vector" as the clear *general-purpose*
127 winner, at least in terms of greatly simplifying the instruction set,
128 reducing the number of instructions required for any given task, and thus
129 reducing power consumption for the same.
131 ## Implicit vs fixed instruction bit-width
133 SIMD again has a severe disadvantage here, over Vector: huge proliferation
134 of specialist instructions that target 8-bit, 16-bit, 32-bit, 64-bit, and
135 have to then have operations *for each and between each*. It gets very
136 messy, very quickly: *six* separate dimensions giving an O(N^6) instruction
137 proliferation profile.
139 The V-Extension on the other hand proposes to set the bit-width of
140 future instructions on a per-register basis, such that subsequent instructions
141 involving that register are *implicitly* of that particular bit-width until
142 otherwise changed or reset.
144 This has some extremely useful properties, without being particularly
145 burdensome to implementations, given that instruction decode already has
146 to direct the operation to a correctly-sized width ALU engine, anyway.
148 Not least: in places where an ISA was previously constrained (due for
149 whatever reason, including limitations of the available operand space),
150 implicit bit-width allows the meaning of certain operations to be
151 type-overloaded *without* pollution or alteration of frozen and immutable
152 instructions, in a fully backwards-compatible fashion.
154 ## Implicit and explicit type-conversion
156 The Draft 2.3 V-extension proposal has (deprecated) polymorphism to help
157 deal with over-population of instructions, such that type-casting from
158 integer (and floating point) of various sizes is automatically inferred
159 due to "type tagging" that is set with a special instruction. A register
160 will be *specifically* marked as "16-bit Floating-Point" and, if added
161 to an operand that is specifically tagged as "32-bit Integer" an implicit
162 type-conversion will take place *without* requiring that type-conversion
163 to be explicitly done with its own separate instruction.
165 However, implicit type-conversion is not only quite burdensome to
166 implement (explosion of inferred type-to-type conversion) but also is
167 never really going to be complete. It gets even worse when bit-widths
168 also have to be taken into consideration. Each new type results in
169 an increased O(N^2) conversion space that, as anyone who has examined
170 python's source code (which has built-in polymorphic type-conversion),
171 knows that the task is more complex than it first seems.
173 Overall, type-conversion is generally best to leave to explicit
174 type-conversion instructions, or in definite specific use-cases left to
175 be part of an actual instruction (DSP or FP)
177 ## Zero-overhead loops vs explicit loops
179 The initial Draft P-SIMD Proposal by Chuanhua Chang of Andes Technology
180 contains an extremely interesting feature: zero-overhead loops. This
181 proposal would basically allow an inner loop of instructions to be
182 repeated indefinitely, a fixed number of times.
184 Its specific advantage over explicit loops is that the pipeline in a DSP
185 can potentially be kept completely full *even in an in-order single-issue
186 implementation*. Normally, it requires a superscalar architecture and
187 out-of-order execution capabilities to "pre-process" instructions in
188 order to keep ALU pipelines 100% occupied.
190 By bringing that capability in, this proposal could offer a way to increase
191 pipeline activity even in simpler implementations in the one key area
192 which really matters: the inner loop.
194 However when looking at much more comprehensive schemes
195 "A portable specification of zero-overhead loop control hardware
196 applied to embedded processors" (ZOLC), optimising only the single
197 inner loop seems inadequate, tending to suggest that ZOLC may be
198 better off being proposed as an entirely separate Extension.
200 ## Single-instruction LOAD/STORE
202 In traditional Vector Architectures there are instructions which
203 result in multiple register-memory transfer operations resulting
204 from a single instruction. They're complicated to implement in hardware,
205 yet the benefits are a huge consistent regularisation of memory accesses
206 that can be highly optimised with respect to both actual memory and any
207 L1, L2 or other caches. In Hwacha EECS-2015-263 it is explicitly made
208 clear the consequences of getting this architecturally wrong:
209 L2 cache-thrashing at the very least.
211 Complications arise when Virtual Memory is involved: TLB cache misses
212 need to be dealt with, as do page faults. Some of the tradeoffs are
213 discussed in <http://people.eecs.berkeley.edu/~krste/thesis.pdf>, Section
214 4.6, and an article by Jeff Bush when faced with some of these issues
215 is particularly enlightening
216 <https://jbush001.github.io/2015/11/03/lost-in-translation.html>
218 Interestingly, none of this complexity is faced in SIMD architectures...
219 but then they do not get the opportunity to optimise for highly-streamlined
220 memory accesses either.
222 With the "bang-per-buck" ratio being so high and the indirect improvement
223 in L1 Instruction Cache usage (reduced instruction count), as well as
224 the opportunity to optimise L1 and L2 cache usage, the case for including
225 Vector LOAD/STORE is compelling.
227 ## Mask and Tagging (Predication)
229 Tagging (aka Masks aka Predication) is a pseudo-method of implementing
230 simplistic branching in a parallel fashion, by allowing execution on
231 elements of a vector to be switched on or off depending on the results
232 of prior operations in the same array position.
234 The reason for considering this is simple: by *definition* it
235 is not possible to perform individual parallel branches in a SIMD
236 (Single-Instruction, **Multiple**-Data) context. Branches (modifying
237 of the Program Counter) will result in *all* parallel data having
238 a different instruction executed on it: that's just the definition of
239 SIMD, and it is simply unavoidable.
241 So these are the ways in which conditional execution may be implemented:
243 * explicit compare and branch: BNE x, y -> offs would jump offs
244 instructions if x was not equal to y
245 * explicit store of tag condition: CMP x, y -> tagbit
246 * implicit (condition-code) such as ADD results in a carry, carry bit
247 implicitly (or sometimes explicitly) goes into a "tag" (mask) register
249 The first of these is a "normal" branch method, which is flat-out impossible
250 to parallelise without look-ahead and effectively rewriting instructions.
251 This would defeat the purpose of RISC.
253 The latter two are where parallelism becomes easy to do without complexity:
254 every operation is modified to be "conditionally executed" (in an explicit
255 way directly in the instruction format *or* implicitly).
257 RVV (Vector-Extension) proposes to have *explicit* storing of the compare
258 in a tag/mask register, and to *explicitly* have every vector operation
259 *require* that its operation be "predicated" on the bits within an
260 explicitly-named tag/mask register.
262 SIMD (P-Extension) has not yet published precise documentation on what its
263 schema is to be: there is however verbal indication at the time of writing
266 > The "compare" instructions in the DSP/SIMD ISA proposed by Andes will
267 > be executed using the same compare ALU logic for the base ISA with some
268 > minor modifications to handle smaller data types. The function will not
271 This is an *implicit* form of predication as the base RV ISA does not have
272 condition-codes or predication. By adding a CSR it becomes possible
273 to also tag certain registers as "predicated if referenced as a destination".
276 // in future operations from now on, if r0 is the destination use r5 as
277 // the PREDICATION register
278 SET_IMPLICIT_CSRPREDICATE r0, r5
279 // store the compares in r5 as the PREDICATION register
281 // r0 is used here. ah ha! that means it's predicated using r5!
284 With enough registers (and in RISC-V there are enough registers) some fairly
285 complex predication can be set up and yet still execute without significant
286 stalling, even in a simple non-superscalar architecture.
288 (For details on how Branch Instructions would be retro-fitted to indirectly
289 predicated equivalents, see Appendix)
293 In the above sections the five different ways where parallel instruction
294 execution has closely and loosely inter-related implications for the ISA and
295 for implementors, were outlined. The pluses and minuses came out as
298 * Fixed vs variable parallelism: <b>variable</b>
299 * Implicit (indirect) vs fixed (integral) instruction bit-width: <b>indirect</b>
300 * Implicit vs explicit type-conversion: <b>explicit</b>
301 * Implicit vs explicit inner loops: <b>implicit but best done separately</b>
302 * Single-instruction Vector LOAD/STORE: <b>Complex but highly beneficial</b>
303 * Tag or no-tag: <b>Complex but highly beneficial</b>
307 * variable-length vectors came out on top because of the high setup, teardown
308 and corner-cases associated with the fixed width of SIMD.
309 * Implicit bit-width helps to extend the ISA to escape from
310 former limitations and restrictions (in a backwards-compatible fashion),
311 whilst also leaving implementors free to simmplify implementations
312 by using actual explicit internal parallelism.
313 * Implicit (zero-overhead) loops provide a means to keep pipelines
314 potentially 100% occupied in a single-issue in-order implementation
315 i.e. *without* requiring a super-scalar or out-of-order architecture,
316 but doing a proper, full job (ZOLC) is an entirely different matter.
318 Constructing a SIMD/Simple-Vector proposal based around four of these six
319 requirements would therefore seem to be a logical thing to do.
321 # Note on implementation of parallelism
323 One extremely important aspect of this proposal is to respect and support
324 implementors desire to focus on power, area or performance. In that regard,
325 it is proposed that implementors be free to choose whether to implement
326 the Vector (or variable-width SIMD) parallelism as sequential operations
327 with a single ALU, fully parallel (if practical) with multiple ALUs, or
328 a hybrid combination of both.
330 In Broadcom's Videocore-IV, they chose hybrid, and called it "Virtual
331 Parallelism". They achieve a 16-way SIMD at an **instruction** level
332 by providing a combination of a 4-way parallel ALU *and* an externally
333 transparent loop that feeds 4 sequential sets of data into each of the
336 Also in the same core, it is worth noting that particularly uncommon
337 but essential operations (Reciprocal-Square-Root for example) are
338 *not* part of the 4-way parallel ALU but instead have a *single* ALU.
339 Under the proposed Vector (varible-width SIMD) implementors would
340 be free to do precisely that: i.e. free to choose *on a per operation
341 basis* whether and how much "Virtual Parallelism" to deploy.
343 It is absolutely critical to note that it is proposed that such choices MUST
344 be **entirely transparent** to the end-user and the compiler. Whilst
345 a Vector (varible-width SIMD) may not precisely match the width of the
346 parallelism within the implementation, the end-user **should not care**
347 and in this way the performance benefits are gained but the ISA remains
348 straightforward. All that happens at the end of an instruction run is: some
349 parallel units (if there are any) would remain offline, completely
350 transparently to the ISA, the program, and the compiler.
352 To make that clear: should an implementor choose a particularly wide
353 SIMD-style ALU, each parallel unit *must* have predication so that
354 the parallel SIMD ALU may emulate variable-length parallel operations.
355 Thus the "SIMD considered harmful" trap of having huge complexity and extra
356 instructions to deal with corner-cases is thus avoided, and implementors
357 get to choose precisely where to focus and target the benefits of their
358 implementation efforts, without "extra baggage".
360 In addition, implementors will be free to choose whether to provide an
361 absolute bare minimum level of compliance with the "API" (software-traps
362 when vectorisation is detected), all the way up to full supercomputing
363 level all-hardware parallelism. Options are covered in the Appendix.
366 ### FMV, FNEG and FABS Instructions
368 These are identical in form to C.MV, except covering floating-point
369 register copying. The same double-predication rules also apply.
370 However when elwidth is not set to default the instruction is implicitly
371 and automatic converted to a (vectorised) floating-point type conversion
372 operation of the appropriate size covering the source and destination
375 (Note that FMV, FNEG and FABS are all actually pseudo-instructions)
377 ### FVCT Instructions
379 These are again identical in form to C.MV, except that they cover
380 floating-point to integer and integer to floating-point. When element
381 width in each vector is set to default, the instructions behave exactly
382 as they are defined for standard RV (scalar) operations, except vectorised
383 in exactly the same fashion as outlined in C.MV.
385 However when the source or destination element width is not set to default,
386 the opcode's explicit element widths are *over-ridden* to new definitions,
387 and the opcode's element width is taken as indicative of the SIMD width
388 (if applicable i.e. if packed SIMD is requested) instead.
390 For example FCVT.S.L would normally be used to convert a 64-bit
391 integer in register rs1 to a 64-bit floating-point number in rd.
392 If however the source rs1 is set to be a vector, where elwidth is set to
393 default/2 and "packed SIMD" is enabled, then the first 32 bits of
394 rs1 are converted to a floating-point number to be stored in rd's
395 first element and the higher 32-bits *also* converted to floating-point
396 and stored in the second. The 32 bit size comes from the fact that
397 FCVT.S.L's integer width is 64 bit, and with elwidth on rs1 set to
398 divide that by two it means that rs1 element width is to be taken as 32.
400 Similar rules apply to the destination register.
404 > What does an ADD of two different-sized vectors do in simple-V?
406 * if the two source operands are not the same, throw an exception.
407 * if the destination operand is also a vector, and the source is longer
408 than the destination, throw an exception.
410 > And what about instructions like JALR?
411 > What does jumping to a vector do?
413 * Throw an exception. Whether that actually results in spawning threads
414 as part of the trap-handling remains to be seen.
416 # Under consideration <a name="issues"></a>
418 From the Chennai 2018 slides the following issues were raised.
419 Efforts to analyse and answer these questions are below.
421 * Should future extra bank be included now?
422 * How many Register and Predication CSRs should there be?
423 (and how many in RV32E)
424 * How many in M-Mode (for doing context-switch)?
425 * Should use of registers be allowed to "wrap" (x30 x31 x1 x2)?
426 * Can CLIP be done as a CSR (mode, like elwidth)
427 * SIMD saturation (etc.) also set as a mode?
428 * Include src1/src2 predication on Comparison Ops?
429 (same arrangement as C.MV, with same flexibility/power)
430 * 8/16-bit ops is it worthwhile adding a "start offset"?
431 (a bit like misaligned addressing... for registers)
432 or just use predication to skip start?
434 ## Future (extra) bank be included (made mandatory)
436 The implications of expanding the *standard* register file from
437 32 entries per bank to 64 per bank is quite an extensive architectural
438 change. Also it has implications for context-switching.
440 Therefore, on balance, it is not recommended and certainly should
441 not be made a *mandatory* requirement for the use of SV. SV's design
442 ethos is to be minimally-disruptive for implementors to shoe-horn
443 into an existing design.
445 ## How large should the Register and Predication CSR key-value stores be?
447 This is something that definitely needs actual evaluation and for
448 code to be run and the results analysed. At the time of writing
449 (12jul2018) that is too early to tell. An approximate best-guess
450 however would be 16 entries.
452 RV32E however is a special case, given that it is highly unlikely
453 (but not outside the realm of possibility) that it would be used
454 for performance reasons but instead for reducing instruction count.
455 The number of CSR entries therefore has to be considered extremely
458 ## How many CSR entries in M-Mode or S-Mode (for context-switching)?
460 The minimum required CSR entries would be 1 for each register-bank:
461 one for integer and one for floating-point. However, as shown
462 in the "Context Switch Example" section, for optimal efficiency
463 (minimal instructions in a low-latency situation) the CSRs for
464 the context-switch should be set up *and left alone*.
466 This means that it is not really a good idea to touch the CSRs
467 used for context-switching in the M-Mode (or S-Mode) trap, so
468 if there is ever demonstrated a need for vectors then there would
469 need to be *at least* one more free. However just one does not make
470 much sense (as it one only covers scalar-vector ops) so it is more
471 likely that at least two extra would be needed.
473 This *in addition* - in the RV32E case - if an RV32E implementation
474 happens also to support U/S/M modes. This would be considered quite
475 rare but not outside of the realm of possibility.
477 Conclusion: all needs careful analysis and future work.
479 ## Should use of registers be allowed to "wrap" (x30 x31 x1 x2)?
481 On balance it's a neat idea however it does seem to be one where the
482 benefits are not really clear. It would however obviate the need for
483 an exception to be raised if the VL runs out of registers to put
484 things in (gets to x31, tries a non-existent x32 and fails), however
485 the "fly in the ointment" is that x0 is hard-coded to "zero". The
486 increment therefore would need to be double-stepped to skip over x0.
487 Some microarchitectures could run into difficulties (SIMD-like ones
488 in particular) so it needs a lot more thought.
490 ## Can CLIP be done as a CSR (mode, like elwidth)
492 RVV appears to be going this way. At the time of writing (12jun2018)
493 it's noted that in V2.3-Draft V0.4 RVV Chapter, RVV intends to do
494 clip by way of exactly this method: setting a "clip mode" in a CSR.
496 No details are given however the most sensible thing to have would be
497 to extend the 16-bit Register CSR table to 24-bit (or 32-bit) and have
498 extra bits specifying the type of clipping to be carried out, on
499 a per-register basis. Other bits may be used for other purposes
500 (see SIMD saturation below)
502 ## SIMD saturation (etc.) also set as a mode?
504 Similar to "CLIP" as an extension to the CSR key-value store, "saturate"
505 may also need extra details (what the saturation maximum is for example).
507 ## Include src1/src2 predication on Comparison Ops?
509 In the C.MV (and other ops - see "C.MV Instruction"), the decision
510 was taken, unlike in ADD (etc.) which are 3-operand ops, to use
511 *both* the src *and* dest predication masks to give an extremely
512 powerful and flexible instruction that covers a huge number of
513 "traditional" vector opcodes.
515 The natural question therefore to ask is: where else could this
516 flexibility be deployed? What about comparison operations?
518 Unfortunately, C.MV is basically "regs[dest] = regs[src]" whilst
519 predicated comparison operations are actually a *three* operand
522 regs[pred] |= 1<< (cmp(regs[src1], regs[src2]) ? 1 : 0)
524 Therefore at first glance it does not make sense to use src1 and src2
525 predication masks, as it breaks the rule of 3-operand instructions
526 to use the *destination* predication register.
528 In this case however, the destination *is* a predication register
529 as opposed to being a predication mask that is applied *to* the
530 (vectorised) operation, element-at-a-time on src1 and src2.
532 Thus the question is directly inter-related to whether the modification
533 of the predication mask should *itself* be predicated.
535 It is quite complex, in other words, and needs careful consideration.
537 ## 8/16-bit ops is it worthwhile adding a "start offset"?
539 The idea here is to make it possible, particularly in a "Packed SIMD"
540 case, to be able to avoid doing unaligned Load/Store operations
541 by specifying that operations, instead of being carried out
542 element-for-element, are offset by a fixed amount *even* in 8 and 16-bit
543 element Packed SIMD cases.
545 For example rather than take 2 32-bit registers divided into 4 8-bit
546 elements and have them ADDed element-for-element as follows:
548 r3[0] = add r4[0], r6[0]
549 r3[1] = add r4[1], r6[1]
550 r3[2] = add r4[2], r6[2]
551 r3[3] = add r4[3], r6[3]
553 an offset of 1 would result in four operations as follows, instead:
555 r3[0] = add r4[1], r6[0]
556 r3[1] = add r4[2], r6[1]
557 r3[2] = add r4[3], r6[2]
558 r3[3] = add r5[0], r6[3]
560 In non-packed-SIMD mode there is no benefit at all, as a vector may
561 be created using a different CSR that has the offset built-in. So this
562 leaves just the packed-SIMD case to consider.
564 Two ways in which this could be implemented / emulated (without special
567 * bit-manipulation that shuffles the data along by one byte (or one word)
568 either prior to or as part of the operation requiring the offset.
569 * just use an unaligned Load/Store sequence, even if there are performance
570 penalties for doing so.
572 The question then is whether the performance hit is worth the extra hardware
573 involving byte-shuffling/shifting the data by an arbitrary offset. On
574 balance given that there are two reasonable instruction-based options, the
575 hardware-offset option should be left out for the initial version of SV,
576 with the option to consider it in an "advanced" version of the specification.
578 # Impementing V on top of Simple-V
580 With Simple-V converting the original RVV draft concept-for-concept
581 from explicit opcodes to implicit overloading of existing RV Standard
582 Extensions, certain features were (deliberately) excluded that need
583 to be added back in for RVV to reach its full potential. This is
584 made slightly complicated by the fact that RVV itself has two
585 levels: Base and reserved future functionality.
587 * Representation Encoding is entirely left out of Simple-V in favour of
588 implicitly taking the exact (explicit) meaning from RV Standard Extensions.
589 * VCLIP and VCLIPI do not have corresponding RV Standard Extension
590 opcodes (and are the only such operations).
591 * Extended Element bitwidths (1 through to 24576 bits) were left out
592 of Simple-V as, again, there is no corresponding RV Standard Extension
593 that covers anything even below 32-bit operands.
594 * Polymorphism was entirely left out of Simple-V due to the inherent
595 complexity of automatic type-conversion.
596 * Vector Register files were specifically left out of Simple-V in favour
597 of fitting on top of the integer and floating-point files. An
598 "RVV re-retro-fit" needs to be able to mark (implicitly marked)
599 registers as being actually in a separate *vector* register file.
600 * Fortunately in RVV (Draft 0.4, V2.3-Draft), the "base" vector
601 register file size is 5 bits (32 registers), whilst the "Extended"
602 variant of RVV specifies 8 bits (256 registers) and has yet to
604 * One big difference: Sections 17.12 and 17.17, there are only two possible
605 predication registers in RVV "Base". Through the "indirect" method,
606 Simple-V provides a key-value CSR table that allows (arbitrarily)
607 up to 16 (TBD) of either the floating-point or integer registers to
608 be marked as "predicated" (key), and if so, which integer register to
609 use as the predication mask (value).
613 # Implementing P (renamed to DSP) on top of Simple-V
615 * Implementors indicate chosen bitwidth support in Vector-bitwidth CSR
616 (caveat: anything not specified drops through to software-emulation / traps)
621 ## V-Extension to Simple-V Comparative Analysis
623 This section has been moved to its own page [[v_comparative_analysis]]
627 This section has been moved to its own page [[p_comparative_analysis]]
629 ## Comparison of "Traditional" SIMD, Alt-RVP, Simple-V and RVV Proposals <a name="parallelism_comparisons"></a>
631 This section compares the various parallelism proposals as they stand,
632 including traditional SIMD, in terms of features, ease of implementation,
633 complexity, flexibility, and die area.
635 ### [[harmonised_rvv_rvp]]
637 This is an interesting proposal under development to retro-fit the AndesStar
642 Primary benefit of Alt-RVP is the simplicity with which parallelism
643 may be introduced (effective multiplication of regfiles and associated ALUs).
645 * plus: the simplicity of the lanes (combined with the regularity of
646 allocating identical opcodes multiple independent registers) meaning
647 that SRAM or 2R1W can be used for entire regfile (potentially).
648 * minus: a more complex instruction set where the parallelism is much
649 more explicitly directly specified in the instruction and
650 * minus: if you *don't* have an explicit instruction (opcode) and you
651 need one, the only place it can be added is... in the vector unit and
652 * minus: opcode functions (and associated ALUs) duplicated in Alt-RVP are
653 not useable or accessible in other Extensions.
654 * plus-and-minus: Lanes may be utilised for high-speed context-switching
655 but with the down-side that they're an all-or-nothing part of the Extension.
656 No Alt-RVP: no fast register-bank switching.
657 * plus: Lane-switching would mean that complex operations not suited to
658 parallelisation can be carried out, followed by further parallel Lane-based
659 work, without moving register contents down to memory (and back)
660 * minus: Access to registers across multiple lanes is challenging. "Solution"
661 is to drop data into memory and immediately back in again (like MMX).
665 Primary benefit of Simple-V is the OO abstraction of parallel principles
666 from actual (internal) parallel hardware. It's an API in effect that's
667 designed to be slotted in to an existing implementation (just after
668 instruction decode) with minimum disruption and effort.
670 * minus: the complexity (if full parallelism is to be exploited)
671 of having to use register renames, OoO, VLIW, register file cacheing,
672 all of which has been done before but is a pain
673 * plus: transparent re-use of existing opcodes as-is just indirectly
674 saying "this register's now a vector" which
675 * plus: means that future instructions also get to be inherently
676 parallelised because there's no "separate vector opcodes"
677 * plus: Compressed instructions may also be (indirectly) parallelised
678 * minus: the indirect nature of Simple-V means that setup (setting
679 a CSR register to indicate vector length, a separate one to indicate
680 that it is a predicate register and so on) means a little more setup
681 time than Alt-RVP or RVV's "direct and within the (longer) instruction"
683 * plus: shared register file meaning that, like Alt-RVP, complex
684 operations not suited to parallelisation may be carried out interleaved
685 between parallelised instructions *without* requiring data to be dropped
686 down to memory and back (into a separate vectorised register engine).
687 * plus-and-maybe-minus: re-use of integer and floating-point 32-wide register
688 files means that huge parallel workloads would use up considerable
689 chunks of the register file. However in the case of RV64 and 32-bit
690 operations, that effectively means 64 slots are available for parallel
692 * plus: inherent parallelism (actual parallel ALUs) doesn't actually need to
693 be added, yet the instruction opcodes remain unchanged (and still appear
694 to be parallel). consistent "API" regardless of actual internal parallelism:
695 even an in-order single-issue implementation with a single ALU would still
696 appear to have parallel vectoristion.
697 * hard-to-judge: if actual inherent underlying ALU parallelism is added it's
698 hard to say if there would be pluses or minuses (on die area). At worse it
699 would be "no worse" than existing register renaming, OoO, VLIW and register
700 file cacheing schemes.
702 ### RVV (as it stands, Draft 0.4 Section 17, RISC-V ISA V2.3-Draft)
704 RVV is extremely well-designed and has some amazing features, including
705 2D reorganisation of memory through LOAD/STORE "strides".
707 * plus: regular predictable workload means that implementations may
708 streamline effects on L1/L2 Cache.
709 * plus: regular and clear parallel workload also means that lanes
710 (similar to Alt-RVP) may be used as an implementation detail,
711 using either SRAM or 2R1W registers.
712 * plus: separate engine with no impact on the rest of an implementation
713 * minus: separate *complex* engine with no RTL (ALUs, Pipeline stages) reuse
715 * minus: no ISA abstraction or re-use either: additions to other Extensions
716 do not gain parallelism, resulting in prolific duplication of functionality
717 inside RVV *and out*.
718 * minus: when operations require a different approach (scalar operations
719 using the standard integer or FP regfile) an entire vector must be
720 transferred out to memory, into standard regfiles, then back to memory,
721 then back to the vector unit, this to occur potentially multiple times.
722 * minus: will never fit into Compressed instruction space (as-is. May
723 be able to do so if "indirect" features of Simple-V are partially adopted).
724 * plus-and-slight-minus: extended variants may address up to 256
725 vectorised registers (requires 48/64-bit opcodes to do it).
726 * minus-and-partial-plus: separate engine plus complexity increases
727 implementation time and die area, meaning that adoption is likely only
728 to be in high-performance specialist supercomputing (where it will
729 be absolutely superb).
733 The only really good things about SIMD are how easy it is to implement and
734 get good performance. Unfortunately that makes it quite seductive...
736 * plus: really straightforward, ALU basically does several packed operations
737 at once. Parallelism is inherent at the ALU, making the addition of
738 SIMD-style parallelism an easy decision that has zero significant impact
739 on the rest of any given architectural design and layout.
740 * plus (continuation): SIMD in simple in-order single-issue designs can
741 therefore result in superb throughput, easily achieved even with a very
742 simple execution model.
743 * minus: ridiculously complex setup and corner-cases that disproportionately
744 increase instruction count on what would otherwise be a "simple loop",
745 should the number of elements in an array not happen to exactly match
746 the SIMD group width.
747 * minus: getting data usefully out of registers (if separate regfiles
748 are used) means outputting to memory and back.
749 * minus: quite a lot of supplementary instructions for bit-level manipulation
750 are needed in order to efficiently extract (or prepare) SIMD operands.
751 * minus: MASSIVE proliferation of ISA both in terms of opcodes in one
752 dimension and parallelism (width): an at least O(N^2) and quite probably
753 O(N^3) ISA proliferation that often results in several thousand
754 separate instructions. all requiring separate and distinct corner-case
756 * minus: EVEN BIGGER proliferation of SIMD ISA if the functionality of
757 8, 16, 32 or 64-bit reordering is built-in to the SIMD instruction.
758 For example: add (high|low) 16-bits of r1 to (low|high) of r2 requires
759 four separate and distinct instructions: one for (r1:low r2:high),
760 one for (r1:high r2:low), one for (r1:high r2:high) and one for
761 (r1:low r2:low) *per function*.
762 * minus: EVEN BIGGER proliferation of SIMD ISA if there is a mismatch
763 between operand and result bit-widths. In combination with high/low
764 proliferation the situation is made even worse.
765 * minor-saving-grace: some implementations *may* have predication masks
766 that allow control over individual elements within the SIMD block.
768 ## Comparison *to* Traditional SIMD: Alt-RVP, Simple-V and RVV Proposals <a name="simd_comparison"></a>
770 This section compares the various parallelism proposals as they stand,
771 *against* traditional SIMD as opposed to *alongside* SIMD. In other words,
772 the question is asked "How can each of the proposals effectively implement
773 (or replace) SIMD, and how effective would they be"?
777 * Alt-RVP would not actually replace SIMD but would augment it: just as with
778 a SIMD architecture where the ALU becomes responsible for the parallelism,
779 Alt-RVP ALUs would likewise be so responsible... with *additional*
780 (lane-based) parallelism on top.
781 * Thus at least some of the downsides of SIMD ISA O(N^5) proliferation by
782 at least one dimension are avoided (architectural upgrades introducing
783 128-bit then 256-bit then 512-bit variants of the exact same 64-bit
785 * Thus, unfortunately, Alt-RVP would suffer the same inherent proliferation
786 of instructions as SIMD, albeit not quite as badly (due to Lanes).
787 * In the same discussion for Alt-RVP, an additional proposal was made to
788 be able to subdivide the bits of each register lane (columns) down into
789 arbitrary bit-lengths (RGB 565 for example).
790 * A recommendation was given instead to make the subdivisions down to 32-bit,
791 16-bit or even 8-bit, effectively dividing the registerfile into
792 Lane0(H), Lane0(L), Lane1(H) ... LaneN(L) or further. If inter-lane
793 "swapping" instructions were then introduced, some of the disadvantages
794 of SIMD could be mitigated.
798 * RVV is designed to replace SIMD with a better paradigm: arbitrary-length
800 * However whilst SIMD is usually designed for single-issue in-order simple
801 DSPs with a focus on Multimedia (Audio, Video and Image processing),
802 RVV's primary focus appears to be on Supercomputing: optimisation of
803 mathematical operations that fit into the OpenCL space.
804 * Adding functions (operations) that would normally fit (in parallel)
805 into a SIMD instruction requires an equivalent to be added to the
806 RVV Extension, if one does not exist. Given the specialist nature of
807 some SIMD instructions (8-bit or 16-bit saturated or halving add),
808 this possibility seems extremely unlikely to occur, even if the
809 implementation overhead of RVV were acceptable (compared to
810 normal SIMD/DSP-style single-issue in-order simplicity).
814 * Simple-V borrows hugely from RVV as it is intended to be easy to
815 topologically transplant every single instruction from RVV (as
816 designed) into Simple-V equivalents, with *zero loss of functionality
818 * With the "parallelism" abstracted out, a hypothetical SIMD-less "DSP"
819 Extension which contained the basic primitives (non-parallelised
820 8, 16 or 32-bit SIMD operations) inherently *become* parallel,
822 * Additionally, standard operations (ADD, MUL) that would normally have
823 to have special SIMD-parallel opcodes added need no longer have *any*
824 of the length-dependent variants (2of 32-bit ADDs in a 64-bit register,
825 4of 32-bit ADDs in a 128-bit register) because Simple-V takes the
826 *standard* RV opcodes (present and future) and automatically parallelises
828 * By inheriting the RVV feature of arbitrary vector-length, then just as
829 with RVV the corner-cases and ISA proliferation of SIMD is avoided.
830 * Whilst not entirely finalised, registers are expected to be
831 capable of being subdivided down to an implementor-chosen bitwidth
832 in the underlying hardware (r1 becomes r1[31..24] r1[23..16] r1[15..8]
833 and r1[7..0], or just r1[31..16] r1[15..0]) where implementors can
834 choose to have separate independent 8-bit ALUs or dual-SIMD 16-bit
835 ALUs that perform twin 8-bit operations as they see fit, or anything
836 else including no subdivisions at all.
837 * Even though implementors have that choice even to have full 64-bit
838 (with RV64) SIMD, they *must* provide predication that transparently
839 switches off appropriate units on the last loop, thus neatly fitting
840 underlying SIMD ALU implementations *into* the arbitrary vector-length
841 RVV paradigm, keeping the uniform consistent API that is a key strategic
843 * With Simple-V fitting into the standard register files, certain classes
844 of SIMD operations such as High/Low arithmetic (r1[31..16] + r2[15..0])
845 can be done by applying *Parallelised* Bit-manipulation operations
846 followed by parallelised *straight* versions of element-to-element
847 arithmetic operations, even if the bit-manipulation operations require
848 changing the bitwidth of the "vectors" to do so. Predication can
849 be utilised to skip high words (or low words) in source or destination.
850 * In essence, the key downside of SIMD - massive duplication of
851 identical functions over time as an architecture evolves from 32-bit
852 wide SIMD all the way up to 512-bit, is avoided with Simple-V, through
853 vector-style parallelism being dropped on top of 8-bit or 16-bit
854 operations, all the while keeping a consistent ISA-level "API" irrespective
855 of implementor design choices (or indeed actual implementations).
857 ### Example Instruction translation: <a name="example_translation"></a>
859 Instructions "ADD r7 r4 r4" would result in three instructions being
860 generated and placed into the FIFO. r7 and r4 are marked as "vectorised":
866 Instructions "ADD r7 r4 r1" would result in three instructions being
867 generated and placed into the FIFO. r7 and r1 are marked as "vectorised"
874 ## Example of vector / vector, vector / scalar, scalar / scalar => vector add
876 function op_add(rd, rs1, rs2) # add not VADD!
877 int i, id=0, irs1=0, irs2=0;
878 rd = int_vec[rd ].isvector ? int_vec[rd ].regidx : rd;
879 rs1 = int_vec[rs1].isvector ? int_vec[rs1].regidx : rs1;
880 rs2 = int_vec[rs2].isvector ? int_vec[rs2].regidx : rs2;
881 predval = get_pred_val(FALSE, rd);
882 for (i = 0; i < VL; i++)
883 if (predval & 1<<i) # predication uses intregs
884 ireg[rd+id] <= ireg[rs1+irs1] + ireg[rs2+irs2];
885 if (int_vec[rd ].isvector) { id += 1; }
886 if (int_vec[rs1].isvector) { irs1 += 1; }
887 if (int_vec[rs2].isvector) { irs2 += 1; }
889 ## Retro-fitting Predication into branch-explicit ISA <a name="predication_retrofit"></a>
891 One of the goals of this parallelism proposal is to avoid instruction
892 duplication. However, with the base ISA having been designed explictly
893 to *avoid* condition-codes entirely, shoe-horning predication into it
894 bcomes quite challenging.
896 However what if all branch instructions, if referencing a vectorised
897 register, were instead given *completely new analogous meanings* that
898 resulted in a parallel bit-wise predication register being set? This
899 would have to be done for both C.BEQZ and C.BNEZ, as well as BEQ, BNE,
902 We might imagine that FEQ, FLT and FLT would also need to be converted,
903 however these are effectively *already* in the precise form needed and
904 do not need to be converted *at all*! The difference is that FEQ, FLT
905 and FLE *specifically* write a 1 to an integer register if the condition
906 holds, and 0 if not. All that needs to be done here is to say, "if
907 the integer register is tagged with a bit that says it is a predication
908 register, the **bit** in the integer register is set based on the
909 current vector index" instead.
911 There is, in the standard Conditional Branch instruction, more than
912 adequate space to interpret it in a similar fashion:
915 31 |30 ..... 25 |24..20|19..15| 14...12| 11.....8 | 7 | 6....0 |
916 imm[12] | imm[10:5] |rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
917 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
918 offset[12,10:5] || src2 | src1 | BEQ | offset[11,4:1] || BRANCH |
924 31 | 30 .. 25 |24 ... 20 | 19 15 | 14 12 | 11 .. 8 | 7 | 6 ... 0 |
925 imm[12] | imm[10:5]| rs2 | rs1 | funct3 | imm[4:1] | imm[11] | opcode |
926 1 | 6 | 5 | 5 | 3 | 4 | 1 | 7 |
927 reserved || src2 | src1 | BEQ | predicate rs3 || BRANCH |
930 Similarly the C.BEQZ and C.BNEZ instruction format may be retro-fitted,
931 with the interesting side-effect that there is space within what is presently
932 the "immediate offset" field to reinterpret that to add in not only a bit
933 field to distinguish between floating-point compare and integer compare,
934 not only to add in a second source register, but also use some of the bits as
935 a predication target as well.
938 15..13 | 12 ....... 10 | 9...7 | 6 ......... 2 | 1 .. 0 |
939 funct3 | imm | rs10 | imm | op |
941 C.BEQZ | offset[8,4:3] | src | offset[7:6,2:1,5] | C1 |
944 Now uses the CS format:
947 15..13 | 12 . 10 | 9 .. 7 | 6 .. 5 | 4..2 | 1 .. 0 |
948 funct3 | imm | rs10 | imm | | op |
949 3 | 3 | 3 | 2 | 3 | 2 |
950 C.BEQZ | pred rs3 | src1 | I/F B | src2 | C1 |
953 Bit 6 would be decoded as "operation refers to Integer or Float" including
954 interpreting src1 and src2 accordingly as outlined in Table 12.2 of the
955 "C" Standard, version 2.0,
956 whilst Bit 5 would allow the operation to be extended, in combination with
957 funct3 = 110 or 111: a combination of four distinct (predicated) comparison
958 operators. In both floating-point and integer cases those could be
959 EQ/NEQ/LT/LE (with GT and GE being synthesised by inverting src1 and src2).
961 ## Register reordering <a name="register_reordering"></a>
980 May not be an actual CSR: may be generated from Vector Length CSR:
981 single-bit is less burdensome on instruction decode phase.
983 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
984 | - | - | - | - | - | - | - | - |
985 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
987 ### Vector Length CSR
1000 ### Virtual Register Reordering
1002 This example assumes the above Vector Length CSR table
1004 | Reg Num | Bits (0) | Bits (1) | Bits (2) |
1005 | ------- | -------- | -------- | -------- |
1006 | r0 | (32..0) | (32..0) |
1009 | r4 | (32..0) | (32..0) | (32..0) |
1012 ### Bitwidth Virtual Register Reordering
1014 This example goes a little further and illustrates the effect that a
1015 bitwidth CSR has been set on a register. Preconditions:
1018 * CSRintbitwidth[2] = 010 # integer r2 is 16-bit
1019 * CSRintvlength[2] = 3 # integer r2 is a vector of length 3
1020 * vsetl rs1, 5 # set the vector length to 5
1022 This is interpreted as follows:
1024 * Given that the context is RV32, ELEN=32.
1025 * With ELEN=32 and bitwidth=16, the number of SIMD elements is 2
1026 * Therefore the actual vector length is up to *six* elements
1027 * However vsetl sets a length 5 therefore the last "element" is skipped
1029 So when using an operation that uses r2 as a source (or destination)
1030 the operation is carried out as follows:
1032 * 16-bit operation on r2(15..0) - vector element index 0
1033 * 16-bit operation on r2(31..16) - vector element index 1
1034 * 16-bit operation on r3(15..0) - vector element index 2
1035 * 16-bit operation on r3(31..16) - vector element index 3
1036 * 16-bit operation on r4(15..0) - vector element index 4
1037 * 16-bit operation on r4(31..16) **NOT** carried out due to length being 5
1039 Predication has been left out of the above example for simplicity, however
1040 predication is ANDed with the latter stages (vsetl not equal to maximum
1043 Note also that it is entirely an implementor's choice as to whether to have
1044 actual separate ALUs down to the minimum bitwidth, or whether to have something
1045 more akin to traditional SIMD (at any level of subdivision: 8-bit SIMD
1046 operations carried out 32-bits at a time is perfectly acceptable, as is
1047 8-bit SIMD operations carried out 16-bits at a time requiring two ALUs).
1048 Regardless of the internal parallelism choice, *predication must
1049 still be respected*, making Simple-V in effect the "consistent public API".
1051 vew may be one of the following (giving a table "bytestable", used below):
1053 | vew | bitwidth | bytestable |
1054 | --- | -------- | ---------- |
1055 | 000 | default | XLEN/8 |
1061 | 110 | rsvd | rsvd |
1062 | 111 | rsvd | rsvd |
1064 Pseudocode for vector length taking CSR SIMD-bitwidth into account:
1066 vew = CSRbitwidth[rs1]
1068 bytesperreg = (XLEN/8) # or FLEN as appropriate
1070 bytesperreg = bytestable[vew] # 1 2 4 8 16
1071 simdmult = (XLEN/8) / bytesperreg # or FLEN as appropriate
1072 vlen = CSRvectorlen[rs1] * simdmult
1074 To index an element in a register rnum where the vector element index is i:
1076 function regoffs(rnum, i):
1077 regidx = floor(i / simdmult) # integer-div rounded down
1078 byteidx = i % simdmult # integer-remainder
1079 return rnum + regidx, # actual real register
1081 byteidx * 8 + (vew-1), # high
1085 SIMD register file splitting still to consider. For RV64, benefits of doubling
1086 (quadrupling in the case of Half-Precision IEEE754 FP) the apparent
1087 size of the floating point register file to 64 (128 in the case of HP)
1088 seem pretty clear and worth the complexity.
1090 64 virtual 32-bit F.P. registers and given that 32-bit FP operations are
1091 done on 64-bit registers it's not so conceptually difficult. May even
1092 be achieved by *actually* splitting the regfile into 64 virtual 32-bit
1093 registers such that a 64-bit FP scalar operation is dropped into (r0.H
1094 r0.L) tuples. Implementation therefore hidden through register renaming.
1096 Implementations intending to introduce VLIW, OoO and parallelism
1097 (even without Simple-V) would then find that the instructions are
1098 generated quicker (or in a more compact fashion that is less heavy
1099 on caches). Interestingly we observe then that Simple-V is about
1100 "consolidation of instruction generation", where actual parallelism
1101 of underlying hardware is an implementor-choice that could just as
1102 equally be applied *without* Simple-V even being implemented.
1104 ## Analysis of CSR decoding on latency <a name="csr_decoding_analysis"></a>
1106 It could indeed have been logically deduced (or expected), that there
1107 would be additional decode latency in this proposal, because if
1108 overloading the opcodes to have different meanings, there is guaranteed
1109 to be some state, some-where, directly related to registers.
1111 There are several cases:
1113 * All operands vector-length=1 (scalars), all operands
1114 packed-bitwidth="default": instructions are passed through direct as if
1115 Simple-V did not exist. Simple-V is, in effect, completely disabled.
1116 * At least one operand vector-length > 1, all operands
1117 packed-bitwidth="default": any parallel vector ALUs placed on "alert",
1118 virtual parallelism looping may be activated.
1119 * All operands vector-length=1 (scalars), at least one
1120 operand packed-bitwidth != default: degenerate case of SIMD,
1121 implementation-specific complexity here (packed decode before ALUs or
1123 * At least one operand vector-length > 1, at least one operand
1124 packed-bitwidth != default: parallel vector ALUs (if any)
1125 placed on "alert", virtual parallelsim looping may be activated,
1126 implementation-specific SIMD complexity kicks in (packed decode before
1129 Bear in mind that the proposal includes that the decision whether
1130 to parallelise in hardware or whether to virtual-parallelise (to
1131 dramatically simplify compilers and also not to run into the SIMD
1132 instruction proliferation nightmare) *or* a transprent combination
1133 of both, be done on a *per-operand basis*, so that implementors can
1134 specifically choose to create an application-optimised implementation
1135 that they believe (or know) will sell extremely well, without having
1136 "Extra Standards-Mandated Baggage" that would otherwise blow their area
1137 or power budget completely out the window.
1139 Additionally, two possible CSR schemes have been proposed, in order to
1140 greatly reduce CSR space:
1142 * per-register CSRs (vector-length and packed-bitwidth)
1143 * a smaller number of CSRs with the same information but with an *INDEX*
1144 specifying WHICH register in one of three regfiles (vector, fp, int)
1145 the length and bitwidth applies to.
1147 (See "CSR vector-length and CSR SIMD packed-bitwidth" section for details)
1149 In addition, LOAD/STORE has its own associated proposed CSRs that
1150 mirror the STRIDE (but not yet STRIDE-SEGMENT?) functionality of
1153 Also bear in mind that, for reasons of simplicity for implementors,
1154 I was coming round to the idea of permitting implementors to choose
1155 exactly which bitwidths they would like to support in hardware and which
1156 to allow to fall through to software-trap emulation.
1158 So the question boils down to:
1160 * whether either (or both) of those two CSR schemes have significant
1161 latency that could even potentially require an extra pipeline decode stage
1162 * whether there are implementations that can be thought of which do *not*
1163 introduce significant latency
1164 * whether it is possible to explicitly (through quite simply
1165 disabling Simple-V-Ext) or implicitly (detect the case all-vlens=1,
1166 all-simd-bitwidths=default) switch OFF any decoding, perhaps even to
1167 the extreme of skipping an entire pipeline stage (if one is needed)
1168 * whether packed bitwidth and associated regfile splitting is so complex
1169 that it should definitely, definitely be made mandatory that implementors
1170 move regfile splitting into the ALU, and what are the implications of that
1171 * whether even if that *is* made mandatory, is software-trapped
1172 "unsupported bitwidths" still desirable, on the basis that SIMD is such
1173 a complete nightmare that *even* having a software implementation is
1174 better, making Simple-V have more in common with a software API than
1177 Whilst the above may seem to be severe minuses, there are some strong
1180 * Significant reduction of V's opcode space: over 95%.
1181 * Smaller reduction of P's opcode space: around 10%.
1182 * The potential to use Compressed instructions in both Vector and SIMD
1183 due to the overloading of register meaning (implicit vectorisation,
1185 * Not only present but also future extensions automatically gain parallelism.
1186 * Already mentioned but worth emphasising: the simplification to compiler
1187 writers and assembly-level writers of having the same consistent ISA
1188 regardless of whether the internal level of parallelism (number of
1189 parallel ALUs) is only equal to one ("virtual" parallelism), or is
1190 greater than one, should not be underestimated.
1192 ## Reducing Register Bank porting
1194 This looks quite reasonable.
1195 <https://www.princeton.edu/~rblee/ELE572Papers/MultiBankRegFile_ISCA2000.pdf>
1197 The main details are outlined on page 4. They propose a 2-level register
1198 cache hierarchy, note that registers are typically only read once, that
1199 you never write back from upper to lower cache level but always go in a
1200 cycle lower -> upper -> ALU -> lower, and at the top of page 5 propose
1201 a scheme where you look ahead by only 2 instructions to determine which
1202 registers to bring into the cache.
1204 The nice thing about a vector architecture is that you *know* that
1205 *even more* registers are going to be pulled in: Hwacha uses this fact
1206 to optimise L1/L2 cache-line usage (avoid thrashing), strangely enough
1207 by *introducing* deliberate latency into the execution phase.
1209 ## Overflow registers in combination with predication
1211 **TODO**: propose overflow registers be actually one of the integer regs
1212 (flowing to multiple regs).
1214 **TODO**: propose "mask" (predication) registers likewise. combination with
1215 standard RV instructions and overflow registers extremely powerful, see
1218 When integer overflow is stored in an easily-accessible bit (or another
1219 register), parallelisation turns this into a group of bits which can
1220 potentially be interacted with in predication, in interesting and powerful
1221 ways. For example, by taking the integer-overflow result as a predication
1222 field and shifting it by one, a predicated vectorised "add one" can emulate
1223 "carry" on arbitrary (unlimited) length addition.
1225 However despite RVV having made room for floating-point exceptions, neither
1226 RVV nor base RV have taken integer-overflow (carry) into account, which
1227 makes proposing it quite challenging given that the relevant (Base) RV
1228 sections are frozen. Consequently it makes sense to forgo this feature.
1230 ## Context Switch Example <a name="context_switch"></a>
1232 An unusual side-effect of Simple-V mapping onto the standard register files
1233 is that LOAD-multiple and STORE-multiple are accidentally available, as long
1234 as it is acceptable that the register(s) to be loaded/stored are contiguous
1235 (per instruction). An additional accidental benefit is that Compressed LD/ST
1238 To illustrate how this works, here is some example code from FreeRTOS
1239 (GPLv2 licensed, portasm.S):
1241 /* Macro for saving task context */
1242 .macro portSAVE_CONTEXT
1243 .global pxCurrentTCB
1244 /* make room in stack */
1245 addi sp, sp, -REGBYTES * 32
1249 STORE x2, 1 * REGBYTES(sp)
1250 STORE x3, 2 * REGBYTES(sp)
1253 STORE x30, 29 * REGBYTES(sp)
1254 STORE x31, 30 * REGBYTES(sp)
1256 /* Store current stackpointer in task control block (TCB) */
1257 LOAD t0, pxCurrentTCB //pointer
1261 /* Saves current error program counter (EPC) as task program counter */
1264 STORE t0, 31 * REGBYTES(sp)
1267 /* Saves current return adress (RA) as task program counter */
1269 STORE ra, 31 * REGBYTES(sp)
1272 /* Macro for restoring task context */
1273 .macro portRESTORE_CONTEXT
1275 .global pxCurrentTCB
1276 /* Load stack pointer from the current TCB */
1277 LOAD sp, pxCurrentTCB
1280 /* Load task program counter */
1281 LOAD t0, 31 * REGBYTES(sp)
1284 /* Run in machine mode */
1288 /* Restore registers,
1289 Skip global pointer because that does not change */
1291 LOAD x4, 3 * REGBYTES(sp)
1292 LOAD x5, 4 * REGBYTES(sp)
1295 LOAD x30, 29 * REGBYTES(sp)
1296 LOAD x31, 30 * REGBYTES(sp)
1298 addi sp, sp, REGBYTES * 32
1302 The important bits are the Load / Save context, which may be replaced
1303 with firstly setting up the Vectors and secondly using a *single* STORE
1304 (or LOAD) including using C.ST or C.LD, to indicate that the entire
1305 bank of registers is to be loaded/saved:
1307 /* a few things are assumed here: (a) that when switching to
1308 M-Mode an entirely different set of CSRs is used from that
1309 which is used in U-Mode and (b) that the M-Mode x1 and x4
1310 vectors are also not used anywhere else in M-Mode, consequently
1311 only need to be set up just the once.
1314 MVECTORCSRx1 = 31, defaultlen
1315 MVECTORCSRx4 = 28, defaultlen
1318 SETVL x0, x0, 31 /* x0 ignored silently */
1319 STORE x1, 0x0(sp) // x1 marked as 31-long vector of default bitwidth
1321 /* Restore registers,
1322 Skip global pointer because that does not change */
1324 SETVL x0, x0, 28 /* x0 ignored silently */
1325 LOAD x4, 3 * REGBYTES(sp) // x4 marked as 28-long default bitwidth
1327 Note that although it may just be a bug in portasm.S, x2 and x3 appear not
1328 to be being restored. If however this is a bug and they *do* need to be
1329 restored, then the SETVL call may be moved to *outside* the Save / Restore
1330 Context assembly code, into the macroVectorSetup, as long as vectors are
1331 never used anywhere else (i.e. VL is never altered by M-Mode).
1333 In effect the entire bank of repeated LOAD / STORE instructions is replaced
1334 by one single (compressed if it is available) instruction.
1336 ## Virtual Memory page-faults on LOAD/STORE
1339 ### Notes from conversations
1341 > I was going through the C.LOAD / C.STORE section 12.3 of V2.3-Draft
1342 > riscv-isa-manual in order to work out how to re-map RVV onto the standard
1343 > ISA, and came across an interesting comments at the bottom of pages 75
1346 > " A common mechanism used in other ISAs to further reduce save/restore
1347 > code size is load- multiple and store-multiple instructions. "
1349 > Fascinatingly, due to Simple-V proposing to use the *standard* register
1350 > file, both C.LOAD / C.STORE *and* LOAD / STORE would in effect be exactly
1351 > that: load-multiple and store-multiple instructions. Which brings us
1352 > on to this comment:
1354 > "For virtual memory systems, some data accesses could be resident in
1355 > physical memory and
1356 > some could not, which requires a new restart mechanism for partially
1357 > executed instructions."
1359 > Which then of course brings us to the interesting question: how does RVV
1360 > cope with the scenario when, particularly with LD.X (Indexed / indirect
1361 > loads), part-way through the loading a page fault occurs?
1363 > Has this been noted or discussed before?
1365 For applications-class platforms, the RVV exception model is
1366 element-precise (that is, if an exception occurs on element j of a
1367 vector instruction, elements 0..j-1 have completed execution and elements
1368 j+1..vl-1 have not executed).
1370 Certain classes of embedded platforms where exceptions are always fatal
1371 might choose to offer resumable/swappable interrupts but not precise
1375 > Is RVV designed in any way to be re-entrant?
1380 > What would the implications be for instructions that were in a FIFO at
1381 > the time, in out-of-order and VLIW implementations, where partial decode
1384 The usual bag of tricks for maintaining precise exceptions applies to
1385 vector machines as well. Register renaming makes the job easier, and
1386 it's relatively cheaper for vectors, since the control cost is amortized
1387 over longer registers.
1390 > Would it be reasonable at least to say *bypass* (and freeze) the
1391 > instruction FIFO (drop down to a single-issue execution model temporarily)
1392 > for the purposes of executing the instructions in the interrupt (whilst
1393 > setting up the VM page), then re-continue the instruction with all
1396 This approach has been done successfully, but it's desirable to be
1397 able to swap out the vector unit state to support context switches on
1398 exceptions that result in long-latency I/O.
1401 > Or would it be better to switch to an entirely separate secondary
1402 > hyperthread context?
1404 > Does anyone have any ideas or know if there is any academic literature
1405 > on solutions to this problem?
1407 The Vector VAX offered imprecise but restartable and swappable exceptions:
1408 http://mprc.pku.edu.cn/~liuxianhua/chn/corpus/Notes/articles/isca/1990/VAX%20vector%20architecture.pdf
1410 Sec. 4.6 of Krste's dissertation assesses some of
1411 the tradeoffs and references a bunch of related work:
1412 http://people.eecs.berkeley.edu/~krste/thesis.pdf
1417 Started reading section 4.6 of Krste's thesis, noted the "IEE85 F.P
1418 exceptions" and thought, "hmmm that could go into a CSR, must re-read
1419 the section on FP state CSRs in RVV 0.4-Draft again" then i suddenly
1420 thought, "ah ha! what if the memory exceptions were, instead of having
1421 an immediate exception thrown, were simply stored in a type of predication
1422 bit-field with a flag "error this element failed"?
1424 Then, *after* the vector load (or store, or even operation) was
1425 performed, you could *then* raise an exception, at which point it
1426 would be possible (yes in software... I know....) to go "hmmm, these
1427 indexed operations didn't work, let's get them into memory by triggering
1428 page-loads", then *re-run the entire instruction* but this time with a
1429 "memory-predication CSR" that stops the already-performed operations
1430 (whether they be loads, stores or an arithmetic / FP operation) from
1431 being carried out a second time.
1433 This theoretically could end up being done multiple times in an SMP
1434 environment, and also for LD.X there would be the remote outside annoying
1435 possibility that the indexed memory address could end up being modified.
1437 The advantage would be that the order of execution need not be
1438 sequential, which potentially could have some big advantages.
1439 Am still thinking through the implications as any dependent operations
1440 (particularly ones already decoded and moved into the execution FIFO)
1441 would still be there (and stalled). hmmm.
1445 > > # assume internal parallelism of 8 and MAXVECTORLEN of 8
1450 > > x3[1]: exception
1456 > > what happens to result elements 2-7? those may be *big* results
1458 > > or in the RVV-Extended may be arbitrary bit-widths far greater.
1464 discussion then led to the question of OoO architectures
1466 > The costs of the imprecise-exception model are greater than the benefit.
1467 > Software doesn't want to cope with it. It's hard to debug. You can't
1468 > migrate state between different microarchitectures--unless you force all
1469 > implementations to support the same imprecise-exception model, which would
1470 > greatly limit implementation flexibility. (Less important, but still
1471 > relevant, is that the imprecise model increases the size of the context
1472 > structure, as the microarchitectural guts have to be spilled to memory.)
1474 ## Zero/Non-zero Predication
1476 >> > it just occurred to me that there's another reason why the data
1477 >> > should be left instead of zeroed. if the standard register file is
1478 >> > used, such that vectorised operations are translated to mean "please
1479 >> > insert multiple register-contiguous operations into the instruction
1480 >> > FIFO" and predication is used to *skip* some of those, then if the
1481 >> > next "vector" operation uses the (standard) registers that were masked
1482 >> > *out* of the previous operation it may proceed without blocking.
1484 >> > if however zeroing is made mandatory then that optimisation becomes
1485 >> > flat-out impossible to deploy.
1487 >> > whilst i haven't fully thought through the full implications, i
1488 >> > suspect RVV might also be able to benefit by being able to fit more
1489 >> > overlapping operations into the available SRAM by doing something
1493 > Luke, this is called density time masking. It doesn’t apply to only your
1494 > model with the “standard register file” is used. it applies to any
1495 > architecture that attempts to speed up by skipping computation and writeback
1496 > of masked elements.
1498 > That said, the writing of zeros need not be explicit. It is possible to add
1499 > a “zero bit” per element that, when set, forces a zero to be read from the
1500 > vector (although the underlying storage may have old data). In this case,
1501 > there may be a way to implement DTM as well.
1504 ## Implementation detail for scalar-only op detection <a name="scalar_detection"></a>
1506 Note 1: this idea is a pipeline-bypass concept, which may *or may not* be
1509 Note 2: this is just one possible implementation. Another implementation
1510 may choose to treat *all* operations as vectorised (including treating
1511 scalars as vectors of length 1), choosing to add an extra pipeline stage
1512 dedicated to *all* instructions.
1514 This section *specifically* covers the implementor's freedom to choose
1515 that they wish to minimise disruption to an existing design by detecting
1516 "scalar-only operations", bypassing the vectorisation phase (which may
1517 or may not require an additional pipeline stage)
1519 [[scalardetect.png]]
1521 >> For scalar ops an implementation may choose to compare 2-3 bits through an
1522 >> AND gate: are src & dest scalar? Yep, ok send straight to ALU (or instr
1525 > Those bits cannot be known until after the registers are decoded from the
1526 > instruction and a lookup in the "vector length table" has completed.
1527 > Considering that one of the reasons RISC-V keeps registers in invariant
1528 > positions across all instructions is to simplify register decoding, I expect
1529 > that inserting an SRAM read would lengthen the critical path in most
1534 > briefly: the trick i mentioned about ANDing bits together to check if
1535 > an op was fully-scalar or not was to be read out of a single 32-bit
1536 > 3R1W SRAM (64-bit if FPU exists). the 32/64-bit SRAM contains 1 bit per
1537 > register indicating "is register vectorised yes no". 3R because you need
1538 > to check src1, src2 and dest simultaneously. the entries are *generated*
1539 > from the CSRs and are an optimisation that on slower embedded systems
1540 > would likely not be needed.
1542 > is there anything unreasonable that anyone can foresee about that?
1543 > what are the down-sides?
1545 ## C.MV predicated src, predicated dest
1547 > Can this be usefully defined in such a way that it is
1548 > equivalent to vector gather-scatter on each source, followed by a
1549 > non-predicated vector-compare, followed by vector gather-scatter on the
1552 ## element width conversion: restrict or remove?
1554 summary: don't restrict / remove. it's fine.
1556 > > it has virtually no cost/overhead as long as you specify
1557 > > that inputs can only upconvert, and operations are always done at the
1558 > > largest size, and downconversion only happens at the output.
1560 > okaaay. so that's a really good piece of implementation advice.
1561 > algorithms do require data size conversion, so at some point you need to
1562 > introduce the feature of upconverting and downconverting.
1564 > > for int and uint, this is dead simple and fits well within the RVV pipeline
1565 > > without any critical path, pipeline depth, or area implications.
1567 <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/g3feFnAoKIM>
1569 ## Under review / discussion: remove CSR vector length, use VSETVL <a name="vsetvl"></a>
1571 **DECISION: 11jun2018 - CSR vector length removed, VSETVL determines
1572 length on all regs**. This section kept for historical reasons.
1574 So the issue is as follows:
1576 * CSRs are used to set the "span" of a vector (how many of the standard
1577 register file to contiguously use)
1578 * VSETVL in RVV works as follows: it sets the vector length (copy of which
1579 is placed in a dest register), and if the "required" length is longer
1580 than the *available* length, the dest reg is set to the MIN of those
1582 * **HOWEVER**... in SV, *EVERY* vector register has its own separate
1583 length and thus there is no way (at the time that VSETVL is called) to
1584 know what to set the vector length *to*.
1585 * At first glance it seems that it would be perfectly fine to just limit
1586 the vector operation to the length specified in the destination
1587 register's CSR, at the time that each instruction is issued...
1588 except that that cannot possibly be guaranteed to match
1589 with the value *already loaded into the target register from VSETVL*.
1591 Therefore a different approach is needed.
1593 Possible options include:
1595 * Removing the CSR "Vector Length" and always using the value from
1596 VSETVL. "VSETVL destreg, counterreg, #lenimmed" will set VL *and*
1597 destreg equal to MIN(counterreg, lenimmed), with register-based
1598 variant "VSETVL destreg, counterreg, lenreg" doing the same.
1599 * Keeping the CSR "Vector Length" and having the lenreg version have
1600 a "twist": "if lengreg is vectorised, read the length from the CSR"
1603 The first option (of the ones brainstormed so far) is a lot simpler.
1604 It does however mean that the length set in VSETVL will apply across-the-board
1605 to all src1, src2 and dest vectorised registers until it is otherwise changed
1606 (by another VSETVL call). This is probably desirable behaviour.
1608 ## Implementation Paradigms <a name="implementation_paradigms"></a>
1610 TODO: assess various implementation paradigms. These are listed roughly
1611 in order of simplicity (minimum compliance, for ultra-light-weight
1612 embedded systems or to reduce design complexity and the burden of
1613 design implementation and compliance, in non-critical areas), right the
1614 way to high-performance systems.
1616 * Full (or partial) software-emulated (via traps): full support for CSRs
1617 required, however when a register is used that is detected (in hardware)
1618 to be vectorised, an exception is thrown.
1619 * Single-issue In-order, reduced pipeline depth (traditional SIMD / DSP)
1620 * In-order 5+ stage pipelines with instruction FIFOs and mild register-renaming
1621 * Out-of-order with instruction FIFOs and aggressive register-renaming
1624 Also to be taken into consideration:
1626 * "Virtual" vectorisation: single-issue loop, no internal ALU parallelism
1627 * Comphrensive vectorisation: FIFOs and internal parallelism
1628 * Hybrid Parallelism
1630 ### Full or partial software-emulation
1632 The absolute, absolute minimal implementation is to provide the full
1633 set of CSRs and detection logic for when any of the source or destination
1634 registers are vectorised. On detection, a trap is thrown, whether it's
1635 a branch, LOAD, STORE, or an arithmetic operation.
1637 Implementors are entirely free to choose whether to allow absolutely every
1638 single operation to be software-emulated, or whether to provide some emulation
1639 and some hardware support. In particular, for an RV32E implementation
1640 where fast context-switching is a requirement (see "Context Switch Example"),
1641 it makes no sense to allow Vectorised-LOAD/STORE to be implemented as an
1642 exception, as every context-switch will result in double-traps.
1646 > For great floating point DSPs check TI’s C3x, C4X, and C6xx DSPs
1648 Idea: basic simple butterfly swap on a few element indices, primarily targetted
1649 at SIMD / DSP. High-byte low-byte swapping, high-word low-word swapping,
1650 perhaps allow reindexing of permutations up to 4 elements? 8? Reason:
1651 such operations are less costly than a full indexed-shuffle, which requires
1652 a separate instruction cycle.
1654 Predication "all zeros" needs to be "leave alone". Detection of
1655 ADD r1, rs1, rs0 cases result in nop on predication index 0, whereas
1656 ADD r0, rs1, rs2 is actually a desirable copy from r2 into r0.
1657 Destruction of destination indices requires a copy of the entire vector
1658 in advance to avoid.
1660 TBD: floating-point compare and other exception handling
1666 Please don't try to use the L1 itself.
1668 Use the Load and Store buffers which capture instruction state prior
1669 to being accessed in the L1 (and prior to data arriving in the case of
1672 Also, use the L1 Miss buffers as these already HAVE to be snooped by
1673 coherence traffic. These are used to monitor that all participating
1674 cache lines remain interference free, and amalgamate same into a CPU
1675 signal accessible ia branch or predicate.
1677 The Load buffers manage inbound traffic
1678 The Store buffers manage outbound traffic.
1680 Done properly, the participating cache lines can exceed the associativity
1681 of the L1 cache without architectural harm (may incur additional latency).
1683 <https://groups.google.com/d/msg/comp.arch/QVl3c9vVDj0/ol_232-pAQAJ>
1685 > > > so, let's say instead of another LR *cancelling* the load
1686 > > > reservation, the SMP core / hardware thread *blocks* for
1687 > > > up to 63 further instructions, waiting for the reservation
1690 > > Can you explain what you mean by this paragraph?
1692 > best put in sequential events, probably.
1694 > <core1> LR <-- 64-instruction countdown starts here
1697 > <core2> LR same address <--- notes that core1 is on 61,
1698 > so pauses for **UP TO** 61 cycles
1700 > <core1> SC <- core1 didn't reach zero, therefore valid, therefore
1701 > core2 is now **UNBLOCKED**, is granted the
1702 > load-reservation (and begins its **own** 64-cycle
1703 > LR instruction countdown)
1708 > <core2> SC <- also valid
1710 Looks to me that you could effect the same functionality by simply
1711 holding onto the cache line in core 1 preventing core 2 from
1712 <architecturally> getting past the LR.
1714 On the other hand, the freeze is similar to how the MP CRAYs did
1719 * SIMD considered harmful <https://www.sigarch.org/simd-instructions-considered-harmful/>
1720 * Link to first proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/GuukrSjgBH8>
1721 * Recommendation by Jacob Bachmeyer to make zero-overhead loop an
1722 "implicit program-counter" <https://groups.google.com/a/groups.riscv.org/d/msg/isa-dev/vYVi95gF2Mo/SHz6a4_lAgAJ>
1723 * Re-continuing P-Extension proposal <https://groups.google.com/a/groups.riscv.org/forum/#!msg/isa-dev/IkLkQn3HvXQ/SEMyC9IlAgAJ>
1724 * First Draft P-SIMD (DSP) proposal <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/vYVi95gF2Mo>
1725 * B-Extension discussion <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/zi_7B15kj6s>
1726 * Broadcom VideoCore-IV <https://docs.broadcom.com/docs/12358545>
1727 Figure 2 P17 and Section 3 on P16.
1728 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-262.html>
1729 * Hwacha <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-263.html>
1730 * Vector Workshop <http://riscv.org/wp-content/uploads/2015/06/riscv-vector-workshop-june2015.pdf>
1731 * Predication <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/XoP4BfYSLXA>
1732 * Branch Divergence <https://jbush001.github.io/2014/12/07/branch-divergence-in-parallel-kernels.html>
1733 * Life of Triangles (3D) <https://jbush001.github.io/2016/02/27/life-of-triangle.html>
1734 * Videocore-IV <https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-3d-Graphics-Pipeline>
1735 * Discussion proposing CSRs that change ISA definition
1736 <https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/InzQ1wr_3Ak>
1737 * Zero-overhead loops <https://pdfs.semanticscholar.org/dbaa/66985cc730d4b44d79f519e96ec9c43ab5b7.pdf>
1738 * Multi-ported VLIW Register File Implementation <https://ce-publications.et.tudelft.nl/publications/1517_multiple_contexts_in_a_multiported_vliw_register_file_impl.pdf>
1739 * Fast context save/restore proposal <https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/57F823FA.6030701%40gmail.com>
1740 * Register File Bank Cacheing <https://www.princeton.edu/~rblee/ELE572Papers/MultiBankRegFile_ISCA2000.pdf>
1741 * Expired Patent on Vector Virtual Memory solutions
1742 <https://patentimages.storage.googleapis.com/fc/f6/e2/2cbee92fcd8743/US5895501.pdf>
1743 * Discussion on RVV "re-entrant" capabilities allowing operations to be
1744 restarted if an exception occurs (VM page-table miss)
1745 <https://groups.google.com/a/groups.riscv.org/d/msg/isa-dev/IuNFitTw9fM/CCKBUlzsAAAJ>
1746 * Dot Product Vector <https://people.eecs.berkeley.edu/~biancolin/papers/arith17.pdf>
1747 * RVV slides 2017 <https://content.riscv.org/wp-content/uploads/2017/12/Wed-1330-RISCVRogerEspasaVEXT-v4.pdf>
1748 * Wavefront skipping using BRAMS <http://www.ece.ubc.ca/~lemieux/publications/severance-fpga2015.pdf>
1749 * Streaming Pipelines <http://www.ece.ubc.ca/~lemieux/publications/severance-fpga2014.pdf>
1750 * Barcelona SIMD Presentation <https://content.riscv.org/wp-content/uploads/2018/05/09.05.2018-9.15-9.30am-RISCV201805-Andes-proposed-P-extension.pdf>
1751 * <http://www.ece.ubc.ca/~lemieux/publications/severance-fpga2015.pdf>
1752 * Full Description (last page) of RVV instructions
1753 <https://inst.eecs.berkeley.edu/~cs152/sp18/handouts/lab4-1.0.pdf>
1754 * PULP Low-energy Cluster Vector Processor
1755 <http://iis-projects.ee.ethz.ch/index.php/Low-Energy_Cluster-Coupled_Vector_Coprocessor_for_Special-Purpose_PULP_Acceleration>