1 \documentclass[slidestop
]{beamer
}
2 \usepackage{beamerthemesplit
}
6 \title{Simple-V RISC-V Extension for Vectorisation and SIMD
}
7 \author{Luke Kenneth Casson Leighton
}
14 \huge{Simple-V RISC-V Extension for Vectors and SIMD
}\\
16 \Large{Flexible Vectorisation
}\\
17 \Large{(aka not so Simple-V?)
}\\
19 \Large{[proposed for
] Chennai
9th RISC-V Workshop
}\\
26 \frame{\frametitle{Credits and Acknowledgements
}
29 \item The Designers of RISC-V
\vspace{15pt
}
30 \item The RVV Working Group and contributors
\vspace{15pt
}
31 \item Allen Baum, Jacob Bachmeyer, Xan Phung, Chuanhua Chang,\\
32 Guy Lemurieux, Jonathan Neuschafer, Roger Bruisse,
33 and others
\vspace{15pt
}
34 \item ISA-Dev Group Members
\vspace{10pt
}
39 \frame{\frametitle{Quick refresher on SIMD
}
42 \item SIMD very easy to implement (and very seductive)
\vspace{10pt
}
43 \item Parallelism is in the ALU
\vspace{10pt
}
44 \item Zero-to-Negligeable impact for rest of core
\vspace{10pt
}
46 Where SIMD Goes Wrong:
\vspace{10pt
}
48 \item See "SIMD instructions considered harmful"
49 https://www.sigarch.org/simd-instructions-considered-harmful
50 \item Corner-cases alone are extremely complex.\\
51 Hardware is easy, but software is hell.
52 \item O($N^
{6}$) ISA opcode proliferation!\\
53 opcode, elwidth, veclen, src1-src2-dest hi/lo
57 \frame{\frametitle{Quick refresher on RVV
}
60 \item Extremely powerful (extensible to
256 registers)
\vspace{10pt
}
61 \item Supports polymorphism, several datatypes (inc. FP16)
\vspace{10pt
}
62 \item Requires a separate Register File (
32 w/ext to
256)
\vspace{10pt
}
63 \item Implemented as a separate pipeline (no impact on scalar)
\vspace{10pt
}
65 However...
\vspace{10pt
}
67 \item 98 percent opcode duplication with rest of RV (CLIP)
68 \item Extending RVV requires customisation not just of h/w:\\
69 gcc and s/w also need customisation (and maintenance)
74 \frame{\frametitle{The Simon Sinek lowdown (Why, How, What)
}
78 Implementors need flexibility in vectorisation to optimise for
79 area or performance depending on the scope:
80 embedded DSP, Mobile GPU's, Server CPU's and more.
\vspace{4pt
}\\
81 Compilers also need flexibility in vectorisation to optimise for cost
82 of pipeline setup, amount of state to context switch
83 and software portability
\vspace{4pt
}
85 By implicitly marking INT/FP regs as "Vectorised",\\
86 SV expresses how existing instructions should act
87 on
[contiguous
] blocks of registers, in parallel.
\vspace{4pt
}
89 Simple-V is an "API" that implicitly extends
90 existing (scalar) instructions with explicit parallelisation.
95 \frame{\frametitle{What's the value of SV? Why adopt it even in non-V?
}
98 \item memcpy becomes much smaller (higher bang-per-buck)
\vspace{10pt
}
99 \item context-switch (LOAD/STORE multiple):
1-
2 instructions
\vspace{10pt
}
100 \item Compressed instrs further reduces I-cache (etc.)
\vspace{10pt
}
101 \item greatly-reduced I-cache load (and less reads)
\vspace{10pt
}
105 \item It's not just about Vectors: it's about instruction effectiveness
106 \item Anything implementor is not interested in HW-optimising,\\
107 let it fall through to exceptions (implement as a trap).
112 \frame{\frametitle{How does Simple-V relate to RVV?
}
115 \item RVV very heavy-duty (excellent for supercomputing)
\vspace{10pt
}
116 \item Simple-V abstracts parallelism (based on best of RVV)
\vspace{10pt
}
117 \item Graded levels: hardware, hybrid or traps (fit impl. need)
\vspace{10pt
}
118 \item Even Compressed instructions become vectorised
\vspace{10pt
}
120 What Simple-V is not:
\vspace{10pt
}
122 \item A full supercomputer-level Vector Proposal
123 \item A replacement for RVV (SV is designed to be over-ridden\\
124 by - or augmented to become, or just be replaced by - RVV)
129 \frame{\frametitle{How is Parallelism abstracted in Simple-V?
}
132 \item Register "typing" turns any op into an implicit Vector op
\vspace{10pt
}
133 \item Primarily at the Instruction issue phase (except SIMD)\\
134 Note: it's ok to pass predication through to ALU (like SIMD)
135 \item Standard (and future, and custom) opcodes now parallel
\vspace{10pt
}
139 \item All LOAD/STORE (inc. Compressed, Int/FP versions)
140 \item All ALU ops (soft / hybrid / full HW, on per-op basis)
141 \item All branches become predication targets (C.FNE added)
142 \item C.MV of particular interest (s/v, v/v, v/s)
147 \frame{\frametitle{Implementation Options
}
150 \item Absolute minimum: Exceptions (if CSRs indicate "V", trap)
151 \item Hardware loop, single-instruction issue\\
152 (Do / Don't send through predication to ALU)
153 \item Hardware loop, parallel (multi-instruction) issue\\
154 (Do / Don't send through predication to ALU)
155 \item Hardware loop, full parallel ALU (not recommended)
159 \item 4 (or more?) options above may be deployed on per-op basis
160 \item SIMD always sends predication bits through to ALU
161 \item Minimum MVL MUST be sufficient to cover regfile LD/ST
162 \item Instr. FIFO may repeatedly split off N scalar ops at a time
165 % Instr. FIFO may need its own slide. Basically, the vectorised op
166 % gets pushed into the FIFO, where it is then "processed". Processing
167 % will remove the first set of ops from its vector numbering (taking
168 % predication into account) and shoving them **BACK** into the FIFO,
169 % but MODIFYING the remaining "vectorised" op, subtracting the now
170 % scalar ops from it.
172 \frame{\frametitle{How are SIMD Instructions Vectorised?
}
175 \item SIMD ALU(s) primarily unchanged
\vspace{10pt
}
176 \item Predication is added to each SIMD element (NO ZEROING!)
\vspace{10pt
}
177 \item End of Vector enables predication (NO ZEROING!)
\vspace{10pt
}
179 Considerations:
\vspace{10pt
}
181 \item Many SIMD ALUs possible (parallel execution)
\vspace{10pt
}
182 \item Very long SIMD ALUs could waste die area (short vectors)
\vspace{10pt
}
183 \item Implementor free to choose (API remains the same)
\vspace{10pt
}
186 % With multiple SIMD ALUs at for example 32-bit wide they can be used
187 % to either issue 64-bit or 128-bit or 256-bit wide SIMD operations
188 % or they can be used to cover several operations on totally different
189 % vectors / registers.
191 \frame{\frametitle{What's the deal / juice / score?
}
194 \item Standard Register File(s) overloaded with CSR "vector span"\\
195 (see pseudocode slides for examples)
196 \item Element width and type concepts remain same as RVV\\
197 (CSRs are used to "interpret" elements in registers)
198 \item CSRs are key-value tables (overlaps allowed)
\vspace{10pt
}
200 Key differences from RVV:
\vspace{10pt
}
202 \item Predication in INT regs as a BIT field (max VL=XLEN)
203 \item Minimum VL must be Num Regs -
1 (all regs single LD/ST)
204 \item SV may condense sparse Vecs: RVV lets ALU do predication
205 \item NO ZEROING: non-predicated elements are skipped
210 \begin{frame
}[fragile
]
211 \frametitle{ADD pseudocode (or trap, or actual hardware loop)
}
214 function op_add(rd, rs1, rs2, predr) # add not VADD!
215 Â int i, id=
0, irs1=
0, irs2=
0;
216 Â for (i=
0; i < MIN(VL, vectorlen
[rd
]); i++)
217 Â if (ireg
[predr
] &
1<<i) # predication uses intregs
218 Â Â ireg
[rd+id
] <= ireg
[rs1+irs1
] + ireg
[rs2+irs2
];
219 if (reg_is_vectorised
[rd
])Â \
{ id +=
1; \
}
220 if (reg_is_vectorised
[rs1
])Â \
{ irs1 +=
1; \
}
221 if (reg_is_vectorised
[rs2
])Â \
{ irs2 +=
1; \
}
225 \item SIMD slightly more complex (case above is elwidth = default)
226 \item Scalar-scalar and scalar-vector and vector-vector now all in one
227 \item OoO may choose to push ADDs into instr. queue (v. busy!)
231 % yes it really *is* ADD not VADD. that's the entire point of
232 % this proposal, that *standard* operations are overloaded to
233 % become vectorised-on-demand
236 \begin{frame
}[fragile
]
237 \frametitle{Predication-Branch (or trap, or actual hardware loop)
}
240 s1 = reg_is_vectorised(src1);
241 s2 = reg_is_vectorised(src2);
242 if (!s2 && !s1) goto branch;
243 for (int i =
0; i < VL; ++i)
244 preg
[rs3
] |=
1 << cmp(s1 ? reg
[src1+i
] : reg
[src1
],
245 s2 ? reg
[src2+i
] : reg
[src2
]);
249 \item SIMD slightly more complex (case above is elwidth = default)
250 \item If s1 and s2 both scalars, Standard branch occurs
251 \item Predication stored in integer regfile as a bitfield
252 \item Scalar-vector and vector-vector supported
256 \begin{frame
}[fragile
]
257 \frametitle{VLD/VLD.S/VLD.X (or trap, or actual hardware loop)
}
260 if (unit-strided) stride = elsize;
261 else stride = areg
[as2
]; // constant-strided
262 for (int i =
0; i < VL; ++i)
263 if (preg_enabled
[rd
] && (
[!
]preg
[rd
] &
1<<i))
264 for (int j =
0; j < seglen+
1; j++)
265 if (vectorised
[rs2
]) offs = vreg
[rs2
][i
]
266 else offs = i*(seglen+
1)*stride;
267 vreg
[rd+j
][i
] = mem
[sreg
[base
] + offs + j*stride
]
271 \item Again: SIMD slightly more complex
272 \item rs2 vectorised taken to implicitly indicate VLD.X
277 \frame{\frametitle{Why are overlaps allowed in Regfiles?
}
280 \item Same register(s) can have multiple "interpretations"
\vspace{6pt
}
281 \item xBitManip plus SIMD plus xBitManip = Hi/Lo bitops
\vspace{6pt
}
282 \item (
32-bit GREV plus
4x8-bit SIMD plus
32-bit GREV)
\vspace{6pt
}
283 \item RGB
565 (video): BEXTW plus
4x8-bit SIMD plus BDEPW
\vspace{6pt
}
284 \item Same register(s) can be offset (no need for VSLIDE)
\vspace{6pt
}
288 \item xBitManip reduces O($N^
{6}$) SIMD down to O($N^
{3}$)
289 \item Hi-Performance: Macro-op fusion (more pipeline stages?)
294 \frame{\frametitle{Why no Zeroing (place zeros in non-predicated elements)?
}
297 \item Zeroing is an implementation optimisation favouring OoO
\vspace{8pt
}
298 \item Simple implementations may skip non-predicated operations
\vspace{8pt
}
299 \item Simple implementations explicitly have to destroy data
\vspace{8pt
}
300 \item Complex implementations may use reg-renames to save power\\
301 Zeroing on predication chains makes optimisation harder
303 Considerations:
\vspace{10pt
}
305 \item Complex not really impacted, Simple impacted a LOT
306 \item Overlapping "Vectors" may issue overlapping ops
307 \item Please don't use Vectors for "security" (use Sec-Ext)
310 % with overlapping "vectors" - bearing in mind that "vectors" are
311 % just a remap onto the standard register file, if the top bits of
312 % predication are zero, and there happens to be a second vector
313 % that uses some of the same register file that happens to be
314 % predicated out, the second vector op may be issued *at the same time*
315 % if there are available parallel ALUs to do so.
318 \frame{\frametitle{Predication key-value CSR store
}
321 \item key is int regfile number or FP regfile number (
1 bit)
\vspace{10pt
}
322 \item register to be predicated if referred to (
5 bits, key)
\vspace{10pt
}
323 \item register to store actual predication in (
5 bits, value)
\vspace{10pt
}
324 \item predication is inverted (
1 bit)
\vspace{10pt
}
328 \item Table should be expanded out for high-speed implementations
329 \item Multiple "keys" (and values) theoretically permitted
330 \item RVV rules about deleting higher-indexed CSRs followed
335 \frame{\frametitle{Register key-value CSR store
}
338 \item key is int regfile number or FP regfile number (
1 bit)
\vspace{10pt
}
339 \item register to be predicated if referred to (
5 bits, key)
\vspace{10pt
}
340 \item register to store actual predication in (
5 bits, value)
\vspace{10pt
}
341 \item TODO
\vspace{10pt
}
345 \item Table should be expanded out for high-speed implementations
346 \item Multiple "keys" (and values) theoretically permitted
347 \item RVV rules about deleting higher-indexed CSRs followed
352 \frame{\frametitle{C.MV extremely flexible!
}
355 \item scalar-to-vector (w/no pred): VSPLAT
356 \item scalar-to-vector (w/dest-pred): Sparse VSPLAT
357 \item scalar-to-vector (w/single dest-pred): VINSERT
358 \item vector-to-scalar (w/src-pred): VEXTRACT
359 \item vector-to-vector (w/no pred): Vector Copy
360 \item vector-to-vector (w/src xor dest pred): Sparse Vector Copy
361 \item vector-to-vector (w/src and dest pred): Vector Gather/Scatter
366 \item Really powerful!
367 \item Any other options?
372 \frame{\frametitle{Opcodes, compared to RVV
}
375 \item All integer and FP opcodes all removed (no CLIP!)
\vspace{8pt
}
376 \item VMPOP, VFIRST etc. all removed (use xBitManip)
\vspace{8pt
}
377 \item VSLIDE removed (use regfile overlaps)
\vspace{8pt
}
378 \item C.MV covers VEXTRACT VINSERT and VSPLAT (and more)
\vspace{8pt
}
379 \item VSETVL, VGETVL, VSELECT stay
\vspace{8pt
}
380 \item Issue: VCLIP is not in RV* (add with custom ext?)
\vspace{8pt
}
381 \item Vector (or scalar-vector) use C.MV (MV is a pseudo-op)
\vspace{8pt
}
382 \item VMERGE: twin predicated C.MVs (one inverted. macro-op'd)
\vspace{8pt
}
387 \frame{\frametitle{Under consideration
}
390 \item Is C.FNE actually needed? Should it be added if it is?
391 \item FP Exceptions: should linear semantics be forced?\\
392 (requires throwing away perfectly good data)
393 \item Is detection of all-scalar ops ok (without slowing pipeline)?
394 \item Can VSELECT be removed? (it's really complex)
395 \item Can CLIP be done as a CSR (mode, like elwidth)
396 \item SIMD saturation (etc.) also set as a mode?
397 \item C.MV src predication no different from dest predication\\
398 What to do? Make one have different meaning?
399 \item 8/
16-bit ops is it worthwhile adding a "start offset"? \\
400 (a bit like misaligned addressing... for registers)\\
401 or just use predication to skip start?
406 \frame{\frametitle{Is this OK (low latency)? Detect scalar-ops (only)
}
408 \includegraphics[height=
2.5in
]{scalardetect.png
}\\
409 {\bf \red Detect when all registers are scalar for a given op
}
414 \frame{\frametitle{TODO (break into separate slides)
}
417 \item Then explain why this proposal is a good way to \\
418 abstract parallelism\\
419 (hopefully also explaining how \\
420 a good compiler can make clever use of this increase parallelism\\
421 Then explain how this can be implemented (at instruction\\
422 issue time???) with\\
423 implementation options, and what these "cost".\\
424 Finally give examples that show simple usage that compares\\
433 \frame{\frametitle{Summary
}
436 \item Designed for flexibility (graded levels of complexity)
\vspace{6pt
}
437 \item Huge range of implementor freedom
\vspace{6pt
}
438 \item Fits RISC-V ethos: achieve more with less
\vspace{6pt
}
439 \item Reduces SIMD ISA proliferation by
3-
4 orders of magnitude \\
440 (without SIMD downsides or sacrificing speed trade-off)
\vspace{6pt
}
441 \item Covers
98\% of RVV, allows RVV to fit "on top"
\vspace{6pt
}
442 \item Not designed for supercomputing (that's RVV), designed for
443 in between: DSPs, RV32E, Embedded
3D GPUs etc.
\vspace{6pt
}
444 \item Not specifically designed for Vectorisation: designed to\\
445 reduce code size (increase efficiency, just
446 like Compressed)
\vspace{6pt
}
451 \frame{\frametitle{slide
}
456 Considerations:
\vspace{10pt
}
465 {\Huge \red The end
\vspace{20pt
}\\