add Georgia Tech section
[crowdsupply.git] / updates / 023_2020mar26_decoder_emulator_started.mdwn
1 So many things happened since the last update they actually need to go
2 in the main update, even in summary form. One big thing:
3 [Raptor CS](
4 sponsored us with remote access to a Monster spec'd TALOS II Workstation!
6 # Introduction
8 Here's the summary (if it can be called a summary):
10 * [An announcement](
11 that we got the funding (which is open to anyone - hint, hint) resulted in
12 at least three people reaching out to join the team. "We don't need
13 permission to own our own hardware" got a *really* positive reaction.
14 * New team member, Jock (hello Jock!) starts on the coriolis2 layout,
15 after Jean-Paul from helped to dramatically improve how coriolis2
16 can be used. This resulted in a
17 [tutorial]( and a
18 [huge bug report discussion](
19 * Work has started on the
20 [POWER ISA decoder](,
21 verified through
22 [calling GNU AS](;a=blob;f=src/soc/decoder/test/;h=9238d3878d964907c5569a3468d6895effb7dc02;hb=56d145e42ac75626423915af22d1493f1e7bb143) (yes, really!)
23 and on a mini-simulator
24 [calling QEMU](;a=blob;f=src/soc/simulator/;h=9eb103bae227e00a2a1d2ec4f43d7e39e4f44960;hb=56d145e42ac75626423915af22d1493f1e7bb143)
25 for verification.
26 * Jacob's simple-soft-float library growing
27 [Power FP compatibility](
28 and python bindings.
29 * Kazan, the Vulkan driver Jacob is writing, is getting
30 a [new shader compiler IR](
31 * A Conference call with OpenPOWER Foundation Director, Hugh, and Timothy
32 Pearson from RaptorCS has been established every two weeks.
33 * The OpenPOWER Foundation is also running some open
34 ["Virtual Coffee"](
35 weekly round-table calls for anyone interested, generally, in OpenPOWER
36 development.
37 * Tim sponsors our team with access to a Monster Talos II system with a
38 whopping 128 GB RAM. htop lists a staggering 72 cores (18 real
39 with 4-way hyperthreading).
40 * [Epic MegaGrants](
41 reached out (hello!) to say they're still considering our
42 request.
43 * A marathon 3-hour session with [NLNet]( resulted
44 in the completion of the
45 [Milestone tasks list(s)](
46 and a
47 [boat-load](
48 of bug reports to the list.
49 * Immanuel Yehowshua is participating in the Georgia Tech
50 [Create-X]( Programme, and is establishing
51 a Public Benefit Corporation in Atlanta, as an ethical vehicle for VC
52 Funding.
53 * A [Load/Store Buffer](
54 design and
55 [further discussion](
56 including on
57 [comp.arch](!topic/comp.arch/cbGAlcCjiZE)
58 inspired additional writeup
59 on the
60 [6600 scoreboard](
61 page.
62 * [Public-Inbox]( was
63 installed successfully on the server, which is in the process of
64 moving to a [new domain name](
65 [Libre-SOC](
66 * Build Servers have been set up with
67 [automated testing](
68 being established
70 Well dang, as you can see, suddenly it just went ballistic. There's
71 almost certainly things left off the list. For such a small team there's
72 a heck of a lot going on. We have an awful lot to do, in a short amount
73 of time: the 180nm tape-out is in October 2020 - only 7 months away.
75 With this update we're doing something slightly different: a request
76 has gone out [to the other team members](
77 to say a little bit about what each of them is doing. This also helps me
78 because these updates do take quite a bit of time to write.
80 # NLNet Funding announcement
82 An announcement went out
83 [last year](
84 that we'd applied for funding, and we got some great responses and
85 feedback (such as "don't use patented AXI4"). The second time, we
86 sent out a "we got it!" message and got some really nice private and
87 public replies, as well as requests from people to join the team.
88 More on that when it happens.
90 # Coriolis2 experimentation started
92 Jock, a really enthusiastic and clearly skilled and experienced python
93 developer, has this to say about coriolis2:
95 As a humble Python developer, I understand the unique status and
96 significance of the Coriolis project, nevertheless I cannot help
97 but notice that it has a huge room for improvement. I genuinely hope
98 that my participation in libre-riscv will also help improve Coriolis.
100 This was the short version, with a much more
101 [detailed insight](
102 listed here which would do well as a bugreport. However the time it would
103 take is quite significant. We do have funding available from NLNet,
104 so if there is anyone that would like to take this on, under the supervision
105 of Jean-Paul at, we can look at facilitating that.
107 One of the key insights that Jock came up with was that the coding style,
108 whilst consistent, is something that specifically has to be learned, and,
109 as such, being contrary to PEP8 in so many ways, creates an artificially
110 high barrier and learning curve.
112 Even particularly experienced cross-language developers such as
113 myself tend to be able to *read* such code, but editing it, when
114 commas separating list items are on the beginning of lines, results in
115 syntax errors automatically introduced *without thinking* because we
116 automatically add them *at the end* because it looks like one is missing.
118 This is why we insisted on PEP8 in the
119 [HDL workflow]( document.
121 Other than that: coriolis2 is actually extremely exciting to work with.
122 Anyone who has done manual PCB layout will know quite how much of a relief
123 it is to have auto-routing: this is what coriolis2 has by the bucket-load,
124 *as well* as auto-placement. We are looking at half a *million* objects
125 (Cells) to place. Without an auto-router / auto-placer this is just a
126 flat-out impossible task.
128 The first step was to
129 [learn and adapt coriolis2](
130 which was needed to find out how much work would be involved, as much as
131 anything else, in order to be able to accurately assign the fixed budgets
132 to the NLNet milestones. Following on from that, when Jock joined,
133 we needed to work out a compact way to express the
134 [layout of blocks](
135 and he's well on the way to achieving that.
137 Some of the pictures from coriolis2 are
138 [stunning]( This was an
139 experimental routing of the IEEE754 FP 64-bit multiplier. It took
140 5 minutes to run, and is around 50,000 gates: as big as most silicon
141 ASICs that have formerly been done with Coriolis2, and 50% of the
142 practical size that can be handed in one go to the auto-place/auto-router.
144 Other designs using coriolis2 have been of the form where the major "blocks"
145 (such as FPMUL, or Register File) are laid-out automatically in a single-level
146 hierarchy, followed by full and total manual layout from that point onwawrds,
147 in what is termed in the industry as a "Floorplan".
148 With around 500,000 gates to do and many blocks being repeated, this approach
149 is not viable for us. We therefore need a *two* level or potentially three
150 level hierarchy.
152 [Explaining this](
153 to Jean-Paul was amusing and challenging. Much bashing of heads against
154 walls and keyboards was involved. The basic plan: rather than have
155 coriolis2 perform an *entire* layout, in a flat and all-or-nothing fashion,
156 we need a much more subtle fine-grained approach, where *sub-blocks* are
157 laid-out, then *included* at a given level of hierarchy as "pre-done blocks".
159 Save and repeat.
161 This apparently had never been done before, and explaining it in words was
162 extremely challenging. Through a massive hack (actively editing the underlying
163 HDL files temporarily in between tasks) was the only way to illustrate it.
164 However once the lightbulb went on, Jean-Paul was able to get coriolis2's
165 c++ code into shape extremely rapidly, and this alone has opened up an
166 *entire new avenue* of potential for coriolis2 to be used in industry
167 for doing much larger ASICs. Which is precisely the kind of thing that
168 our NLNet sponsors (and the EU, from the Horizon 2020 Grant) love. hooray.
169 Now if only we could actually go to a conference and talk about it.
171 # POWER ISA decoder and Simulator
173 *(kindly written by Michael)*
175 The decoder we have is based on that of IBM's
176 [microwatt reference design](
177 As microwatt's decoder is quite regular, consisting of a bunch of large
178 switch statements returning fields of a struct, we elected not to
179 pursue a direct conversion of the VHDL to nmigen. Instead, we
180 extracted the information in the switch statements into several
181 [CSV tables](,
182 and leveraged nmigen to construct the decoder from these
183 tables. We applied the same technique to extract the subfields
184 (register numbers, branch offset, immediates, etc.) from the
185 instruction, where Luke converted the information in the POWER ISA
186 specification to text, and wrote a module in python to extract those
187 fields from an instruction.
189 To test the decoder, we initially verified it against the tables we
190 extracted, and manually against the [POWER ISA
191 specification]( Later
192 however, we came up with the idea of [verifying the
193 decoder](;a=blob;f=src/soc/decoder/test/;h=9238d3878d964907c5569a3468d6895effb7dc02;hb=433ab59cf9b7ab1ae10754798fc1c110e705db76)
194 against the output of the GNU assembler. This is done by selecting an
195 instruction type (integer reg/reg, integer immediate, load store,
196 etc), and randomly selecting the opcode, registers, immediates, and
197 other operands. We then feed this instruction to GNU AS to assemble,
198 and then the assembled instruction is sent to our decoder. From this,
199 we can then verify that the output of the decoder matches what was
200 generated earlier.
202 We also explored using a similar idea to test the functionality of the
203 entire SOC. By using the [QEMU]( PowerPC
204 emulator, we can compare the execution of our SOC against that of the
205 emulator to verify that our decoder and backend are working correctly.
206 We would write snippets of test code (or potentially randomly generate
207 instructions) and send the resulting binary to both the SOC and
208 QEMU. We would then simulate our SOC until it was finished executing
209 instructions, and use Qemu's gdb interface to do the same. We would
210 then use Qemu's gdb interface to compare the register file and memory
211 with that of our SOC to verify that it is working correctly. I did
212 some experimentation using this technique to verify a [rudimentary
213 simulator](;a=blob;f=src/soc/simulator/;h=aadaf667eff7317b1aa514993cd82b9abedf1047;hb=433ab59cf9b7ab1ae10754798fc1c110e705db76)
214 of the SOC backend, and it seemed to work quite well.
216 *(Note from Luke: this automated approach, taking either other people's
217 regularly-written code or actual PDF specifications, not only saves us a
218 vast amount of time, it also ensures that our implementation is
219 correct and does not contain transcription errors).*
221 # simple-soft-float Library and POWER FP emulation
223 The [simple-soft-float](
224 library is a floating-point library Jacob wrote with the intention
225 of being a reference implementation of IEEE 754 for hardware testing
226 purposes. It's specifically designed to be written to be easier to
227 understand instead of having the code obscured in pursuit of speed:
229 * Being easier to understand helps prevent bugs where the code does not
230 match the IEEE spec.
231 * It uses the [algebraics](
232 library that Jacob wrote since that allows using numbers that behave
233 like exact real numbers, making reasoning about the code simpler.
234 * It is written in Rust rather than highly-macro-ified C, since that helps with
235 readability since operations aren't obscured, as well as safety, since Rust
236 proves at compile time that the code won't seg-fault unless you specifically
237 opt-out of those guarantees by using `unsafe`.
239 It currently supports 16, 32, 64, 128-bit FP for RISC-V, along with
240 having a `DynamicFloat` type which allows dynamically specifying all
241 aspects of how a particular floating-point type behaves -- if one wanted,
242 they could configure it as a 2048-bit floating-point type.
244 It also has Python bindings, thanks to the awesome
245 [PyO3]( library for writing Python bindings in Rust.
247 We decided to write simple-soft-float instead
248 of extending the industry-standard [Berkeley
249 softfloat]( library
250 because of a range of issues, including not supporting Power FP, requiring
251 recompilation to switch which ISA is being emulated, not supporting
252 all the required operations, architectural issues such as depending on
253 global variables, etc. We are still testing simple-soft-float against
254 Berkeley softfloat where we can, however, since Berkeley softfloat is
255 widely used and highly likely to be correct.
257 simple-soft-float is [gaining support for Power
258 FP](, which requires
259 rewriting a lot of the status-flag handling code since Power supports a
260 much larger set of floating-point status flags and exceptions than most
261 other ISAs.
263 Thanks to Raptor CS for giving us remote access to a Power9 system,
264 since that makes it much easier verifying that the test cases are correct
265 (more on this below).
267 API Docs for stable releases of both
268 [simple-soft-float]( and
269 [algebraics]( are available on
271 The algebraics library was chosen as the
272 [Crate of the Week for October 8, 2019 on This Week in
273 Rust](
275 One of the really important things about these libraries: they're not
276 specifically coded exclusively for Libre-SOC: like Berkeley softfloat itself
277 (and also like the [IEEE754 FPU](
278 they're intended for *general-purpose* use by other projects. These are
279 exactly the kinds of side-benefits for the wider Libre community that
280 sponsorship, from individuals, Foundations (such as NLNet) and Companies
281 (such as Purism and Raptor CS) brings.
283 # Kazan Getting a New Shader Compiler IR
285 After spending several weeks only to discover that translating directly from
286 SPIR-V to LLVM IR, Vectorizing, and all the other front-end stuff all in a
287 single step is not really feasible, Jacob has switched to [creating a new
288 shader compiler IR]( to allow
289 decomposing the translation process into several smaller steps.
291 The IR and
292 SPIR-V to IR translator are being written simultaneously, since that allows
293 more easily finding the things that need to be represented in the shader
294 compiler IR. Because writing both of the IR and SPIR-V translator together is
295 such a big task, we decided to pick an arbitrary point ([translating a totally
296 trivial shader into the IR](
297 and split it into tasks at that point so Jacob would be able to get paid
298 after several months of work.
300 The IR uses structured control-flow inspired by WebAssembly's control-flow
301 constructs as well as
302 [SSA]( but, instead
303 of using traditional phi instructions, it uses block and loop parameters and
304 return values (inspired by [Cranelift's EBB
305 parameters](
306 as well as both of the [Rust]( and [Lua]( programming languages).
308 The IR has a single pointer type for all data pointers (`data_ptr`), unlike LLVM IR where pointer types have a type they point to (like `* i32`, where `i32` is the type the pointer points to).
310 Because having a serialized form of the IR is important for any good IR, like
311 LLVM IR, it has a user-friendly textual form that can be both read and
312 written without losing any information (assuming the IR is valid, comments are
313 ignored). A binary form may be added later.
315 Some example code (the IR is likely to change somewhat):
317 ```
318 # this is a comment, comments go from the `#` character
319 # to the end of the line.
321 fn function1[] -> ! {
322 # declares a function named function1 that takes
323 # zero parameters and doesn't return
324 # (the return type is !, taken from Rust).
325 # If the function could return, there would instead be
326 # a list of return types:
327 # fn my_fn[] -> [i32, i64] {...}
328 # my_fn returns an i32 and an i64. The multiple
329 # returned values is inspired by Lua's multiple return values.
331 # the hints for this function
332 hints {
333 # there are no inlining hints for this function
334 inlining_hint: none,
335 # this function doesn't have a side-effect hint
336 side_effects: normal,
337 }
339 # function local variables
340 {
341 # the local variable is an i32 with an
342 # alignment of 4 bytes
343 i32, align: 0x4 -> local_var1: data_ptr;
344 # the pointer to the local variable is
345 # assigned to local_var1 which has the type data_ptr
346 }
348 # the function body is a single block -- block1.
349 # block1's return types are instead attached to the
350 # function signature above
351 # (the `-> !` in the `fn function1[] -> !`).
352 block1 {
353 # the first instruction is a loop named loop1.
354 # the initial value of loop_var is the_const,
355 # which is a named constant.
356 # the value of the_const is the address of the
357 # function `function1`.
358 loop loop1[the_const: fn function1] -> ! {
359 # loop1 takes 1 parameter, which is assigned
360 # to loop_var. the type of loop_var is a pointer to a
361 # function which takes no parameters and doesn't
362 # return.
363 -> [loop_var: fn[] -> !];
365 # the loop body is a single block -- block2.
366 # block2's return value definitions are instead
367 # attached to the loop instruction above
368 # (the `-> !` in the `loop loop1[...] -> !`).
369 block2 {
371 # block3 is a block instruction, it returns
372 # two values, which are assigned to a and b.
373 # Both of a and b have type i32.
374 block block3 -> [a: i32, b: i32] {
375 # the only way a block can return is by
376 # being broken out of using the break
377 # instruction. It is invalid for execution
378 # to reach the end of a block.
380 # this break instruction breaks out of
381 # block3, making block3 return the
382 # constants 1 and 2, both of type i32.
383 break block3[1i32, 2i32];
384 };
386 # an add instruction. The instruction adds
387 # the value `a` (returned by block3 above) to
388 # the constant `increment` (which is an i32
389 # with the value 0x1), and stores the
390 # result in the value `"a"1`. The source-code
391 # location for the add instruction is specified
392 # as being line 12, column 34, in the file
393 # `source_file.vertex`.
394 add [a, increment: 0x1i32]
395 -> ["a"1: i32] @ "source_file.vertex":12:34;
397 # The `"a"1` name is stored as just `a` in
398 # the IR, where the 1 is a numerical name
399 # suffix to differentiate between the two
400 # values with name `a`. This allows robustly
401 # handling duplicate names, by using the
402 # numerical name suffix to disambiguate.
403 #
404 # If a name is specified without the numerical
405 # name suffix, the suffix is assumed to be the
406 # number 0. This also allows handling names that
407 # have unusual characters or are just the empty
408 # string by using the form with the numerical
409 # suffix:
410 # `""0` (empty string)
411 # `"\n"0` (a newline)
412 # `"\u{12345}"0` (the unicode scalar value 0x12345)
415 # this continue instruction jumps back to
416 # the beginning of loop1, supplying the new
417 # values of the loop parameters. In this case,
418 # we just supply loop_var as the value for
419 # the parameter, which just gets assigned to
420 # loop_var in the next iteration.
421 continue loop1[loop_var];
422 }
423 };
424 }
425 }
426 ```
428 # OpenPOWER Conference calls
430 We've now established a routine two-week conference call with Hugh Blemings,
431 OpenPOWER Foundation Director, and Timothy Pearson, CEO of Raptor CS. This
432 allows us to keep up-to-date (each way) on both our new venture and also
433 the newly-announced OpenPOWER Foundation effort as it progresses.
435 One of the most important things that we, Libre-SOC, need, and are
436 discussing with Hugh and Tim is: a way to switch on/off functionality
437 in the (limited) 32-bit opcode space, so that we have one mode for
438 "POWER 3.0B compliance" and another for "things that are absolutely
439 essential to make a decent GPU". With these two being strongly
440 mutually exclusively incompatible, this is just absolutely critical.
442 Khronos Vulkan Floating-point Compliance is, for example, critical not
443 just from a Khronos Trademark Compliance perspective, it's essential
444 from a power-saving and thus commercial success perspective. If we
445 have absolute strict compliance with IEEE754 for POWER 3.0B, this will
446 result in far more silicon than any commercially-competitive GPU on
447 the market, and we will not be able to sell product. Thus it is
448 *commercially* essential to be able to swap between POWER Compliance
449 and Khronos Compliance *at the silicon level*.
451 POWER 3.0B does not have c++ style LR/SC atomic operations for example,
452 and if we have half a **million** 3D GPU data structures **per second**
453 that need SMP-level inter-core mutexes, and the current POWER 3.0B
454 multi-instruction atomic operations are used - conforming strictly to
455 the standard - we're highly likely to use 10 to 15 **percent** processing
456 power consumed on spin-locking. Finding out from Tim on one of these
457 calls that this is something that c++ atomics is something that end-users
458 have been asking about is therefore a good sign.
460 Adding new and essential features that could well end up in a future version
461 of the POWER ISA *need* to be firewalled in a clean way, and we've been
462 asked to [draft a letter](
463 to some of the (very busy) engineers with a huge amount of knowledge
464 and experience inside IBM, for them to consider. Some help in reviewing
465 it would be greatly appreciated.
467 These and many other things are why the calls with Tim and Hugh are a
468 good idea. The amazing thing is that they're taking us seriously, and
469 we can discuss things like those above with them.
471 Other nice things we learned (more on this below) is that Epic Games
472 and RaptorCS are collaborating to get POWER9 supported in Unreal Engine.
473 And that the idea has been very tentatively considered to use our design
474 for the "boot management" processor, running
475 [OpenBMC]( These are early days,
476 it's just ideas, ok! Aside from anything, we actually have to get a chip
477 done, first.
479 # OpenPower Virtual Coffee Meetings
481 The "Virtual Coffee Meetings", announced
482 [here](
483 are literally open to anyone interested in OpenPOWER (if you're strictly
484 Libre there's a dial-in method). These calls are not recorded, it's
485 just an informal conversation.
487 What's a really nice surprise is finding
488 out that Paul Mackerras, whom I used to work with 20 years ago, is *also*
489 working on OpenPOWER, specifically
490 [microwatt](, being managed
491 by Anton Blanchard.
493 A brief discussion led to learning that Paul is looking at adding TLB
494 (Virtual Memory) support to microwatt, specifically the RADIX TLB.
495 I therefore pointed him at the same resource
496 [(power-gem5)](
497 that Hugh had kindly pointed me at, the week before, and did a
498 [late night write-up](
500 My feeling is that these weekly round-table meetings are going to be
501 really important for everyone involved in OpenPOWER. It's a community:
502 we help each other.
504 # Sponsorship by RaptorCS with a TALOS II Workstation
506 With many thanks to Timothy from
507 [RaptorCS](, we've a new shiny
508 online server that needs
509 [setting up](
510 This machine is not just a "nice-to-have", it's actually essential for
511 us to be able to verify against. As you can see in the bugreport, the idea
512 is to bootstrap our way from running IEEE754 FP on a *POWER* system
513 (using typically gnu libm), verifying Jacob's algorithmic FP library
514 particularly and specifically for its rounding modes and exception modes.
516 Once that is done, then apart from having a general-purpose library that
517 is compliant with POWER IEEE754 which *anyone else can use*, we can use
518 that to run unit tests against our[
519 hardware IEEE754 FP library](;a=summary) -
520 again, a resource that anyone may use in any arbitrary project - verifying
521 that it is also correct. This stepping-stone "bootstrap" method we are
522 deploying all over the place, however to do so we need access to resources
523 that have correctly-compliant implementations in the first place. Thus,
524 the critical importance of access to a TALOS II POWER9 workstation.
526 # Epic Megagrants
528 Several months back I got word of the existence of Epic Games' "Megagrants".
529 In December 2019 they announced that so far they've given
530 [USD $13 million](
531 to 200 recipients, so far: one of them, the Blender Foundation, was
532 [USD $1.2 million](!
533 This is an amazing and humbling show of support for the 3D Community,
534 world-wide.
536 It's not just "games", or products specifically using the Unreal Engine:
537 they're happy to look at anything that "enhances Libre / Open source"
538 capabilities for the 3D Graphics Community.
540 A full hybrid 3D-capable CPU-GPU-VPU which is fully-documented not just in
541 its capabilities, that [documentation]( and
542 [full source code]( kinda extends
543 right the way through the *entire development process* down to the bedrock
544 of the actual silicon - not just the firmware, bootloader and BIOS,
545 *everything* - in my mind it kinda qualifies in way that can, in some
546 delightful way, be characterised delicately as "complete overkill".
548 Interestingly, guys, if you're reading this: Tim, the CEO of RaptorCS
549 informs us that you're working closely with his team to get the Unreal
550 Engine up and running on the POWER architecture? Wouldn't that be highly
551 amusing, for us to be able to run the Unreal Engine on the Libre-SOC,
552 given that it's going to be POWER compatible hardware, as a test,
553 first initially in FPGA and then in 18-24 months, on actual silicon, eh?
555 So, as I mentioned
556 [on the list](
557 (reiterating what I put in the original application), we're happy with
558 USD $25,000, we're happy with USD $10 million. It's really up to you guys,
559 at Epic Games, as to what level you'd like to see us get to, and how fast.
561 USD $600,000 for example we can instead of paying USD $1million to a proprietary
562 company to license a DDR3 PHY for a limited one-time use and only a 32-bit
563 wide interface, we can contract SymbioticEDA to *design* a DDR3 PHY for us,
564 which both we *and the rest of the worldwide Silicon Community can use
565 without limitation* because we will ask SymbioticEDA to make the design
566 (and layout) libre-licensed, for anyone to use.
568 USD 250,000 pays for the mask charges that will allow us to do the 40nm
569 quad-core ASIC that we have on the roadmap for the second chip. USD
570 $1m pays for 28nm masks (and so on, in an exponential ramp-up). No, we
571 don't want to do that straight away: yes we do want to go through a first
572 proving test ASIC in 180nm, which, thanks to NLNet, is already funded.
573 This is just good sane sensible use of funds.
575 Even USD $25,000 helps us to cover things such as administration of the
576 website (which is taking up a *lot* of time) and little things that we
577 didn't quite foresee when putting in the NLNet Grant Applications.
579 Lastly, one of the conditions as I understood it from the Megagrants
580 process is that the funds are paid in "stages". This is exactly
581 what NLNet does for (and with) us, right now. If you wanted to save
582 administrative costs, there may be some benefit to having a conversation
583 with the [30-year-old](
584 NLNet Charitable Foundation. Something to think about?
586 # NLNet Milestone tasks
588 Part of applying for NLNet's Grants is a requirement to create a list
589 of tasks, each of which is assigned a budget. On 100% completion of the task,
590 donations can be sent out. With *six* new proposals accepted, each of which
591 required between five (minimum) and *ninteen* separate and distinct tasks,
592 a call with Michiel and Joost turned into an unexpected three hour online
593 marathon, scrambling to write almost fifty bugreports as part of the Schedule
594 to be attached to each Memorandum of Understanding. The mailing list
595 got a [leeetle bit busy](
596 right around here.
598 Which emphasised for us the important need to subdivide the mailing list into
599 separate lists (below).
601 # Georgia Tech CREATE-X
603 (*This section kindly written by Yehowshua*)
605 Yehowshua is a student at Georgia Tech currently pursuing a Masters in
606 Computer Engineering - to graduate this summer. He had started working
607 on LibreSOC in December and wanted to to get LibreSOC more funding so
608 I could work on it full time.
610 He originally asked if the ECE Chair at Georgia Tech would be willing
611 to fund an in-department effort to deliver an SOC in collaboration
612 with LibreSOC(an idea to which he was quite receptive). Through Luke,
613 Yehowshua got in contact with Chistopher Klaus who suggested Yehowshua
614 should look into Klaus's startup accelerator program Create-X and perhaps
615 consider taking LibreSOC down the startup route. Robert Rhinehart, who
616 had funded LibreSOC a little in the past (*note from Luke: he donated
617 the ZC706 and also funded modernisation of Richard Herveille's excellent
618 [vga_lcd]( Library*)
619 also suggested that Yehowshua
620 incorporate LibreSOC with help from Create-X and said he would be willing
621 to be a seed investor. All this happened by February.
623 As of March, Yehowshua has been talking with Robert about what type of
624 customers would be interested in LibreSOC. Robert is largely interested in
625 biological applications. Yehowshua also had a couple meetings with Rahul
626 from Create-X. Yehowshua has started the incorporation of LibreSOC. The
627 parent company will probably be called Systèmes-Libres with LibreSOC
628 simply being one of the products we will offer. Yehowshua also attended
629 HPCA in late February and had mentioned LIbreSOC during his talk. People
630 seemed to find the idea quite interesting
632 He will later be speaking with some well know startup lawyers that have
633 an HQ in Atlanta to discuss business related things such as S Corps,
634 C corps, taxes, wages, equity etc…
636 Yehowshua plans for Systèmes-Libres to hire full time employees. Part
637 time work on Libre-SOC will still be possible through donations and
638 support from NL Net and companies like purism.
640 Currently, Yehowshua plans to take the Create-X summer launch program
641 and fund Systèmes-Libres by August. Full time wages would probably be
642 set around 100k USD.
644 # LOAD/STORE Buffer and 6600 design documentation
646 A critical part of this project is not just to create a chip, it's to
647 *document* the chip design, the decisions along the way, for both
648 educational, research, and ongoing maintenance purposes. With an
649 augmented CDC 6600 design being chosen as the fundamental basis,
650 [documenting that](
651 as well as the key differences is particularly important. At the very least,
652 the extremely simple and highly effective hardware but timing-critical
653 design aspects of the circular loops in the 6600 were recognised by James
654 Thornton (the co-designer of the 6600) as being paradoxically challenging
655 to understand why so few gates could be so effective (being as they were,
656 literally the world's first ever out-of-order superscalar architecture).
657 Consequently, documenting it just to be able to *develop* it is extremely
658 important.
660 We're getting to the point where we need to connect the LOAD/STORE Computation
661 Units up to an actual memory architecture. We've chosen
662 [minerva](
663 as the basis because it is written in nmigen, works, and, crucially, uses
664 wishbone (which we decided to use as the main Bus Backbone a few months ago).
666 However, unlike minerva, which is a single-issue 32-bit embedded chip,
667 where it's perfectly ok to have one single LD/ST operation per clock,
668 and not only that but to have that operation take a few clock cycles,
669 to get anything like the level of performance needed of a GPU, we need
670 at least four 64-bit LOADs or STOREs *every clock cycle*.
672 For a first ASIC from a team that's never done a chip before, this is,
673 officially, "Bonkers Territory". Where minerva is doing 32-bit-wide
674 Buses (and does not support 64-bit LD/ST at all), we need internal
675 data buses of a minimum whopping **2000** wires wide.
677 Let that sink in for a moment.
679 The reason why the internal buses need to be 2000 wires wide comes down
680 to the fact that we need, realistically, 6 to eight LOAD/STORE Computation
681 Units. 4 of them will be operational, 2 to 4 of them will be waiting
682 with pending instructions from the multi-issue Vectorisation Engine.
684 We chose to use a system which expands the first 4 bits of the address,
685 plus the operation width (1,2,4,8 bytes) into a "bitmap" - a byte-mask -
686 that corresponds directly with the 16 byte "cache line" byte enable
687 columns, in the L1 Cache. These bitmaps can then be "merged" such
688 that requests that go to the same cache line can be served *in the
689 same clock cycle* to multiple LOAD/STORE Computation Units. This
690 being absolutely critical for effective Vector Processing.
692 Additionally, in order to deal with misaligned memory requests, each of those
693 needs to put out *two* such 16-byte-wide requests (see where this is going?)
694 out to the L1 Cache.
695 So, we now have eight times two times 128 bits which is a staggering
696 2048 wires *just for the data*. There do exist ways to get that down
697 (potentially to half), and there do exist ways to get that cut in half
698 again, however doing so would miss opportunities for merging of requests
699 into cache lines.
701 At that point, thanks to Mitch Alsup's input (Mitch is the designer of
702 the Motorola 68000, Motorola 88120, key architecture on AMD's Opteron
703 Series, the AMD K9, AMDGPU and Samsung's latest GPU), we learned that
704 L1 cache design critically depends on what type of SRAM you have. We
705 initially, naively, wanted dual-ported L1 SRAM and that's when Staf
706 and Mitch taught us that this results in half-duty rate. Only
707 1-Read **or** 1-Write SRAM Cells give you fast enough (single-cycle)
708 data rates to be useable for L1 Caches.
710 Part of the conversation has wandered into
711 [why we chose dynamic pipelines](
712 as well as receiving that
713 [important advice](
714 from both Mitch Alsup and Staf Verhaegen.
716 (Staf is also [sponsored by NLNet](
717 to create Libre-licensed Cell Libraries, busting through one of the -
718 many - layers of NDAs and reducing NREs and unnecessary and artificial
719 barriers for ASIC development: I helped him put in the submission, and
720 he was really happy to do the Cell Libraries that we will be using for
721 LibreSOC's 180nm test tape-out in October 2020.)
723 # Public-Inbox and Domain Migration
725 As mentioned before, one of the important aspects of this project is
726 the documentation and archiving. It also turns out that when working
727 over an extremely unreliable or ultra-expensive mobile broadband link,
728 having *local* (offline) access to every available development resource
729 is critically important.
731 Hence why we are going to the trouble of installing public-inbox, due
732 to its ability to not only have a mailing list entirely stored in a
733 git repository, the "web service" which provides access to that git-backed
734 archive can be not only mirrored elsewhere, it can be *run locally on
735 your own local machine* even when offline. This in combination
736 with the right mailer setup can store-and-forward any replies to the
737 (offline-copied) messages, such that they can be sent when internet
738 connectivity is restored, yet remain a productive collaborative developer.
740 Now you know why we absolutely do not accept "slack", or other proprietary
741 "online oh-so-convenient" service. Not only is it highly inappropriate for
742 Libre Projects, not only do we become critically dependent on the Corporation
743 running the service (yes, github has been entirely offline, several times),
744 if we have remote developers (such as myself, working from Scotland last
745 month with sporadic access to a single Cell Tower) or developers in emerging
746 markets where their only internet access is via a Library or Internet Cafe,
747 we absolutely do not want to exclude or penalise such people, just because
748 they have less resources.
750 Fascinatingly, Linus Torvals is *specifically*
751 [on record](
752 about making sure that "Linux development does not favour wealthy people".
754 We are also, as mentioned before, moving to a new domain name. We'll take
755 the opportunity to fix some of the issues with HTTPS (wrong certificate),
756 and also do some
757 [better mailing list names](
758 at the same time.
760 TODO (Veera?) bit about what was actually done, how it links into mailman2.
762 # OpenPOWER HDL Mailing List opens up
764 It is early days, however it is fantastic to see responses from IBM with
765 regards to requests for access to the POWER ISA Specification
766 documents in
767 [machine-readable form](
768 I took Jeff at his word and explained, in some detail,
769 [exactly why](
770 machine readable versions of specifications are critically important.
772 The takeaway is: *we haven't got time to do manual transliteration of the spec*
773 into "code". We're expending considerable effort making sure that we
774 "bounce" or "bootstrap" off of pre-existing resources, using computer
775 programs to do so.
777 This "trick" is something that I learned over 20 years ago, when developing
778 an SMB Client and Server in something like two weeks flat. I wrote a
779 parser which read the packet formats *from the IETF Draft Specification*,
780 and outputted c-code.
782 This leaves me wondering, as I mention on the HDL list, if we can do the same
783 thing with large sections of the POWER Spec.
785 # Build Servers
787 TODO
789 # Conclusion
791 I'm not going to mention anything about the current world climate: you've
792 seen enough news reports. I will say (more about this through the
793 [EOMA68]( updates) that
794 I anticipated something like what is happening right now, over ten years
795 ago. I wasn't precisely expecting what *has* happened, just the consequences:
796 world-wide travel shut-down, and for people - the world over - to return to
797 local community roots.
799 However what I definitely wasn't expecting was a United States President
800 to be voted in who was eager and, frankly, stupid enough, to start *and
801 escalate* a Trade war with China. The impact on the U.S economy alone, and the
802 reputation of the whole country, has been detrimental in the extreme.
804 This combination leaves us - world-wide - with the strong possibility that
805 seemed so "preposterous" that I could in no way discuss it widely, let alone
806 mention it on something like a Crowdsupply update, that thanks to the
807 business model on which their entire product lifecycle is predicated,
808 in combination with the extremely high NREs and development costs for
809 ASICs (custom silicon costs USD $100 million, these days), several
810 large Corporations producing proprietary binary-only drivers for
811 hardware on which we critically rely for our internet-connected way
812 of life **may soon go out of business**.
814 Right at a critical time where video conferencing is taking off massively,
815 your proprietary hardware - your smartphone, your tablet, your laptop,
816 everything you rely on for connectivity to the rest of the world, all of
817 a sudden **you may not be able to get software updates** or, worse,
818 your products could even be
819 [remotely shut down](
820 **without warning**.
822 I do not want to hammer the point home too strongly but you should be
823 getting, in no uncertain terms, exactly how strategically critical, in
824 the current world climate, this project just became. We need to get it
825 accelerated, completed, and into production, in an expedited and responsible
826 fashion.