X-Git-Url: https://git.libre-soc.org/?a=blobdiff_plain;f=updates%2F023_2020mar26_decoder_emulator_started.mdwn;h=4c6d08a7211a4d13bc787a98e97f7b441b38e81a;hb=d61cf4ed006eb3144b9fab72fa3df3fd7346a4fa;hp=b92cc1bbc94aaad0ea9d8e8ceec073a9f60ada13;hpb=31df5af0a3f7c3aefdeed14ba551722df6b0d5b8;p=crowdsupply.git diff --git a/updates/023_2020mar26_decoder_emulator_started.mdwn b/updates/023_2020mar26_decoder_emulator_started.mdwn index b92cc1b..4c6d08a 100644 --- a/updates/023_2020mar26_decoder_emulator_started.mdwn +++ b/updates/023_2020mar26_decoder_emulator_started.mdwn @@ -25,7 +25,9 @@ Here's the summary (if it can be called a summary): for verification. * Jacob's simple-soft-float library growing [Power FP compatibility](http://bugs.libre-riscv.org/show_bug.cgi?id=258) - and python bindings. + and python bindings. +* Kazan, the Vulkan driver Jacob is writing, is getting + a [new shader compiler IR](http://bugs.libre-riscv.org/show_bug.cgi?id=161). * A Conference call with OpenPOWER Foundation Director, Hugh, and Timothy Pearson from RaptorCS has been established every two weeks. * The OpenPOWER Foundation is also running some open @@ -68,7 +70,7 @@ Here's the summary (if it can be called a summary): Well dang, as you can see, suddenly it just went ballistic. There's almost certainly things left off the list. For such a small team there's a heck of a lot going on. We have an awful lot to do, in a short amount -of time: the 180nm tape-out is in October 2020 - only 7 months away. +of time: the 180nm tape-out is in October 2020. With this update we're doing something slightly different: a request has gone out [to the other team members](http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-March/005428.html) @@ -87,16 +89,140 @@ More on that when it happens. # Coriolis2 experimentation started -TODO by Jock http://bugs.libre-riscv.org/show_bug.cgi?id=217#c44 +Jock, a really enthusiastic and clearly skilled and experienced python +developer, has this to say about coriolis2: + + As a humble Python developer, I understand the unique status and + significance of the Coriolis project, nevertheless I cannot help + but notice that it has a huge room for improvement. I genuinely hope + that my participation in libre-riscv will also help improve Coriolis. + +This was the short version, with a much more +[detailed insight](http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-March/005478.html) +listed here which would do well as a bugreport. However the time it would +take is quite significant. We do have funding available from NLNet, +so if there is anyone that would like to take this on, under the supervision +of Jean-Paul at LIP6.fr, we can look at facilitating that. + +One of the key insights that Jock came up with was that the coding style, +whilst consistent, is something that specifically has to be learned, and, +as such, being contrary to PEP8 in so many ways, creates an artificially +high barrier and learning curve. + +Even particularly experienced cross-language developers such as +myself tend to be able to *read* such code, but editing it, when +commas separating list items are on the beginning of lines, results in +syntax errors automatically introduced *without thinking* because we +automatically add them *at the end* because it looks like one is missing. + +This is why we insisted on PEP8 in the +[HDL workflow](http://libre-riscv.org/HDL_workflow) document. + +Other than that: coriolis2 is actually extremely exciting to work with. +Anyone who has done manual PCB layout will know quite how much of a relief +it is to have auto-routing: this is what coriolis2 has by the bucket-load, +*as well* as auto-placement. We are looking at half a *million* objects +(Cells) to place. Without an auto-router / auto-placer this is just a +flat-out impossible task. + +The first step was to +[learn and adapt coriolis2](http://bugs.libre-riscv.org/show_bug.cgi?id=178) +which was needed to find out how much work would be involved, as much as +anything else, in order to be able to accurately assign the fixed budgets +to the NLNet milestones. Following on from that, when Jock joined, +we needed to work out a compact way to express the +[layout of blocks](http://bugs.libre-riscv.org/show_bug.cgi?id=217#c44) +and he's well on the way to achieving that. + +Some of the pictures from coriolis2 are +[stunning](bugs.libre-riscv.org/attachment.cgi?id=29). This was an +experimental routing of the IEEE754 FP 64-bit multiplier. It took +5 minutes to run, and is around 50,000 gates: as big as most silicon +ASICs that have formerly been done with Coriolis2, and 50% of the +practical size that can be handed in one go to the auto-place/auto-router. + +Other designs using coriolis2 have been of the form where the major "blocks" +(such as FPMUL, or Register File) are laid-out automatically in a single-level +hierarchy, followed by full and total manual layout from that point onwawrds, +in what is termed in the industry as a "Floorplan". +With around 500,000 gates to do and many blocks being repeated, this approach +is not viable for us. We therefore need a *two* level or potentially three +level hierarchy. + +[Explaining this](http://bugs.libre-riscv.org/show_bug.cgi?id=178#c146) +to Jean-Paul was amusing and challenging. Much bashing of heads against +walls and keyboards was involved. The basic plan: rather than have +coriolis2 perform an *entire* layout, in a flat and all-or-nothing fashion, +we need a much more subtle fine-grained approach, where *sub-blocks* are +laid-out, then *included* at a given level of hierarchy as "pre-done blocks". + +Save and repeat. + +This apparently had never been done before, and explaining it in words was +extremely challenging. Through a massive hack (actively editing the underlying +HDL files temporarily in between tasks) was the only way to illustrate it. +However once the lightbulb went on, Jean-Paul was able to get coriolis2's +c++ code into shape extremely rapidly, and this alone has opened up an +*entire new avenue* of potential for coriolis2 to be used in industry +for doing much larger ASICs. Which is precisely the kind of thing that +our NLNet sponsors (and the EU, from the Horizon 2020 Grant) love. hooray. +Now if only we could actually go to a conference and talk about it. # POWER ISA decoder and Simulator -TODO +*(kindly written by Michael)* + +The decoder we have is based on that of IBM's +[microwatt reference design](https://github.com/antonblanchard/microwatt). +As microwatt's decoder is quite regular, consisting of a bunch of large +switch statements returning fields of a struct, we elected not to +pursue a direct conversion of the VHDL to nmigen. Instead, we +extracted the information in the switch statements into several +[CSV tables](https://libre-riscv.org/openpower/isatables/), +and leveraged nmigen to construct the decoder from these +tables. We applied the same technique to extract the subfields +(register numbers, branch offset, immediates, etc.) from the +instruction, where Luke converted the information in the POWER ISA +specification to text, and wrote a module in python to extract those +fields from an instruction. + +To test the decoder, we initially verified it against the tables we +extracted, and manually against the [POWER ISA +specification](https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0). Later +however, we came up with the idea of +[verifying the decoder](https://git.libre-riscv.org/?p=soc.git;a=blob;f=src/soc/decoder/test/test_decoder_gas.py;h=9238d3878d964907c5569a3468d6895effb7dc02;hb=433ab59cf9b7ab1ae10754798fc1c110e705db76) +against the output of the GNU assembler. This is done by selecting an +instruction type (integer reg/reg, integer immediate, load store, +etc), and randomly selecting the opcode, registers, immediates, and +other operands. We then feed this instruction to GNU AS to assemble, +and then the assembled instruction is sent to our decoder. From this, +we can then verify that the output of the decoder matches what was +generated earlier. + +We also explored using a similar idea to test the functionality of the +entire SOC. By using the [QEMU](https://www.qemu.org/) PowerPC +emulator, we can compare the execution of our SOC against that of the +emulator to verify that our decoder and backend are working correctly. +We would write snippets of test code (or potentially randomly generate +instructions) and send the resulting binary to both the SOC and +QEMU. We would then simulate our SOC until it was finished executing +instructions, and use Qemu's gdb interface to do the same. We would +then use Qemu's gdb interface to compare the register file and memory +with that of our SOC to verify that it is working correctly. I did +some experimentation using this technique to verify a +[rudimentary simulator](https://git.libre-riscv.org/?p=soc.git;a=blob;f=src/soc/simulator/test_sim.py;h=aadaf667eff7317b1aa514993cd82b9abedf1047;hb=433ab59cf9b7ab1ae10754798fc1c110e705db76) +of the SOC backend, and it seemed to work quite well. + +*(Note from Luke: this automated approach, taking either other people's +regularly-written code or actual PDF specifications, not only saves us a +vast amount of time, it also ensures that our implementation is +correct and does not contain transcription errors).* # simple-soft-float Library and POWER FP emulation -The -[simple-soft-float](https://salsa.debian.org/Kazan-team/simple-soft-float) +(*written kindly by Jacob*) + +The [simple-soft-float](https://salsa.debian.org/Kazan-team/simple-soft-float) library is a floating-point library Jacob wrote with the intention of being a reference implementation of IEEE 754 for hardware testing purposes. It's specifically designed to be written to be easier to @@ -136,7 +262,7 @@ rewriting a lot of the status-flag handling code since Power supports a much larger set of floating-point status flags and exceptions than most other ISAs. -Thanks to RaptorCS for giving us remote access to a Power9 system, +Thanks to Raptor CS for giving us remote access to a Power9 system, since that makes it much easier verifying that the test cases are correct (more on this below). @@ -144,17 +270,109 @@ API Docs for stable releases of both [simple-soft-float](https://docs.rs/simple-soft-float) and [algebraics](https://docs.rs/algebraics) are available on docs.rs. +The algebraics library was chosen as the +[Crate of the Week for October 8, 2019 on This Week in +Rust](https://this-week-in-rust.org/blog/2019/10/08/this-week-in-rust-307/#crate-of-the-week). + One of the really important things about these libraries: they're not -specifically coded exclusively for Libre-SOC: like softfloat-3 itself +specifically coded exclusively for Libre-SOC: like Berkeley softfloat itself (and also like the [IEEE754 FPU](https://git.libre-riscv.org/?p=ieee754fpu.git)) they're intended for *general-purpose* use by other projects. These are exactly the kinds of side-benefits for the wider Libre community that sponsorship, from individuals, Foundations (such as NLNet) and Companies (such as Purism and Raptor CS) brings. +# Kazan Getting a New Shader Compiler IR + +(*written kindly by Jacob, a dedicated update on Kazan will definitely +feature in the future*) + +After spending several weeks only to discover that translating directly from +SPIR-V to LLVM IR, Vectorizing, and all the other front-end stuff all in a +single step is not really feasible, Jacob has switched to [creating a new +shader compiler IR](http://bugs.libre-riscv.org/show_bug.cgi?id=161) to allow +decomposing the translation process into several smaller steps. + +The IR and +SPIR-V to IR translator are being written simultaneously, since that allows +more easily finding the things that need to be represented in the shader +compiler IR. Because writing both of the IR and SPIR-V translator together is +such a big task, we decided to pick an arbitrary point ([translating a totally +trivial shader into the IR](http://bugs.libre-riscv.org/show_bug.cgi?id=177)) +and split it into tasks at that point so Jacob would be able to get paid +after several months of work. + +The IR uses structured control-flow inspired by WebAssembly's control-flow +constructs as well as +[SSA](https://en.wikipedia.org/wiki/Static_single_assignment_form) but, instead +of using traditional phi instructions, it uses block and loop parameters and +return values (inspired by +[Cranelift's EBB parameters](https://github.com/bytecodealliance/wasmtime/blob/master/cranelift/docs/ir.md#static-single-assignment-form) +as well as both of the [Rust](https://www.rust-lang.org/) and +[Lua](https://www.lua.org/) programming languages). + +The IR has a single pointer type for all data pointers (`data_ptr`), +unlike LLVM IR where pointer types have a type they point to (like `* +i32`, where `i32` is the type the pointer points to). + +Because having a serialized form of the IR is important for any good IR, +like LLVM IR, it has a user-friendly textual form that can be both read +and written without losing any information (assuming the IR is valid, +comments are ignored). A binary form may be added later. + +Some example IR is +[available in the Kazan repo](https://salsa.debian.org/Kazan-team/kazan/-/blob/master/docs/Shader%20Compiler%20IR%20Example.md). + # OpenPOWER Conference calls -TODO +We've now established a routine two-week conference call with Hugh Blemings, +OpenPOWER Foundation Director, and Timothy Pearson, CEO of Raptor CS. This +allows us to keep up-to-date (each way) on both our new venture and also +the newly-announced OpenPOWER Foundation effort as it progresses. + +One of the most important things that we, Libre-SOC, need, and are +discussing with Hugh and Tim is: a way to switch on/off functionality +in the (limited) 32-bit opcode space, so that we have one mode for +"POWER 3.0B compliance" and another for "things that are absolutely +essential to make a decent GPU". With these two being strongly +mutually exclusively incompatible, this is just absolutely critical. + +Khronos Vulkan Floating-point Compliance is, for example, critical not +just from a Khronos Trademark Compliance perspective, it's essential +from a power-saving and thus commercial success perspective. If we +have absolute strict compliance with IEEE754 for POWER 3.0B, this will +result in far more silicon than any commercially-competitive GPU on +the market, and we will not be able to sell product. Thus it is +*commercially* essential to be able to swap between POWER Compliance +and Khronos Compliance *at the silicon level*. + +POWER 3.0B does not have c++ style LR/SC atomic operations for example, +and if we have half a **million** 3D GPU data structures **per second** +that need SMP-level inter-core mutexes, and the current POWER 3.0B +multi-instruction atomic operations are used - conforming strictly to +the standard - we're highly likely to use 10 to 15 **percent** processing +power consumed on spin-locking. Finding out from Tim on one of these +calls that this is something that c++ atomics is something that end-users +have been asking about is therefore a good sign. + +Adding new and essential features that could well end up in a future version +of the POWER ISA *need* to be firewalled in a clean way, and we've been +asked to [draft a letter](https://libre-riscv.org/openpower/isans_letter/) +to some of the (very busy) engineers with a huge amount of knowledge +and experience inside IBM, for them to consider. Some help in reviewing +it would be greatly appreciated. + +These and many other things are why the calls with Tim and Hugh are a +good idea. The amazing thing is that they're taking us seriously, and +we can discuss things like those above with them. + +Other nice things we learned (more on this below) is that Epic Games +and RaptorCS are collaborating to get POWER9 supported in Unreal Engine. +And that the idea has been very tentatively considered to use our design +for the "boot management" processor, running +[OpenBMC](https://github.com/openbmc/openbmc). These are early days, +it's just ideas, ok! Aside from anything, we actually have to get a chip +done, first. # OpenPower Virtual Coffee Meetings @@ -183,19 +401,144 @@ we help each other. # Sponsorship by RaptorCS with a TALOS II Workstation -TODO http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-March/005291.html +With many thanks to Timothy from +[RaptorCS](https://raptorcs.com), we've a new shiny +online server that needs +[setting up](http://bugs.libre-riscv.org/show_bug.cgi?id=265). +This machine is not just a "nice-to-have", it's actually essential for +us to be able to verify against. As you can see in the bugreport, the idea +is to bootstrap our way from running IEEE754 FP on a *POWER* system +(using typically gnu libm), verifying Jacob's algorithmic FP library +particularly and specifically for its rounding modes and exception modes. + +Once that is done, then apart from having a general-purpose library that +is compliant with POWER IEEE754 which *anyone else can use*, we can use +that to run unit tests against our[ +hardware IEEE754 FP library](https://git.libre-riscv.org/?p=ieee754fpu.git;a=summary) - +again, a resource that anyone may use in any arbitrary project - verifying +that it is also correct. This stepping-stone "bootstrap" method we are +deploying all over the place, however to do so we need access to resources +that have correctly-compliant implementations in the first place. Thus, +the critical importance of access to a TALOS II POWER9 workstation. # Epic Megagrants -TODO +Several months back I got word of the existence of Epic Games' "Megagrants". +In December 2019 they announced that so far they've given +[USD $13 million](https://www.unrealengine.com/en-US/blog/epic-megagrants-reaches-13-million-milestone-in-2019) +to 200 recipients, so far: one of them, the Blender Foundation, was +[USD $1.2 million](https://www.blender.org/press/epic-games-supports-blender-foundation-with-1-2-million-epic-megagrant/)! +This is an amazing and humbling show of support for the 3D Community, +world-wide. + +It's not just "games", or products specifically using the Unreal Engine: +they're happy to look at anything that "enhances Libre / Open source" +capabilities for the 3D Graphics Community. + +A full hybrid 3D-capable CPU-GPU-VPU which is fully-documented not just in +its capabilities, that [documentation](http://libre-riscv.org) and +[full source code](http://git.libre-riscv.org) kinda extends +right the way through the *entire development process* down to the bedrock +of the actual silicon - not just the firmware, bootloader and BIOS, +*everything* - in my mind it kinda qualifies in way that can, in some +delightful way, be characterised delicately as "complete overkill". + +Interestingly, guys, if you're reading this: Tim, the CEO of RaptorCS +informs us that you're working closely with his team to get the Unreal +Engine up and running on the POWER architecture? Wouldn't that be highly +amusing, for us to be able to run the Unreal Engine on the Libre-SOC, +given that it's going to be POWER compatible hardware, as a test, +first initially in FPGA and then in 18-24 months, on actual silicon, eh? + +So, as I mentioned +[on the list](http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-March/005262.html) +(reiterating what I put in the original application), we're happy with +USD $25,000, we're happy with USD $10 million. It's really up to you guys, +at Epic Games, as to what level you'd like to see us get to, and how fast. + +USD $600,000 for example we can instead of paying USD $1million to a +proprietary company to license a DDR3 PHY for a limited one-time use and +only a 32-bit wide interface, we can contract SymbioticEDA to *design* +a DDR3 PHY for us, which both we *and the rest of the worldwide Silicon +Community can use without limitation* because we will ask SymbioticEDA +to make the design (and layout) libre-licensed, for anyone to use. + +USD 250,000 pays for the mask charges that will allow us to do the 40nm +quad-core ASIC that we have on the roadmap for the second chip. USD +$1m pays for 28nm masks (and so on, in an exponential ramp-up). No, we +don't want to do that straight away: yes we do want to go through a first +proving test ASIC in 180nm, which, thanks to NLNet, is already funded. +This is just good sane sensible use of funds. + +Even USD $25,000 helps us to cover things such as administration of the +website (which is taking up a *lot* of time) and little things that we +didn't quite foresee when putting in the NLNet Grant Applications. + +Lastly, one of the conditions as I understood it from the Megagrants +process is that the funds are paid in "stages". This is exactly +what NLNet does for (and with) us, right now. If you wanted to save +administrative costs, there may be some benefit to having a conversation +with the [30-year-old](https://nlnet.nl/foundation/history/) +NLNet Charitable Foundation. Something to think about? # NLNet Milestone tasks -TODO +Part of applying for NLNet's Grants is a requirement to create a list +of tasks, each of which is assigned a budget. On 100% completion of the task, +donations can be sent out. With *six* new proposals accepted, each of which +required between five (minimum) and *ninteen* separate and distinct tasks, +a call with Michiel and Joost turned into an unexpected three hour online +marathon, scrambling to write almost fifty bugreports as part of the Schedule +to be attached to each Memorandum of Understanding. The mailing list +got a +[leeetle bit busy](http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-March/005003.html) +right around here. + +Which emphasised for us the important need to subdivide the mailing list into +separate lists (below). # Georgia Tech CREATE-X -TODO +(*This section kindly written by Yehowshua*) + +Yehowshua is a student at Georgia Tech currently pursuing a Masters in +Computer Engineering - to graduate this summer. He had started working +on Libre-SOC in December and wanted to to get Libre-SOC more funding so +I could work on it full time. + +He originally asked if the ECE Chair at Georgia Tech would be willing +to fund an in-department effort to deliver an SOC in collaboration +with Libre-SOC(an idea to which he was quite receptive). Through Luke, +Yehowshua got in contact with Chistopher Klaus who suggested Yehowshua +should look into Klaus's startup accelerator program Create-X and perhaps +consider taking Libre-SOC down the startup route. Robert Rhinehart, who +had funded Libre-SOC a little in the past (*note from Luke: he donated +the ZC706 and also funded modernisation of Richard Herveille's excellent +[vga_lcd](https://github.com/RoaLogic/vga_lcd) Library*) +also suggested that Yehowshua +incorporate Libre-SOC with help from Create-X and said he would be willing +to be a seed investor. All this happened by February. + +As of March, Yehowshua has been talking with Robert about what type of +customers would be interested in Libre-SOC. Robert is largely interested in +biological applications. Yehowshua also had a couple meetings with Rahul +from Create-X. Yehowshua has started the incorporation of Libre-SOC. The +parent company will probably be called Systèmes-Libres with Libre-SOC +simply being one of the products we will offer. Yehowshua also attended +HPCA in late February and had mentioned Libre-SOC during his talk. People +seemed to find the idea quite interesting + +He will later be speaking with some well know startup lawyers that have +an HQ in Atlanta to discuss business related things such as S Corps, +C corps, taxes, wages, equity etc… + +Yehowshua plans for Systèmes-Libres to hire full time employees. Part +time work on Libre-SOC will still be possible through donations and +support from NL Net and companies like purism. + +Currently, Yehowshua plans to take the Create-X summer launch program +and fund Systèmes-Libres by August. Full time wages would probably be +set around 100k USD. # LOAD/STORE Buffer and 6600 design documentation @@ -208,8 +551,10 @@ as well as the key differences is particularly important. At the very least, the extremely simple and highly effective hardware but timing-critical design aspects of the circular loops in the 6600 were recognised by James Thornton (the co-designer of the 6600) as being paradoxically challenging -to understand why so few gates could be so effective. Consequently, -documenting it just to be able to *develop* it is extremely important. +to understand why so few gates could be so effective (being as they were, +literally the world's first ever out-of-order superscalar architecture). +Consequently, documenting it just to be able to *develop* it is extremely +important. We're getting to the point where we need to connect the LOAD/STORE Computation Units up to an actual memory architecture. We've chosen @@ -269,20 +614,153 @@ from both Mitch Alsup and Staf Verhaegen. (Staf is also [sponsored by NLNet](https://nlnet.nl/project/Chips4Makers/) to create Libre-licensed Cell Libraries, busting through one of the - -many - layers of NDAs and reducing NREs for ASIC development: I helped him -put in the submission, and he was really happy to do the Cell Libraries -that we will be using for LibreSOC's 180nm test tape-out in October 2020.) +many - layers of NDAs and reducing NREs and unnecessary and artificial +barriers for ASIC development: I helped him put in the submission, and +he was really happy to do the Cell Libraries that we will be using for +Libre-SOC's 180nm test tape-out in October 2020.) # Public-Inbox and Domain Migration -TODO - -# Build Servers - -TODO +As mentioned before, one of the important aspects of this project is +the documentation and archiving. It also turns out that when working +over an extremely unreliable or ultra-expensive mobile broadband link, +having *local* (offline) access to every available development resource +is critically important. + +Hence why we are going to the trouble of installing public-inbox, due +to its ability to not only have a mailing list entirely stored in a +git repository, the "web service" which provides access to that git-backed +archive can be not only mirrored elsewhere, it can be *run locally on +your own local machine* even when offline. This in combination +with the right mailer setup can store-and-forward any replies to the +(offline-copied) messages, such that they can be sent when internet +connectivity is restored, yet remain a productive collaborative developer. + +Now you know why we absolutely do not accept "slack", or other proprietary +"online oh-so-convenient" service. Not only is it highly inappropriate for +Libre Projects, not only do we become critically dependent on the Corporation +running the service (yes, github has been entirely offline, several times), +if we have remote developers (such as myself, working from Scotland last +month with sporadic access to a single Cell Tower) or developers in emerging +markets where their only internet access is via a Library or Internet Cafe, +we absolutely do not want to exclude or penalise such people, just because +they have less resources. + +Fascinatingly, Linus Torvals is *specifically* +[on record](https://www.linuxjournal.com/content/line-length-limits) +about making sure that "Linux development does not favour wealthy people". + +We are also, as mentioned before, moving to a new domain name. We'll take +the opportunity to fix some of the issues with HTTPS (wrong certificate), +and also do some +[better mailing list names](http://bugs.libre-riscv.org/show_bug.cgi?id=184) +at the same time. + +TODO (Veera?) bit about what was actually done, how it links into mailman2. + +# OpenPOWER HDL Mailing List opens up + +It is early days, however it is fantastic to see responses from IBM with +regards to requests for access to the POWER ISA Specification +documents in +[machine-readable form](http://lists.mailinglist.openpowerfoundation.org/pipermail/openpower-hdl-cores/2020-March/000007.html) +I took Jeff at his word and explained, in some detail, +[exactly why](http://lists.mailinglist.openpowerfoundation.org/pipermail/openpower-hdl-cores/2020-March/000008.html) +machine readable versions of specifications are critically important. + +The takeaway is: *we haven't got time to do manual transliteration of the spec* +into "code". We're expending considerable effort making sure that we +"bounce" or "bootstrap" off of pre-existing resources, using computer +programs to do so. + +This "trick" is something that I learned over 20 years ago, when developing +an SMB Client and Server in something like two weeks flat. I wrote a +parser which read the packet formats *from the IETF Draft Specification*, +and outputted c-code. + +This leaves me wondering, as I mention on the HDL list, if we can do the same +thing with large sections of the POWER Spec (*answer as of 3rd April 2020: +[yes](https://git.libre-riscv.org/?p=soc.git;a=blob;f=src/soc/decoder/power_pseudo.py;h=f2e575e8c5b707e7ec2f8d2ea6ca6d36060e08ad;hb=af3c6727c8bb59623bf5672b867407b5516e8338)*) + +# Build Servers, Process Automation, and Reducing Cognitive Load + +(*written kindly by Cole*) + +Over the past month, Jacob, and a new project member Cole, set up a new +build server for the project. The build server is an old computer that +Jacob wasn't using anymore, that he decided to make available to the +project for running continuous integration (CI) testing for the many +modules and submodules of the project. The build server is a gitlab test +runner instance using a Docker backend. As Luke has taken pains to make +clear +[many times](https://libre-riscv.org/HDL_workflow/), +very large and complex python projects are guaranteed +to fail without proper, extensive test coverage. This new build server +will allow us to +[automate](http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-April/005687.html) +the running, monitoring, and reporting of these +tests, giving us the ability to push a commit and have it automatically +"verified" as cohesive with the existing codebase. Automating feedback, +will help provide more confidence to the engineers that their code isn't +breaking some other functionality in a codebase they are working on, +and should also help improve the ease of long-term maintainability +of the code. The more we can automate the menial tasks that have to +be repeated frequently, and are important for success of the project +but are not related to progressing the engineering of the Libre-SOC, +the more productive project members can be. + +To help continue to ease such administrative burdens on the engineers, +Cole is also working on a repository of setup automation scripts. The first +script is one that will replicate the setup of Jacob's build server, +so that others who want to contribute computational resources to the +project may do so easily. Cole is also working on a collection of modular +scripts to automate the setup of the development environment for the +HDL workflow and the layout of the SOC, including the installation of +development branches of a substainal number of very complex pieces of +software. This should help ease the process of onboarding new members +to the project, especially some interns that we have coming onboard in +the next few months to do the layout of the chip. These scripts will be +available via the git.libre-riscv.org repository dev-env-setup, at the +[following link](http://git.libre-riscv.org/?p=dev-env-setup.git) # Conclusion -TODO - +I'm not going to mention anything about the current world climate: you've +seen enough news reports. I will say (more about this through the +[EOMA68](https://www.crowdsupply.com/eoma68/micro-desktop) updates) that +I anticipated something like what is happening right now, over ten years +ago. I wasn't precisely expecting what *has* happened, just the consequences: +world-wide travel shut-down, and for people - the world over - to return to +local community roots. + +However what I definitely wasn't expecting was a United States President +to be voted in who was eager and willing to start *and escalate* a Trade +war with China, even during the current world climate where both local +and global collaboration, **not** competition, is more important than +ever before. The impact of his decisions on the U.S economy alone, and +the reputation of the whole country, has been detrimental in the extreme. + +This combination leaves us - world-wide - with the strong possibility that +seemed so "preposterous" that I could in no way discuss it widely, let alone +mention it on something like a Crowdsupply update, that thanks to the +business model on which their entire product lifecycle is predicated, +in combination with the extremely high NREs and development costs for +ASICs (custom silicon costs USD $100 million, these days), several +large Corporations producing proprietary binary-only drivers for +hardware on which we critically rely for our internet-connected way +of life **may soon go out of business**. + +Right at a critical time where video conferencing is taking off massively, +your proprietary hardware - your smartphone, your tablet, your laptop, +everything you rely on for connectivity to the rest of the world, all of +a sudden **you may not be able to get software updates** or, worse, +your products could even be +[remotely shut down](https://www.theguardian.com/technology/2016/apr/05/revolv-devices-bricked-google-nest-smart-home) +**without warning**. + +I do not want to hammer the point home too strongly but you should be +getting, in no uncertain terms, exactly how strategically critical, in +the current world climate, this project just became. We need to get it +accelerated, completed, and into production, in an expedited and responsible +fashion.