3 This section describes the workflow and some best practices for developing
4 the LibreSoC hardware. We use nmigen, yosys and symbiyosys, and this
5 page is intended not just to help you get set up, it is intended to
6 help advise you of some tricks and practices that will help you become
7 effective team contributors.
9 It is particularly important to bear in mind that we are not just
10 "developing code", here: we are creating a "lasting legacy educational
11 resource" for other people to learn from, and for businesses and students
12 alike to be able to use, learn from and augment for their own purposes.
14 It is also important to appreciate and respect that we are funded under
15 NLNet's Privacy and Enhanced Trust Programme <http://nlnet.nl/PET>. Full
16 transparency, readability, documentation, effective team communication
17 and formal mathematical proofs for all code at all levels is therefore
20 # Collaboration resources
22 ## Main contact method: mailing list
24 To respect the transparency requirements, conversations need to be
25 public and archived (i.e not skype, not telegram, not discord,
26 and anyone seriously suggesting slack will be thrown to the
27 lions). Therefore we have a mailing list. Everything goes through
28 there. <http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev>
29 therefore please do google "mailing list etiquette" and at the very
30 minimum look up and understand the following:
32 * This is a technical mailing list with complex topics. Top posting
33 is completely inappropriate. Don't do it unless you have mitigating
34 circumstances, and even then please apologise and explain ("hello sorry
35 using phone at airport flight soon, v. quick reply: ....")
36 * Always trim context but do not cut excessively to the point where people
37 cannot follow the discussion. Especially do not cut the attribution
38 ("On monday xxx wrote")
39 * Use inline replies i.e. reply at the point in the relevant part of
40 the conversation, as if you were actually having a conversation.
41 * Follow standard IETF reply formatting, using ">" for cascaded
42 indentation of other people's replies. If using gmail, please: SWITCH
43 OFF RICH TEXT EDITING.
44 * Please for god's sake do not use "my replies are in a different
45 colour". Only old and highly regarded people still using AOL are allowed
46 to get away with that (such as Mitch).
47 * Start a new topic with a relevant subject line. If an existing
48 discussion changes direction, change the subject line to reflect the
49 new topic (or start a new conversation entirely, without using the
51 * DMARC is a pain on the neck. Try to avoid GPG signed messages. sigh.
52 * Don't send massive attachments. Put them online (no, not on facebook or
53 google drive or anywhere else that demands privacy violations) and provide
54 the link. Which should not require any kind of login to access. ask the
55 listadmin if you don't have anywhere suitable: FTP access can be arranged.
57 If discussions result in any actionable items, it is important not to
58 lose track of them. Create a bugreport, find the discussion in the
59 archives <http://lists.libre-riscv.org/pipermail/libre-riscv-dev/>,
60 and put the link actually in the bugtracker as one of the comments.
62 At some point it may become better to use <http://bugs.libre-riscv.org>
63 itself to continue the discussion rather than to keep on dropping copies
64 of links into the bugtracker. The bugtracker sends copies of comments
65 *to* the list however this is 'one-way' (note from lkcl: because this
66 involves running an automated perl script from email, on every email,
67 on the server, that is a high security risk, and i'm not doing it. sorry.)
69 Also, please do not use the mailing list as an "information or document
70 store or poor-man's editor". We have the wiki for that. Edit a page and
71 tell people what you did (summarise rather than drop the entire contents
72 at the list) and include the link to the page.
74 Or, if it is more appropriate, commit a document (or source code)
75 into the relevant git repository then look up the link in the gitweb
76 source tree browser and post that (in the bugtracker or mailing list)
77 See <http://git.libre-riscv.org>
81 bugzilla. old and highly effective. sign up in the usual way. any
82 problems, ask on the list.
84 Please do not ask for the project to be transferred to github or other
85 proprietary nonfree service "because it's soooo convenient", as the
86 lions are getting wind and gout from overfeeding on that one.
90 Runs the main libre-riscv.org site (including this page). effective,
91 stunningly light on resources, and uses a git repository not a database.
92 That means it can be edited offline.
94 Usual deal: register an account and you can start editing and contributing
97 Hint: to create a new page, find a suitable page that would link to it,
98 first, then put the link in of the page you want to create, as if the
99 page already exists. Save that page, and you will find a questionmark
100 next to the new link you created. click that link, and it will fire up a
101 "create new page" editor.
103 Hint again: the wiki is backed by a git repository. Don't go overboard
104 but at the same time do not be afraid that you might "damage" or "lose"
105 pages. Although it would be a minor pain, the pages can always be
106 reverted or edited by the sysadmins to restore things if you get in a tiz.
108 Assistance in creating a much better theme greatly appreciated. e.g.
109 <http://www.math.cmu.edu/~gautam/sj/blog/20140720-ikiwiki-navbar.html>
113 we use git. more on this below. we also use gitolite3 running on a
114 dedicated server. again, it is extremely effective and low resource
115 utilisation. reminder: lions are involved if github is mentioned.
117 gitweb is provided which does a decent job. <http://git.libre-riscv.org>
121 as an aside: all this is "old school" and run on a single core 512MB
122 VM with only a 20GB HDD allocation. it costs only 8 GBP per month from
123 mythic-beasts and means that the project is in no way dependent on anyone
124 else - not microsoft, not google, not facebook, not amazon.
126 we tried gitlab. it didn't go well.
130 RAM is the biggest requirement. Minimum 16GB, the more the better (32
131 or 64GB starts to reach "acceptable" levels. Disk space is not hugely
132 critical: 256GB SSD should be more than adequate. Simulations and
133 FPGA compilations however are where raw processing power is a must.
134 High end Graphics Cards are nonessential.
136 What is particularly useful is to have hi-res screens (curved is
137 *strongly* recommended if the LCD is over 24in wide, to avoid eyeballs
138 going "prism" through longterm use), and to have several of them: the
139 more the better. Either a DisplayLink UD160A (or more modern variant)
140 or simply using a second machine (lower spec hardware because it will
141 run editors) is really effective.
143 Also it is really recommended to have a UHD monitor (4k - 3840x2160),
144 or at least 2560x1200. If given a choice, 4:3 aspect ratio is better
145 than 16:9 particularly when using several of them. However, caveat
146 (details below): please when editing do not assume that everyone will
147 have access to such high resolution screens.
151 First install and become familiar with Debian (ubuntu if you absolutely
152 must) for standardisation cross-team and so that toolchain installation
153 is greatly simplified. yosys in particular warns that trying to use
154 Windows, BSD or MacOS will get you into a world of pain.
156 Only a basic GUI desktop is necessary: fvwm2, xfce4, lxde are perfectly
157 sufficient (alongside wicd-gtk for network management). Other more
158 complex desktops can be used however may consume greater resources.
160 # editors and editing
162 Whilst this is often a personal choice, the fact that many editors are
163 GUI based and run fullscreen with the entire right hand side *and* middle
164 *and* the majority of the left side of the hi-res screen entirely unused
165 and bereft of text leaves experienced developers both amused and puzzled.
167 At the point where such fullscreen users commit code with line lengths
168 well over 160 characters, that amusement quickly evaporates.
170 Where the problems occur with fullscreen editor usage is when a project
171 is split into dozens if not hundreds of small files (as this one is). At
172 that point it becomes pretty much essential to have as many as six to
173 eight files open *and on-screen* at once, without overlaps i.e. not in
174 hidden tabs, next to at least two if not three additional free and clear
175 terminals into which commands are regularly and routinely typed (make,
176 git commit, nosetests3 etc). Illustrated with the following 3840x2160
177 screenshot (click to view full image):
179 [[!img 2020-01-24_11-56.png size=640x ]]
181 (hint/tip: fvwm2 set up with "mouse-over to raise focus, rather than
182 additionally requiring a mouseclick, can save a huge amount of cumulative
183 development time here, switching between editor terminal(s) and the
186 Once this becomes necessary, it it turn implies that having greater
187 than 80 chars per line - and running editors fullscreen - is a severe
188 hindance to an essential *and highly effective* workflow technique.
190 Additionally, care should be taken to respect that not everyone will have
191 200+ column editor windows and the eyesight of a hawk. They may only have
192 a 1280 x 800 laptop which barely fits two 80x53 xterms side by side.
193 Consequently, having excessively long functions is also a hindrance to
194 others, as such developers with limited screen resources would need to
195 continuously page-up and page-down to read the code even of a single
198 This helps explain in part, below, why compliance with pep8 is enforced,
199 including its 80 character limit. In short: not everyone has the same
200 "modern" GUI workflow or has access to the same computing resources as
201 you, so please do respect that.
203 # Software prerequisites
205 Whilst many resources online advocate "sudo" in front of all root-level
206 commands below, this quickly becomes tiresome. run "sudo bash", get a
207 root prompt, and save yourself some typing.
210 * apt-get install vim exuberant-ctags
211 * apt-get install build-essential
212 * apt-get install git python3.7 python3.7-dev python-nosetest3
213 * apt-get install graphviz xdot gtkwave
214 * return to user prompt (ctrl-d)
216 This will get you python3 and other tools that are needed. graphviz is
217 essential for showing the interconnections between cells, and gtkwave
218 is essential for debugging.
222 Look up good tutorials on how to use git effectively. There are so many
223 it is hard to recommend one. This is however essential. If you are not
224 comfortable with git, and you let things stay that way, it will seriously
225 impede development progress.
227 If working all day you should expect to be making at least two commits per
228 hour, so should become familiar with it very quickly. If you are *not*
229 doing around 2 commits per hour, something is wrong and you should read
230 the workflow instructions below more carefully, and also ask for advice
233 Worth noting: *this project does not use branches*. All code is committed
234 to master and we *require* that it be either zero-impact additions or that
235 relevant unit tests pass 100%. This ensures that people's work does not
236 get "lost" or isolated and out of touch due to major branch diversion,
237 and that people communicate and coordinate with each other.
241 Follow the source code (git clone) instructions here:
242 <http://www.clifford.at/yosys/download.html>
244 Do not try to use a fixed revision (currently 0.9), nmigen is evolving
245 and frequently interacts with yosys
249 Follow the instructions here:
250 <https://symbiyosys.readthedocs.io/en/latest/quickstart.html#installing>
252 You do not have to install all of those (avy, boolector can be left
253 out if desired) however the more that are installed the more effective
254 the formal proof scripts will be (less resource utilisation in certain
259 nmigen may be installed as follows:
263 * git clone https://github.com/m-labs/nmigen.git
266 * python3 setup.py develop
269 testing can then be carried out with "python3 setup.py test"
271 ## Softfloat and sfpy
273 These are a test suite dependency for the ieee754fpu library, and
274 will be changed in the future to use Jacob's algorithmic numeric
275 library. In the meantime the README describing the process is here:
276 <https://git.libre-riscv.org/?p=ieee754fpu.git;a=blob;f=README.md;h=d219864a341e4b656680de476e385b6a7f70fb9b;hb=HEAD>
278 # Registering for git repository access
280 After going through the onboarding process and having agreed to take
281 responsibility for certain tasks, ask on the mailing list for git
282 repository access, sending in a public key (id_rsa.pub). If you do
283 not have one then generate it with ssh-keygen -t rsa. You will find it
286 NEVER SEND ANYONE THE PRIVATE KEY. By contrast the public key, on
287 account of being public, is perfectly fine to make... err... public.
289 Create a file ~/.ssh/config with the following lines:
291 Host git.libre-riscv.org
294 Wait for the Project Admin to confirm that the ssh key has been added
295 to the required repositories. Once confirmed, you can clone any of the
296 repos at http://git.libre-riscv.org:
298 git clone gitolite3@git.libre-riscv.org:REPONAME.git
300 # Checking out the HDL repositories
304 * git clone gitolite3@git.libre-riscv.org:soc.git
305 * git clone gitolite3@git.libre-riscv.org:ieee754fpu.git
307 In each of these directories, track down the setup.py file, then, as root
308 (sudo bash) run the following:
310 * python3 setup.py develop
312 The reason for using "develop" mode is that the code may be edited
313 in-place yet still imported "globally". There are variants on this theme
314 for multi-user machine use however it is often just easier to get your
315 own machine these days.
317 If "python3 setup.py install" is used it is a pain: edit, then
318 install. edit, then install. It gets extremely tedious, hence why
319 "develop" was created.
325 * communicate on the mailing list or the bugtracker an intent to take
326 responsibility for a particular task.
327 * assign yourself as the bug's owner
328 * *keep in touch* about what you are doing, and why you are doing it.
329 * if you cannot do something that you have taken responsibility for,
330 then unless it is a dire personal emergency please say so, on-list. we
331 won't mind. we'll help sort it out.
333 regarding the above it is important that you read, understand, and agree
334 to the [[charter]] because the charter is about ensuring that we operate
335 as an effective organisation. It's *not* about "setting rules and meting
338 for actual code development:
340 * plan in advance to write not just code but a full test suite for
341 that code. **this is not optional**. large python projects that do not
342 have unit tests **FAIL** (see separate section below).
343 * only commit code that has been tested (or is presently unused). other
344 people will be depending on you, so do take care not to screw up.
345 not least because, as it says in the [[charter]] it will be your
346 responsibility to fix. that said, do not feel intimidated: ask for help
347 and advice, and you'll get it straight away.
348 * commit often. several times a day, and "git push" it. this is
349 collaboration. if something is left even overnight uncommitted and not
350 pushed so that other people can see it, it is a red flag. if you find
351 yourself thinking "i'll commit it when it's finished" or "i don't want to
352 commit something that people might criticise" *this is not collaboration*,
353 it is making yourself a bottleneck. pair-programming is supposed to help
354 avoid this kind of thing however pair-programming is difficult to organise
355 for remote collaborative libre projects (suggestions welcomed here)
356 * **do not commit autogenerated output**. write a shell script and commit
357 that, or add a Makefile to run the command that generates the output, but
358 **do not** add the actual output of **any** command to the repository.
359 ever. this is really important. even if it is a human-readable file
360 rather than a binary object file.
361 * if the command needed to create any given autogenerated output is not
362 currently in the list of known project dependencies, first consult on
363 the list if it is okay to make that command become a hard dependency of
364 the project (hint: java, node.js php and .NET commands may cause delays
365 in response time due to other list participants laughing hysterically),
366 and after a decision is made, document the dependency and how its source
367 code is obtained and built (hence why it has to be discussed carefully)
368 * if you find yourself repeating commands regularly, chances are high
369 that someone else will need to run them, too. clearly this includes
370 yourself, therefore, to make everyone's lives easier including your own,
371 put them into a .sh shell script (and/or a Makefile), commit them to
372 the repository and document them at the very minimum in the README,
373 INSTALL.txt or somewhere in a docs folder as appropriate. if unsure,
374 ask on the mailing list for advice.
375 * edit files making minimal *single purpose* modifications (even if
376 it involves multiple files. Good extreme example: globally changing
377 a function name across an entire codebase is one purpose, one commit,
378 yet hundreds of files. miss out one of those files, requiring multiple
379 commits, and it actually becomes a nuisance).
380 * prior to committing make sure that relevant unit tests pass, or that
381 the change is a zero-impact addition (no unit tests fail at the minimum)
382 * commit no more than around 5 to 10 lines at a time, with a CLEAR message
383 (no "added this" or "changed that").
384 * if as you write you find that the commit message involves a *list* of
385 changes or the word "and", then STOP. do not proceed: it is a "red flag"
386 that the commit has not been properly broken down into separate-purpose
387 commits. ask for advice on-list on how to proceed.
388 * if it is essential to commit large amounts of code, ensure that it
389 is **not** in use **anywhere** by any other code. then make a *small*
390 (single purpose) followup commit which actually puts that code into use.
392 this last rule is kinda flexible, because if you add the code *and* add
393 the unit test *and* added it into the main code *and* ran all relevant
394 unit tests on all cascade-impacted areas by that change, that's perfectly
395 fine too. however if it is the end of a day, and you need to stop and
396 do not have time to run the necessary unit tests, do *not* commit the
397 change which integrates untested code: just commit the new code (only)
398 and follow up the next day *after* running the full relevant unit tests.
400 the reason for all the above is because python is a weakly typed language.
401 make one tiny change at the base level of the class hierarchy and the
402 effect may be disastrous.
404 it is therefore worth reiterating: make absolutely certain that you *only*
405 commit working code or zero-impact code.
407 therefore, if you are absolutely certain that a new addition (new file,
408 new class, new function) is not going to have any side-effects, committing
409 it (a large amount of code) is perfectly fine.
411 as a general rule, however, do not use this an an excuse to write code
412 first then write unit tests as an afterthought. write *less* code *in
413 conjunction* with its (more basic) unit tests, instead.
415 the reason for separating out commits to single purpose only becomes
416 obvious (and regretted if not followed) when, months later, a mistake
417 has to be tracked down and reverted. if the commit does not have an
418 easy-to-find message, it cannot even be located, and once found, if the
419 commit confuses several unrelated changes, not only the diff is larger
420 than it should be, the reversion process becomes extremely painful.
422 * all code needs to conform to pep8. use either pep8checker or better
423 run autopep8. however whenever committing whitespace changes, *make a
424 separate commit* with a commit message "whitespace" or "autopep8 cleanup".
425 * pep8 REQUIRES no more than 80 chars per line. this is non-negotiable. if
426 you think you need greater than 80 chars, it *fundamentally* indicates
427 poor code design. split the code down further into smaller classes
429 * TBD there is a docstring checker. at the minimum make sure to have
430 an SPD license header, module header docstring, class docstring and
431 function docstrings on at least non-obvious functions.
432 * make liberal but not excessive use of comments. describe a group of
433 lines of code, with terse but useful comments describing the purpose,
434 documenting any side-effects, and anything that could trip you or other
435 developers up. unusual coding techniques should *definitely* contain
437 * unless they are very closely related, only have one module (one class)
438 per file. a file only 25 lines long including imports and docstrings
439 is perfectly fine however don't force yourself. again, if unsure,
441 * *keep files short and simple*. see below as to why
442 * create a decent directory hierarchy but do not go mad. ask for advice
444 * please do not use "from module import \*". it is extremely bad practice,
445 causes unnecessary resource utilisation, makes code readability and
446 tracking extremely difficult, and results in unintended side-effects.
447 * try to keep both filenames and variable names short but not ridiculously
448 obtuse. an interesting compromise on imports is "from ridiculousfilename
449 import longsillyname as lsn", and to assign variables as well: "comb =
450 m.d.comb" followed by multiple "comb += nmigen_stmt" lines is a good trick
451 that can reduce code indentation by 6 characters without reducing clarity.
453 regarding code structure: we decided to go with small modules that are
454 both easy to analyse, as well as fit onto a single page and be readable
455 when displayed as a visual graph on a full UHD monitor. this is done
458 * using the capability of nmigen (TODO crossref to example) output the
459 module to a yosys ilang (.il) file
460 * in a separate terminal window, run yosys
461 * at the yosys prompt type "read_ilang modulename.il"
462 * type "show top" and a graphviz window should appear. note that typing
463 show, then space, then pressing the tab key twice will give a full list
464 of submodules (one of which will be "top")
466 you can now fullsize the graphviz window and scroll around. if it looks
467 reasonably obvious at 100% zoom, i.e the connections can be clearly
468 related in your mind back to the actual code (by matching the graph names
469 against signals and modules in the original nmigen code) and the words are
470 not tiny when zoomed out, and connections are not total incomprehensible
471 spaghetti, then congratulations, you have well-designed code. If not,
472 then this indicates a need to split the code further into submodules
473 and do a bit more work.
475 The reasons for doing a proper modularisation job are several-fold:
477 * firstly, we will not be doing a full automated layout-and-hope
478 using alliance/coriolis2, we will be doing leaf-node thru tree node
479 half-automated half-manual layout, finally getting to the floorplan,
480 then revising and iteratively adjusting.
481 * secondly, examining modules at the gate level (or close to it) is just
482 good practice. poor design creeps in by *not* knowing what the tools
483 are actually doing (word to experienced developers: yes, we know that
484 the yosys graph != final netlist).
485 * thirdly, unit testing, particularly formal proofs, is far easier on
486 small sections of code, and complete in a reasonable time.
490 This deserves its own special section. It is extremely important to
491 appreciate that without unit tests, python projects are simply unviable.
492 Python itself has over 25,000 individual tests.
494 This can be quite overwhelming to a beginner developer, especially one
495 used to writing scripts of only 100 lines in length.
497 Thanks to Samuel Falvo we learned that writing unit tests as a formal
498 proof is not only shorter, it's also far more readable and also, if
499 written properly, provides 100% coverage of corner-cases that would
500 otherwise be overlooked or require tens to hundreds of thousands of
503 No this is not a joke or even remotely hypothetical, this is an actual
506 The ieee754fpu requires several hundreds of thousands of tests to be
507 run (currently needing several days to run them all), and even then we
508 cannot be absolutely certain that all possible combinations of input have
509 been tested. With 2^128 permutations to try with 2 64 bit FP numbers
510 it is simply impossible to even try.
512 This is where formal proofs come into play.
514 Samuel illustrated to us that "ordinary" unit tests can then be written
515 to *augment* the formal ones, serving the purpose of illustrating how
516 to use the module, more than anything.
518 However it is appreciated that writing formal proofs is a bit of a
519 black art. This is where team collaboration particularly kicks in,
520 so if you need help, ask on the mailing list.
524 Find appropriate tutorials for nmigen and yosys, as well as symbiyosys.
526 * Although a verilog example this is very useful to do
527 <https://symbiyosys.readthedocs.io/en/latest/quickstart.html#first-step-a-simple-bmc-example>
528 * This tutorial looks pretty good and will get you started
529 <http://blog.lambdaconcept.com/doku.php?id=nmigen:nmigen_install> and
530 walks not just through simulation, it takes you through using gtkwave
532 * There exist several nmigen examples which are also executable
533 <https://github.com/m-labs/nmigen/tree/master/examples/> exactly as
534 described in the above tutorial (python3 filename.py -h)