# HDL workflow
-This section describes the workflow for developing the LibreSoC.
+This section describes the workflow and some best practices for developing
+the LibreSoC hardware. We use nmigen, yosys and symbiyosys, and this
+page is intended not just to help you get set up, it is intended to
+help advise you of some tricks and practices that will help you become
+effective team contributors.
+
+It is particularly important to bear in mind that we are not just
+"developing code", here: we are creating a "lasting legacy educational
+resource" for other people to learn from, and for businesses and students
+alike to be able to use, learn from and augment for their own purposes.
+
+It is also important to appreciate and respect that we are funded under
+NLNet's Privacy and Enhanced Trust Programme <http://nlnet.nl/PET>. Full
+transparency, readability, documentation, effective team communication
+and formal mathematical proofs for all code at all levels is therefore
+paramount.
+
+# Collaboration resources
+
+## Main contact method: mailing list
+
+To respect the transparency requirements, conversations need to be
+public and archived (i.e not skype, not telegram, not discord,
+and anyone seriously suggesting slack will be thrown to the
+lions). Therefore we have a mailing list. Everything goes through
+there. <http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev>
+therefore please do google "mailing list etiquette" and at the very
+minimum look up and understand the following:
+
+* This is a technical mailing list with complex topics. Top posting
+ is completely inappropriate. Don't do it unless you have mitigating
+ circumstances, and even then please apologise and explain ("hello sorry
+ using phone at airport flight soon, v. quick reply: ....")
+* Always trim context but do not cut excessively to the point where people
+ cannot follow the discussion. Especially do not cut the attribution
+ ("On monday xxx wrote")
+* Use inline replies i.e. reply at the point in the relevant part of
+ the conversation, as if you were actually having a conversation.
+* Follow standard IETF reply formatting, using ">" for cascaded
+ indentation of other people's replies. If using gmail, please: SWITCH
+ OFF RICH TEXT EDITING.
+* Please for god's sake do not use "my replies are in a different
+ colour". Only old and highly regarded people still using AOL are allowed
+ to get away with that (such as Mitch).
+* Start a new topic with a relevant subject line. If an existing
+ discussion changes direction, change the subject line to reflect the
+ new topic (or start a new conversation entirely, without using the
+ "reply" button)
+* DMARC is a pain on the neck. Try to avoid GPG signed messages. sigh.
+* Don't send massive attachments. Put them online (no, not on facebook or
+ google drive or anywhere else that demands privacy violations) and provide
+ the link. Which should not require any kind of login to access. ask the
+ listadmin if you don't have anywhere suitable: FTP access can be arranged.
+
+If discussions result in any actionable items, it is important not to
+lose track of them. Create a bugreport, find the discussion in the
+archives <http://lists.libre-riscv.org/pipermail/libre-riscv-dev/>,
+and put the link actually in the bugtracker as one of the comments.
+
+At some point it may become better to use <http://bugs.libre-riscv.org>
+itself to continue the discussion rather than to keep on dropping copies
+of links into the bugtracker. The bugtracker sends copies of comments
+*to* the list however this is 'one-way' (note from lkcl: because this
+involves running an automated perl script from email, on every email,
+on the server, that is a high security risk, and i'm not doing it. sorry.)
+
+Also, please do not use the mailing list as an "information or document
+store or poor-man's editor". We have the wiki for that. Edit a page and
+tell people what you did (summarise rather than drop the entire contents
+at the list) and include the link to the page.
+
+Or, if it is more appropriate, commit a document (or source code)
+into the relevant git repository then look up the link in the gitweb
+source tree browser and post that (in the bugtracker or mailing list)
+See <http://git.libre-riscv.org>
+
+## Bugtracker
+
+bugzilla. old and highly effective. sign up in the usual way. any
+problems, ask on the list.
+
+Please do not ask for the project to be transferred to github or other
+proprietary nonfree service "because it's soooo convenient", as the
+lions are getting wind and gout from overfeeding on that one.
+
+## ikiwiki
+
+Runs the main libre-riscv.org site (including this page). effective,
+stunningly light on resources, and uses a git repository not a database.
+That means it can be edited offline.
+
+Usual deal: register an account and you can start editing and contributing
+straight away.
+
+Hint: to create a new page, find a suitable page that would link to it,
+first, then put the link in of the page you want to create, as if the
+page already exists. Save that page, and you will find a questionmark
+next to the new link you created. click that link, and it will fire up a
+"create new page" editor.
+
+Hint again: the wiki is backed by a git repository. Don't go overboard
+but at the same time do not be afraid that you might "damage" or "lose"
+pages. Although it would be a minor pain, the pages can always be
+reverted or edited by the sysadmins to restore things if you get in a tiz.
+
+Assistance in creating a much better theme greatly appreciated. e.g.
+<http://www.math.cmu.edu/~gautam/sj/blog/20140720-ikiwiki-navbar.html>
+
+## git
+
+we use git. more on this below. we also use gitolite3 running on a
+dedicated server. again, it is extremely effective and low resource
+utilisation. reminder: lions are involved if github is mentioned.
+
+gitweb is provided which does a decent job. <http://git.libre-riscv.org>
+
+## server
+
+as an aside: all this is "old school" and run on a single core 512MB
+VM with only a 20GB HDD allocation. it costs only 8 GBP per month from
+mythic-beasts and means that the project is in no way dependent on anyone
+else - not microsoft, not google, not facebook, not amazon.
+
+we tried gitlab. it didn't go well.
# Hardware
-RAM is the biggest requirement. Minimum 16GB, the more the better. Disk space is not hugely critical: 256GB SSD should be more than adequate. Simulations however are where raw processing power is a must. High end Graphics Cards are nonessential.
+RAM is the biggest requirement. Minimum 16GB, the more the better (32
+or 64GB starts to reach "acceptable" levels. Disk space is not hugely
+critical: 256GB SSD should be more than adequate. Simulations and
+FPGA compilations however are where raw processing power is a must.
+High end Graphics Cards are nonessential.
+
+What is particularly useful is to have hi-res screens (curved is
+*strongly* recommended if the LCD is over 24in wide, to avoid eyeballs
+going "prism" through longterm use), and to have several of them: the
+more the better. Either a DisplayLink UD160A (or more modern variant)
+or simply using a second machine (lower spec hardware because it will
+run editors) is really effective.
+
+Also it is really recommended to have a UHD monitor (4k - 3840x2160),
+or at least 2560x1200. If given a choice, 4:3 aspect ratio is better
+than 16:9 particularly when using several of them. However, caveat
+(details below): please when editing do not assume that everyone will
+have access to such high resolution screens.
# Operating System
-First install Debian (ubuntu if you absolutely must) for standardisation cross-team and so that toochain installation is greatly simplified.
+First install and become familiar with Debian (ubuntu if you absolutely
+must) for standardisation cross-team and so that toolchain installation
+is greatly simplified. yosys in particular warns that trying to use
+Windows, BSD or MacOS will get you into a world of pain.
+
+Only a basic GUI desktop is necessary: fvwm2, xfce4, lxde are perfectly
+sufficient (alongside wicd-gtk for network management). Other more
+complex desktops can be used however may consume greater resources.
+
+# editors and editing
+
+Whilst this is often a personal choice, the fact that many editors are
+GUI based and run fullscreen with the entire right hand side *and* middle
+*and* the majority of the left side of the hi-res screen entirely unused
+and bereft of text leaves experienced developers both amused and puzzled.
+
+At the point where such fullscreen users commit code with line lengths
+well over 160 characters, that amusement quickly evaporates.
+
+Where the problems occur with fullscreen editor usage is when a project
+is split into dozens if not hundreds of small files (as this one is). At
+that point it becomes pretty much essential to have as many as six to
+eight files open *and on-screen* at once, without overlaps i.e. not in
+hidden tabs, next to at least two if not three additional free and clear
+terminals into which commands are regularly and routinely typed (make,
+git commit, nosetests3 etc). Illustrated with the following 3840x2160
+screenshot (click to view full image), where *every one* of those 80x70
+xterm windows is *relevant to the task at hand*.
+
+[[!img 2020-01-24_11-56.png size=640x ]]
+
+(hint/tip: fvwm2 set up with "mouse-over to raise focus, rather than
+additionally requiring a mouseclick, can save a huge amount of cumulative
+development time here, switching between editor terminal(s) and the
+command terminals).
-A basic GUI desktop is necessary: fvwm2, xfce, lxde are perfectly sufficient (alongside wicd-gtk for network management). Other more complex desktops can be used however may consume greater resources.
+Once this becomes necessary, it it turn implies that having greater
+than 80 chars per line - and running editors fullscreen - is a severe
+hindance to an essential *and highly effective* workflow technique.
+
+Additionally, care should be taken to respect that not everyone will have
+200+ column editor windows and the eyesight of a hawk. They may only have
+a 1280 x 800 laptop which barely fits two 80x53 xterms side by side.
+Consequently, having excessively long functions is also a hindrance to
+others, as such developers with limited screen resources would need to
+continuously page-up and page-down to read the code even of a single
+function, in full.
+
+This helps explain in part, below, why compliance with pep8 is enforced,
+including its 80 character limit. In short: not everyone has the same
+"modern" GUI workflow or has access to the same computing resources as
+you, so please do respect that.
# Software prerequisites
-Whilst many resources advocate sudo in front of all of the root-level command below, this is extremely irritating. run "sudo bash", get a root prompt, and save some typing.
+Whilst many resources online advocate "sudo" in front of all root-level
+commands below, this quickly becomes tiresome. run "sudo bash", get a
+root prompt, and save yourself some typing.
+* sudo bash
+* apt-get install vim exuberant-ctags
* apt-get install build-essential
* apt-get install git python3.7 python3.7-dev python-nosetest3
-* apt-get install graphviz xdot
+* apt-get install graphviz xdot gtkwave
* return to user prompt (ctrl-d)
+This will get you python3 and other tools that are needed. graphviz is
+essential for showing the interconnections between cells, and gtkwave
+is essential for debugging.
+
+## git
+
+Look up good tutorials on how to use git effectively. There are so many
+it is hard to recommend one. This is however essential. If you are not
+comfortable with git, and you let things stay that way, it will seriously
+impede development progress.
+
+If working all day you should expect to be making at least two commits per
+hour, so should become familiar with it very quickly. If you are *not*
+doing around 2 commits per hour, something is wrong and you should read
+the workflow instructions below more carefully, and also ask for advice
+on the mailing list.
+
+Worth noting: *this project does not use branches*. All code is committed
+to master and we *require* that it be either zero-impact additions or that
+relevant unit tests pass 100%. This ensures that people's work does not
+get "lost" or isolated and out of touch due to major branch diversion,
+and that people communicate and coordinate with each other.
+
## yosys
-Follow the source code (git clone) instructions here: <http://www.clifford.at/yosys/download.html>
+Follow the source code (git clone) instructions here:
+<http://www.clifford.at/yosys/download.html>
-Do not try to use a fixed revision, nmigen is evolving and frequently interacts with yosys
+Do not try to use a fixed revision (currently 0.9), nmigen is evolving
+and frequently interacts with yosys
## symbiyosys
-Follow the instructions here: <https://symbiyosys.readthedocs.io/en/latest/quickstart.html#installing>
+Follow the instructions here:
+<https://symbiyosys.readthedocs.io/en/latest/quickstart.html#installing>
-You do not have to install all of those (avy, boolector can be left out if desired).
+You do not have to install all of those (avy, boolector can be left
+out if desired) however the more that are installed the more effective
+the formal proof scripts will be (less resource utilisation in certain
+circumstances).
## nmigen
* python3 setup.py develop
* ctrl-d
-testing can then be carried out with "python3 setp.py test"
+testing can then be carried out with "python3 setup.py test"
+
+## Softfloat and sfpy
+
+These are a test suite dependency for the ieee754fpu library, and
+will be changed in the future to use Jacob's algorithmic numeric
+library. In the meantime the README describing the process is here:
+<https://git.libre-riscv.org/?p=ieee754fpu.git;a=blob;f=README.md;h=d219864a341e4b656680de476e385b6a7f70fb9b;hb=HEAD>
+
+# Registering for git repository access
+
+After going through the onboarding process and having agreed to take
+responsibility for certain tasks, ask on the mailing list for git
+repository access, sending in a public key (id_rsa.pub). If you do
+not have one then generate it with ssh-keygen -t rsa. You will find it
+in ~/.ssh
+
+NEVER SEND ANYONE THE PRIVATE KEY. By contrast the public key, on
+account of being public, is perfectly fine to make... err... public.
+
+Create a file ~/.ssh/config with the following lines:
+
+ Host git.libre-riscv.org
+ Port 922
+
+Wait for the Project Admin to confirm that the ssh key has been added
+to the required repositories. Once confirmed, you can clone any of the
+repos at http://git.libre-riscv.org:
+
+ git clone gitolite3@git.libre-riscv.org:REPONAME.git
+
+# Checking out the HDL repositories
+
+* mkdir ~/src
+* cd !$
+* git clone gitolite3@git.libre-riscv.org:soc.git
+* git clone gitolite3@git.libre-riscv.org:ieee754fpu.git
+
+In each of these directories, track down the setup.py file, then, as root
+(sudo bash) run the following:
+
+* python3 setup.py develop
+
+The reason for using "develop" mode is that the code may be edited
+in-place yet still imported "globally". There are variants on this theme
+for multi-user machine use however it is often just easier to get your
+own machine these days.
+
+If "python3 setup.py install" is used it is a pain: edit, then
+install. edit, then install. It gets extremely tedious, hence why
+"develop" was created.
+
+# Development Rules
+
+team communication:
+
+* communicate on the mailing list or the bugtracker an intent to take
+ responsibility for a particular task.
+* assign yourself as the bug's owner
+* *keep in touch* about what you are doing, and why you are doing it.
+* if you cannot do something that you have taken responsibility for,
+ then unless it is a dire personal emergency please say so, on-list. we
+ won't mind. we'll help sort it out.
+
+regarding the above it is important that you read, understand, and agree
+to the [[charter]] because the charter is about ensuring that we operate
+as an effective organisation. It's *not* about "setting rules and meting
+out punishment".
+
+for actual code development:
+
+* plan in advance to write not just code but a full test suite for
+ that code. **this is not optional**. large python projects that do not
+ have unit tests **FAIL** (see separate section below).
+* only commit code that has been tested (or is presently unused). other
+ people will be depending on you, so do take care not to screw up.
+ not least because, as it says in the [[charter]] it will be your
+ responsibility to fix. that said, do not feel intimidated: ask for help
+ and advice, and you'll get it straight away.
+* commit often. several times a day, and "git push" it. this is
+ collaboration. if something is left even overnight uncommitted and not
+ pushed so that other people can see it, it is a red flag. if you find
+ yourself thinking "i'll commit it when it's finished" or "i don't want to
+ commit something that people might criticise" *this is not collaboration*,
+ it is making yourself a bottleneck. pair-programming is supposed to help
+ avoid this kind of thing however pair-programming is difficult to organise
+ for remote collaborative libre projects (suggestions welcomed here)
+* **do not commit autogenerated output**. write a shell script and commit
+ that, or add a Makefile to run the command that generates the output, but
+ **do not** add the actual output of **any** command to the repository.
+ ever. this is really important. even if it is a human-readable file
+ rather than a binary object file.
+* if the command needed to create any given autogenerated output is not
+ currently in the list of known project dependencies, first consult on
+ the list if it is okay to make that command become a hard dependency of
+ the project (hint: java, node.js php and .NET commands may cause delays
+ in response time due to other list participants laughing hysterically),
+ and after a decision is made, document the dependency and how its source
+ code is obtained and built (hence why it has to be discussed carefully)
+* if you find yourself repeating commands regularly, chances are high
+ that someone else will need to run them, too. clearly this includes
+ yourself, therefore, to make everyone's lives easier including your own,
+ put them into a .sh shell script (and/or a Makefile), commit them to
+ the repository and document them at the very minimum in the README,
+ INSTALL.txt or somewhere in a docs folder as appropriate. if unsure,
+ ask on the mailing list for advice.
+* edit files making minimal *single purpose* modifications (even if
+ it involves multiple files. Good extreme example: globally changing
+ a function name across an entire codebase is one purpose, one commit,
+ yet hundreds of files. miss out one of those files, requiring multiple
+ commits, and it actually becomes a nuisance).
+* prior to committing make sure that relevant unit tests pass, or that
+ the change is a zero-impact addition (no unit tests fail at the minimum)
+* commit no more than around 5 to 10 lines at a time, with a CLEAR message
+ (no "added this" or "changed that").
+* if as you write you find that the commit message involves a *list* of
+ changes or the word "and", then STOP. do not proceed: it is a "red flag"
+ that the commit has not been properly broken down into separate-purpose
+ commits. ask for advice on-list on how to proceed.
+* if it is essential to commit large amounts of code, ensure that it
+ is **not** in use **anywhere** by any other code. then make a *small*
+ (single purpose) followup commit which actually puts that code into use.
+
+this last rule is kinda flexible, because if you add the code *and* add
+the unit test *and* added it into the main code *and* ran all relevant
+unit tests on all cascade-impacted areas by that change, that's perfectly
+fine too. however if it is the end of a day, and you need to stop and
+do not have time to run the necessary unit tests, do *not* commit the
+change which integrates untested code: just commit the new code (only)
+and follow up the next day *after* running the full relevant unit tests.
+
+the reason for all the above is because python is a weakly typed language.
+make one tiny change at the base level of the class hierarchy and the
+effect may be disastrous.
+
+it is therefore worth reiterating: make absolutely certain that you *only*
+commit working code or zero-impact code.
+
+therefore, if you are absolutely certain that a new addition (new file,
+new class, new function) is not going to have any side-effects, committing
+it (a large amount of code) is perfectly fine.
+
+as a general rule, however, do not use this an an excuse to write code
+first then write unit tests as an afterthought. write *less* code *in
+conjunction* with its (more basic) unit tests, instead.
+
+the reason for separating out commits to single purpose only becomes
+obvious (and regretted if not followed) when, months later, a mistake
+has to be tracked down and reverted. if the commit does not have an
+easy-to-find message, it cannot even be located, and once found, if the
+commit confuses several unrelated changes, not only the diff is larger
+than it should be, the reversion process becomes extremely painful.
+
+* all code needs to conform to pep8. use either pep8checker or better
+ run autopep8. however whenever committing whitespace changes, *make a
+ separate commit* with a commit message "whitespace" or "autopep8 cleanup".
+* pep8 REQUIRES no more than 80 chars per line. this is non-negotiable. if
+ you think you need greater than 80 chars, it *fundamentally* indicates
+ poor code design. split the code down further into smaller classes
+ and functions.
+* TBD there is a docstring checker. at the minimum make sure to have
+ an SPD license header, module header docstring, class docstring and
+ function docstrings on at least non-obvious functions.
+* make liberal but not excessive use of comments. describe a group of
+ lines of code, with terse but useful comments describing the purpose,
+ documenting any side-effects, and anything that could trip you or other
+ developers up. unusual coding techniques should *definitely* contain
+ a warning.
+* unless they are very closely related, only have one module (one class)
+ per file. a file only 25 lines long including imports and docstrings
+ is perfectly fine however don't force yourself. again, if unsure,
+ ask on-list.
+* *keep files short and simple*. see below as to why
+* create a decent directory hierarchy but do not go mad. ask for advice
+ if unsure
+* please do not use "from module import \*". it is extremely bad practice,
+ causes unnecessary resource utilisation, makes code readability and
+ tracking extremely difficult, and results in unintended side-effects.
+* try to keep both filenames and variable names short but not ridiculously
+ obtuse. an interesting compromise on imports is "from ridiculousfilename
+ import longsillyname as lsn", and to assign variables as well: "comb =
+ m.d.comb" followed by multiple "comb += nmigen_stmt" lines is a good trick
+ that can reduce code indentation by 6 characters without reducing clarity.
+
+regarding code structure: we decided to go with small modules that are
+both easy to analyse, as well as fit onto a single page and be readable
+when displayed as a visual graph on a full UHD monitor. this is done
+as follows:
+
+* using the capability of nmigen (TODO crossref to example) output the
+ module to a yosys ilang (.il) file
+* in a separate terminal window, run yosys
+* at the yosys prompt type "read_ilang modulename.il"
+* type "show top" and a graphviz window should appear. note that typing
+ show, then space, then pressing the tab key twice will give a full list
+ of submodules (one of which will be "top")
+
+you can now fullsize the graphviz window and scroll around. if it looks
+reasonably obvious at 100% zoom, i.e the connections can be clearly
+related in your mind back to the actual code (by matching the graph names
+against signals and modules in the original nmigen code) and the words are
+not tiny when zoomed out, and connections are not total incomprehensible
+spaghetti, then congratulations, you have well-designed code. If not,
+then this indicates a need to split the code further into submodules
+and do a bit more work.
+
+The reasons for doing a proper modularisation job are several-fold:
+
+* firstly, we will not be doing a full automated layout-and-hope
+ using alliance/coriolis2, we will be doing leaf-node thru tree node
+ half-automated half-manual layout, finally getting to the floorplan,
+ then revising and iteratively adjusting.
+* secondly, examining modules at the gate level (or close to it) is just
+ good practice. poor design creeps in by *not* knowing what the tools
+ are actually doing (word to experienced developers: yes, we know that
+ the yosys graph != final netlist).
+* thirdly, unit testing, particularly formal proofs, is far easier on
+ small sections of code, and complete in a reasonable time.
+
+## Unit tests
+
+This deserves its own special section. It is extremely important to
+appreciate that without unit tests, python projects are simply unviable.
+Python itself has over 25,000 individual tests.
+
+This can be quite overwhelming to a beginner developer, especially one
+used to writing scripts of only 100 lines in length.
+
+Thanks to Samuel Falvo we learned that writing unit tests as a formal
+proof is not only shorter, it's also far more readable and also, if
+written properly, provides 100% coverage of corner-cases that would
+otherwise be overlooked or require tens to hundreds of thousands of
+tests to be run.
+
+No this is not a joke or even remotely hypothetical, this is an actual
+real-world problem.
+
+The ieee754fpu requires several hundreds of thousands of tests to be
+run (currently needing several days to run them all), and even then we
+cannot be absolutely certain that all possible combinations of input have
+been tested. With 2^128 permutations to try with 2 64 bit FP numbers
+it is simply impossible to even try.
+
+This is where formal proofs come into play.
+
+Samuel illustrated to us that "ordinary" unit tests can then be written
+to *augment* the formal ones, serving the purpose of illustrating how
+to use the module, more than anything.
+
+However it is appreciated that writing formal proofs is a bit of a
+black art. This is where team collaboration particularly kicks in,
+so if you need help, ask on the mailing list.
+
+# TODO Tutorials
+
+Find appropriate tutorials for nmigen and yosys, as well as symbiyosys.
+* Although a verilog example this is very useful to do
+ <https://symbiyosys.readthedocs.io/en/latest/quickstart.html#first-step-a-simple-bmc-example>
+* This tutorial looks pretty good and will get you started
+ <http://blog.lambdaconcept.com/doku.php?id=nmigen:nmigen_install> and
+ walks not just through simulation, it takes you through using gtkwave
+ as well.
+* There exist several nmigen examples which are also executable
+ <https://github.com/m-labs/nmigen/tree/master/examples/> exactly as
+ described in the above tutorial (python3 filename.py -h)
projects / libreriscv.git / blobdiff