HDL_workflow.mdwn

   1 # HDL workflow
   2
   3 This section describes the workflow for developing the LibreSoC.
   4
   5 # Hardware
   6
   7 RAM is the biggest requirement. Minimum 16GB, the more the better (32 or 64GB starts to reach "acceptable" levels.  Disk space is not hugely critical: 256GB SSD should be more than adequate.  Simulations however are where raw processing power is a must.  High end Graphics Cards are nonessential.
   8
   9 # Operating System
  10
  11 First install Debian (ubuntu if you absolutely must) for standardisation cross-team and so that toochain installation is greatly simplified.
  12
  13 A basic GUI desktop is necessary: fvwm2, xfce, lxde are perfectly sufficient (alongside wicd-gtk for network management). Other more complex desktops can be used however may consume greater resources.
  14
  15 # Software prerequisites
  16
  17 Whilst many resources online advocate sudo in front of all of the root-level commands below, this is extremely irritating. run "sudo bash", get a root prompt, and save some typing.
  18
  19 * apt-get install build-essential
  20 * apt-get install git python3.7 python3.7-dev python-nosetest3
  21 * apt-get install graphviz xdot
  22 * return to user prompt (ctrl-d)
  23
  24 ## yosys
  25
  26 Follow the source code (git clone) instructions here: <http://www.clifford.at/yosys/download.html>
  27
  28 Do not try to use a fixed revision, nmigen is evolving and frequently interacts with yosys
  29
  30 ## symbiyosys
  31
  32 Follow the instructions here: <https://symbiyosys.readthedocs.io/en/latest/quickstart.html#installing>
  33
  34 You do not have to install all of those (avy, boolector can be left out if desired).
  35
  36 ## nmigen
  37
  38 nmigen may be installed as follows:
  39
  40 * mkdir ~/src
  41 * cd !$
  42 * git clone https://github.com/m-labs/nmigen.git
  43 * cd nmigen
  44 * sudo bash
  45 * python3 setup.py develop
  46 * ctrl-d
  47
  48 testing can then be carried out with "python3 setp.py test"
  49
  50 # Registering for git repository access
  51
  52 After going through the onboarding process and having agreed to take responsibility for certain tasks, ask on the mailing list for git repository access, sending in a public key (id_rsa.pub). If you do not have one generate it with ssh-keygen -t rsa.
  53
  54 NEVER SEND ANYONE THE PRIVATE KEY.  By contrast the public key, on account of being public, is perfectly fine to make... err... public.
  55
  56 Create a file ~/.ssh/config with the following lines:
  57
  58     Host git.libre-riscv.org
  59     Port 922
  60
  61 Wait for the Project Admin to confirm that the ssh key has been added to the required repositories.  Once confirmed,
  62 you can clone any of the repos at http://git.libre-riscv.org:
  63
  64     git clone gitolite3@git.libre-riscv.org:REPONAME.git
  65
  66 # Development Rules
  67
  68 team communication:
  69
  70 * communicate on the mailing list or the bugtracker an intent to take responsibility for a particular task.
  71 * assign yourself as the bug's owner
  72 * *keep in touch* about what you are doing, and why.
  73 * if you cannot do sonething that you gave taken responsibility for, then unless it is a dire emergency please say so, on-list. we won't mind. we'll help sort it out.
  74
  75 for actual code development:
  76
  77 * **do not commit autogenerated output**. write a shell script and commit that, or add a Makefile to run the command that generates the output, but **do not** add the actual output of **any** command to the repository.  ever.  this is really important.  even if it is a human-readable file rather than a binary object file.
  78 * if the command needed to create any given autogenerated output is not currently in the list of dependencies, first consult on the list if it is okay to make that command become a hard dependency of the project (hint: java, node.js php and .NET commands will cause delays in responses due to list participants laughing hysterically too much), and after a decision is made, document the dependency and how its source code is obtained and built.
  79 * plan in advance to write not just code but a full test suite for that code.  **this is not optional**. large python projects that do not have unit tests **FAIL**.
  80 * edit files making minimal *single purpose* modifications.
  81 * prior to committing make sure that relevant unit tests pass, or that the change is a zero-impact addition.
  82 * commit no more than 5 to 10 lines at a time, with a CLEAR message (no "added this" or "changed that"). if the commit involves a list of changes or the word "and" it is a "red flag" that the commit has not been properly broken down.
  83 * if it is essential to commit large amounts of code, ensure that it is **not** in use **anywhere** by any other code.
  84
  85 the reason for the above is because python is a weakly typed language.  make one tiny change at the base level of the class hierarchy and the effect may be disastrous.
  86
  87 * all code needs to conform to pep8.  use either pep8checker or better run autopep8.  however whenever committing whitespace changes, *make a separate commit* with a commit message "whitespace" or "autopep8 cleanup".
  88 * pep8 REQUIRES no more than 80 chars per line. this is non-negotiable. if you think you need greater than 80 chars, it *fundamentally* indicates poor code design. split the code down further into smaller classes and functions.
  89 * TBD there is a docstring checker.  at the minimum make sure to have an SPD license heafer, module header docstring, class docstring and function docstrings on non-obvious functions
  90 * make liberal but not excessive use of comments.
  91 * unless they are very closely related, only have one module (one class) per file. a file only 25 lines long including imports and docstrings is perfectly fine.
  92 * *keep files short and simple*. see below as to why
  93 * create a decent directory hierarchy but do not go mad. ask for advice if unsure
  94 * please do not use "from module import *". it is extremely bad practice, causes unnecessary resource utilisation, makes code readability extremely difficult, and results in unintended side-effects.
  95 * try to keep both filenames and variable names short but not ridiculously obtuse. an interesting compromise on imports is "from ridiculousfilename import longsillyname as lsn", and to assign variables as well: "comb = m.d.comb" followed by multiple "comb += nmigenstmt" lines is a good trick that can reduce code indentation by 6 characters without reducing clarity.
  96
  97 regarding code structure: we decided to go with small modules that are both easy to analyse, as well as fit onto a single page and be readable when displayed as a visual graph on a full UHD monitor.  this is done as follows:
  98
  99 * using the capability of nmigen (TODO crossref to example) output the module to a yosys ilang (.il) file
 100 * in a separate terminal window, run yosys
 101 * at the yosys prompt type "read_ilang modulename.il"
 102 * type "show top" and a graphviz window should appear. note that typing show, then space, then pressing the tab key twice will give a full list of submodules (one of which will be "top")
 103
 104 you can now fullsize the graphviz window and scroll around.  if it looks reasonably obvious, i.e the connections can be clearly related in your mind back to the actual code (by matching the graph names against signals and modules in the original nmigen code) and the words are not tiny when zoomed out, and connections are not total incomprehensible spaghetti, then congratulations, you have well-designed code. If not, then this indicates a need to split the code further into submodules.
 105
 106 The reasons for doing a proper modularisatoion job are several-fold:
 107
 108 * firstly, we will not be doing a full automated layout-and-hope using alliance/ciriolis2, we will be doing leaf-node thru tree node half-automated half-manual layout, finally getting to the floorplan, then revising and iteratively adjusting.
 109 * secondly, examining modules at the gate level (or close to it) is just good practice.  poor design creeps in by *not* knowing what the tools are actually doing.
 110 * thirdly, unit testing, particularly formal proofs, is far easier on small sections of code.
 111