(no commit message)

[libreriscv.git] / HDL_workflow.mdwn

diff --git a/HDL_workflow.mdwn b/HDL_workflow.mdwn
index c97d9efdb47b3d0e684e2318497a50a78bbe2c1c..45a0cb13ce86972c7d081281b1b86a35a101c0d1 100644 (file)
--- a/HDL_workflow.mdwn
+++ b/HDL_workflow.mdwn
@@ -33,16 +33,13 @@ either actively prevents or makes that much harder.
 The main message here: **use the right tool for the right job**.
 
 * mailing list: general communication and discussion.
 The main message here: **use the right tool for the right job**.
 
 * mailing list: general communication and discussion.

-* irc channel #libre-soc: real(ish)-time communication.
+* irc channel #libre-soc on irc.libera.chat: real(ish)-time communication.

 * bugtracker: task-orientated, goal-orientated *focussed* discussion.
 * ikiwiki: document store, information store, and (editable) main website
 * git repositories: code stores (**not binary or auto-generated output store**)
 * ftp server (<https://ftp.libre-soc.org/>): large (temporary,
   auto-generated) file store.
 
 * bugtracker: task-orientated, goal-orientated *focussed* discussion.
 * ikiwiki: document store, information store, and (editable) main website
 * git repositories: code stores (**not binary or auto-generated output store**)
 * ftp server (<https://ftp.libre-soc.org/>): large (temporary,
   auto-generated) file store.
 

-We will add an IRC channel at some point when there are enough people
-to warrant having one (and it will be publicly archived)
-

 Note also the lack of a "forum" in the above list.  this is very
 deliberate.  forums are a serious distraction when it comes to technical
 heavily goal-orientated development. recent internet users may enjoy
 Note also the lack of a "forum" in the above list.  this is very
 deliberate.  forums are a serious distraction when it comes to technical
 heavily goal-orientated development. recent internet users may enjoy


@@ -109,7 +106,8 @@ on the server, that is a high security risk, and i'm not doing it. sorry.)
 ### Mailing list != editable document store
 
 Also, please do not use the mailing list as an "information or document
 ### Mailing list != editable document store
 
 Also, please do not use the mailing list as an "information or document

-store or poor-man's editor". We have the wiki for that.  Edit a page and
+store or poor-man's editor" **including not sending large images**.
+We have the wiki for that.  Edit a page and

 tell people what you did (summarise rather than drop the entire contents
 at the list) and include the link to the page.
 
 tell people what you did (summarise rather than drop the entire contents
 at the list) and include the link to the page.
 


@@ -284,6 +282,11 @@ access to ultra-high resolution screens.
 
 # Software prerequisites<a name="software-prerequisites"></a>
 
 
 # Software prerequisites<a name="software-prerequisites"></a>
 

+**Please make sure if you install manually that you install dependencies
+in strict order.  Failing to adhere to this will result in pip3 downloading
+unauthorised older software versions.  See
+<http://lists.libre-soc.org/pipermail/libre-soc-dev/2021-September/003666.html>**
+

 Whilst many resources online advocate "`sudo`" in front of all root-level
 commands below, this quickly becomes tiresome. run "`sudo bash`", get a
 root prompt, and save yourself some typing.
 Whilst many resources online advocate "`sudo`" in front of all root-level
 commands below, this quickly becomes tiresome. run "`sudo bash`", get a
 root prompt, and save yourself some typing.


@@ -291,7 +294,7 @@ root prompt, and save yourself some typing.
 * sudo bash
 * apt-get install vim exuberant-ctags
 * apt-get install build-essential
 * sudo bash
 * apt-get install vim exuberant-ctags
 * apt-get install build-essential

-* apt-get install git python3.7 python3.7-dev python-nosetest3
+* apt-get install git python3.7 python3.7-dev python3-nose

 * apt-get install graphviz xdot gtkwave
 * apt-get install python3-venv
 * apt-get install python-virtualenv  # this is an alternative to python3-venv
 * apt-get install graphviz xdot gtkwave
 * apt-get install python3-venv
 * apt-get install python-virtualenv  # this is an alternative to python3-venv


@@ -334,20 +337,27 @@ relevant unit tests pass 100%.  This ensures that people's work does not
 get "lost" or isolated and out of touch due to major branch diversion,
 and that people communicate and coordinate with each other.
 
 get "lost" or isolated and out of touch due to major branch diversion,
 and that people communicate and coordinate with each other.
 

+This is not a hard rule: under special cirmstances branches can be useful.
+They should not however be considered "routine".
+
+For advice on commit messages see
+[here](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html),
+and [here](https://github.com/torvalds/subsurface-for-dirk/blob/master/README.md#contributing)).
+

 ## yosys
 
 Follow the source code (git clone) instructions here, do **not** use
 the "stable" version (do not download the tarball):
 ## yosys
 
 Follow the source code (git clone) instructions here, do **not** use
 the "stable" version (do not download the tarball):

-<http://www.clifford.at/yosys/download.html>
+<https://github.com/YosysHQ/yosys>

 
 Or, alternatively, use the
 
 Or, alternatively, use the

-[yosys-et-al](https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=yosys-et-al;hb=HEAD)
+[hdl-tools-yosys](https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=hdl-tools-yosys;hb=HEAD)

 script (which also installs symbiyosys and its dependencies)
 
 Do not try to use a fixed revision of yosys (currently 0.9), nmigen is
 evolving and frequently interacts with yosys.
 
 script (which also installs symbiyosys and its dependencies)
 
 Do not try to use a fixed revision of yosys (currently 0.9), nmigen is
 evolving and frequently interacts with yosys.
 

-[Yosys](http://www.clifford.at/yosys/) is a framework for Verilog RTL.
+[Yosys](https://github.com/YosysHQ/yosys is a framework for Verilog RTL.

 [Verilog](https://en.wikipedia.org/wiki/Verilog) is a hardware description
 language.
 RTL [Register Transfer
 [Verilog](https://en.wikipedia.org/wiki/Verilog) is a hardware description
 language.
 RTL [Register Transfer


@@ -373,7 +383,11 @@ flows.
 
 ## nmigen
 
 
 ## nmigen
 

-[nmigen](https://m-labs.hk/gateware/nmigen/) may be installed as follows:
+**PLEASE NOTE: it is critical to install nmigen as the first dependency
+prior to installing any further python-based Libre-SOC HDL repositories.
+If "pip3 list" shows that nmigen has been auto-installed please remove it**
+
+[nmigen](https://nmigen.info/) may be installed as follows:

 
 * mkdir ~/src
 * cd !$
 
 * mkdir ~/src
 * cd !$


@@ -429,7 +443,7 @@ In the meantime, sfpy can be built as follows:
 You can test your installation by doing the following:
 
     python3
 You can test your installation by doing the following:
 
     python3

-    >>> from sfpy import *
+    >>> from sfpy import Posit8

     >>> Posit8(1.3)
 
 It should print out `Posit8(1.3125)`
     >>> Posit8(1.3)
 
 It should print out `Posit8(1.3125)`


@@ -438,20 +452,27 @@ It should print out `Posit8(1.3125)`
 
 As we are doing POWER ISA, POWER ISA compilers, toolchains and
 emulators are required.
 
 As we are doing POWER ISA, POWER ISA compilers, toolchains and
 emulators are required.

+Again, if you want to save yourself some typing, use the dev scripts.
+[install-hdl-apt-reqs](https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=install-hdl-apt-reqs;hb=HEAD)
+script will install the qemu;
+[ppc64-gdb-gcc](https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=ppc64-gdb-gcc;hb=HEAD)
+script will install the toolchain and the corresponding debugger.
+The steps are provided below only for reference; when in doubt,
+consider checking and running the scripts.

 
 Install powerpc64 gcc:
 
 
 Install powerpc64 gcc:
 

-    apt-get install gcc-9-powerpc64-linux-gnu
+    apt-get install gcc-8-powerpc64-linux-gnu

 
 Install qemu:
 
     apt-get install qemu-system-ppc
 
 Install gdb from source.  Obtain the required tarball matching
 
 Install qemu:
 
     apt-get install qemu-system-ppc
 
 Install gdb from source.  Obtain the required tarball matching

-the version of gcc (9.1) from here <https://ftp.gnu.org/gnu/gdb/>,
+the version of gcc (8.3) from here <https://ftp.gnu.org/gnu/gdb/>,

 unpack it, then:
 
 unpack it, then:
 

-    cd gdb-9.1 (or other location)
+    cd gdb-8.3 (or other location)

     mkdir build
     cd build
      ../configure --srcdir=.. --host=x86_64-linux --target=powerpc64-linux-gnu
     mkdir build
     cd build
      ../configure --srcdir=.. --host=x86_64-linux --target=powerpc64-linux-gnu


@@ -462,37 +483,25 @@ unpack it, then:
 programs.  [qemu](https://www.qemu.org/) emulates processors, you can
 run programs under qemu.
 
 programs.  [qemu](https://www.qemu.org/) emulates processors, you can
 run programs under qemu.
 

-## power_instruction_analyzer (pia)
-
-We have a custom tool built in rust by programmerjake to help analyze
-the power instructions execution on *actual* hardware.
+## power-instruction-analyzer (pia)

 
 

-Note: a very recent version of pip3 is required for this to work.
+We have a custom tool built in Rust by programmerjake to help analyze
+the OpenPower instructions' execution on *actual* hardware.

 
 

-Install rust:
+Install Rust:

 
     curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
 
 
     curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
 

-Make sure we have the correct and up-to-date rust compiler (rustc):
+Make sure we have the correct and up-to-date rust compiler (rustc & cargo):

 
     rustup default stable
     rustup update
 
 
     rustup default stable
     rustup update
 

-Use rust's package manager *cargo* to install the rust-python build
-tool maturin:
-
-    cargo install maturin
-
-Install from git source by doing the following:
+Install the Python extension from git source by doing the following:

 
     git clone https://salsa.debian.org/Kazan-team/power-instruction-analyzer.git pia
     cd pia
 
     git clone https://salsa.debian.org/Kazan-team/power-instruction-analyzer.git pia
     cd pia

-    maturin build --cargo-extra-args=--features=python-extension
-    python3 -m pip install --user target/wheels/*.whl
-
-Note: an ongoing bug in maturin interferes with successful installation.
-This can be worked around by explicitly installing only the `.whl`
-files needed rather than installing everything (`\*.whl`).
+    ./libre-soc-install.sh

 
 ## Chips4Makers JTAG
 
 
 ## Chips4Makers JTAG
 


@@ -538,7 +547,9 @@ See [[HDL_workflow/coriolis2]] page, for those people doing layout work.
 
 A portable FPGA place and route tool.
 
 
 A portable FPGA place and route tool.
 

-See [[HDL_workflow/nextpnr]] page for installation instructions of nextpnr with ECP5 support for Lattice FPGA ECP5 series.
+See [[HDL_workflow/nextpnr]] page for installation instructions of nextpnr with ECP5 support for Lattice FPGA ECP5 series.  Also see
+[[HDL_workflow/ECP5_FPGA]] for connecting up to JTAG with a ULX3S
+and the Lattice VERSA_ECP5.

 
 ## Verilator
 
 
 ## Verilator
 


@@ -572,7 +583,16 @@ cocotb is a COroutine based COsimulation TestBench environment for verifying VHD
 
 See [[HDL_workflow/cocotb]] page for installation instructions.
 
 
 See [[HDL_workflow/cocotb]] page for installation instructions.
 

-# Registering for git repository access
+## Symbiflow
+
+A fully open source toolchain for the development of FPGAs. Currently it targets Xilinx 7-series, Lattice iCE40 and ECP5, Quicklogic EOS S3.
+
+Needed for the Arty A7 100t Digilent FPGA board.
+
+See [[HDL_workflow/symbiflow]] for installation instructions
+and dependencies.
+
+# Registering for git repository access<a name="gitolite3_access"></a>

 
 After going through the onboarding process and having agreed to take
 responsibility for certain tasks, ask on the mailing list for git
 
 After going through the onboarding process and having agreed to take
 responsibility for certain tasks, ask on the mailing list for git


@@ -588,6 +608,14 @@ Create a file `~/.ssh/config` with the following lines:
     Host git.libre-soc.org
     Port 922
 
     Host git.libre-soc.org
     Port 922
 

+Test that you have access with this command:
+
+    ssh -v -p922 gitolite3@git.libre-soc.org
+
+Please note: **DO NOT TYPE A PASSWORD** - the server gets hit by a lot of
+port-scanning, and detection of password failures are used to instantly
+ban IP addresses.
+

 Wait for the Project Admin to confirm that the ssh key has been added
 to the required repositories.  Once confirmed, you can clone any of the
 repos at https://git.libre-soc.org/:
 Wait for the Project Admin to confirm that the ssh key has been added
 to the required repositories.  Once confirmed, you can clone any of the
 repos at https://git.libre-soc.org/:


@@ -598,6 +626,16 @@ Alternatively, the .ssh/config can be skipped and this used:
 
      git clone ssh://gitolite3@git.libre-soc.org:922/REPONAME.git
 
 
      git clone ssh://gitolite3@git.libre-soc.org:922/REPONAME.git
 

+Note: **DO NOT ATTEMPT TO LOG IN TO THE SERVER WITH A PERSONAL ACCOUNT**.
+fail2ban is running and, due to repeated persistent port-scanning spammers
+is set up to instantly ban any unauthorised ssh access for up to two weeks.
+This keeps log file sizes down on the server (which is resource-constrained).
+If you are wondering why this is done, it's a *lot* of port-scans.
+
+Therefore, *only* ssh in to server with the gitolite3 account, *only*
+on port 922, and *only* once the systems administrator has given you
+the all-clear that the ssh key has been added.
+

 # git configuration
 
 Although there are methods online which describe how (and why) these
 # git configuration
 
 Although there are methods online which describe how (and why) these


@@ -625,6 +663,10 @@ Before running the following, install the
 dependencies.  This is easiest done with this script
 <https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=install-hdl-apt-reqs;hb=HEAD>
 
 dependencies.  This is easiest done with this script
 <https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=install-hdl-apt-reqs;hb=HEAD>
 

+**It is critically important to install these in STRICT order, otherwise
+pip3 interferes and performs unauthorised downloads without informing
+you of what it is doing**.
+

 * mkdir ~/src
 * cd !$
 * git clone gitolite3@git.libre-soc.org:nmigen.git
 * mkdir ~/src
 * cd !$
 * git clone gitolite3@git.libre-soc.org:nmigen.git


@@ -635,7 +677,7 @@ dependencies.  This is easiest done with this script
 * git clone gitolite3@git.libre-soc.org:nmigen-soc.git
 * git clone gitolite3@git.libre-soc.org:soc.git
 
 * git clone gitolite3@git.libre-soc.org:nmigen-soc.git
 * git clone gitolite3@git.libre-soc.org:soc.git
 

-In each of these directories, in the order listed, track down the
+In each of these directories, **in the order listed***, track down the

 `setup.py` file, then, as root (`sudo bash`), run the following:
 
 * python3 setup.py develop
 `setup.py` file, then, as root (`sudo bash`), run the following:
 
 * python3 setup.py develop


@@ -646,7 +688,9 @@ for multi-user machine use however it is often just easier to get your
 own machine these days.
 
 The reason for the order is because soc depends on ieee754fpu, and
 own machine these days.
 
 The reason for the order is because soc depends on ieee754fpu, and

-ieee754fpu depends on nmutil
+ieee754fpu depends on nmutil.  If you do not follow the listed order
+pip3 will go off and download an arbitrary version without your
+consent.

 
 If "`python3 setup.py install`" is used it is a pain: edit, then
 install. edit, then install. It gets extremely tedious, hence why
 
 If "`python3 setup.py install`" is used it is a pain: edit, then
 install. edit, then install. It gets extremely tedious, hence why


@@ -887,6 +931,11 @@ the double import of the class from two separate locations, immediately.
 
 Really.  don't.  use.  wildcards.
 
 
 Really.  don't.  use.  wildcards.
 

+More about this here:
+
+* <https://www.asmeurer.com/removestar/>
+* <https://rules.sonarsource.com/python/RSPEC-2208>
+

 ### Keep file and variables short but clear
 
 * try to keep both filenames and variable names short but not ridiculously
 ### Keep file and variables short but clear
 
 * try to keep both filenames and variable names short but not ridiculously


@@ -1042,10 +1091,10 @@ Find appropriate tutorials for nmigen and yosys, as well as symbiyosys.
 * Although a verilog example this is very useful to do
   <https://symbiyosys.readthedocs.io/en/latest/quickstart.html#first-step-a-simple-bmc-example>
 * This tutorial looks pretty good and will get you started
 * Although a verilog example this is very useful to do
   <https://symbiyosys.readthedocs.io/en/latest/quickstart.html#first-step-a-simple-bmc-example>
 * This tutorial looks pretty good and will get you started

-  <http://blog.lambdaconcept.com/doku.php?id=nmigen:nmigen_install> and
-  walks not just through simulation, it takes you through using gtkwave
-  as well.
+  <https://web.archive.org/web/20210123052724/http://blog.lambdaconcept.com/doku.php?id=nmigen:nmigen_install>
+  and walks not just through simulation, it takes you through using
+  gtkwave as well.

 * There exist several nmigen examples which are also executable
 * There exist several nmigen examples which are also executable

-  <https://github.com/m-labs/nmigen/tree/master/examples/> exactly as
+  <https://github.com/nmigen/nmigen/tree/master/examples/> exactly as

   described in the above tutorial (python3 filename.py -h)
   described in the above tutorial (python3 filename.py -h)

-
+* More nmigen tutorials at [[learning_nmigen]]