-# Build instructions for Microwatt
+# Tutorial for setting up Microwatt chroot and running simulations
-WIP integrate from <https://libre-soc.org/irclog/%23libre-soc.2022-01-26.log.html>
+Useful Links (External):
-* install verilator: <https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=verilator-install;hb=HEAD>
-* install yosys <https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=hdl-tools-yosys;hb=HEAD>
-* install ghdl-yosys-plugin
+* <https://codeconstruct.com.au/docs/microwatt-orangecrab/>
+* <https://shenki.github.io/boot-linux-on-microwatt/>
+* <https://github.com/gregdavill/OrangeCrab-test-sw>
+* [Verilator docs, commands](https://verilator.org/guide/latest/exe_verilator.html)
+* [Verilator runtime command documentation](https://verilator.org/guide/latest/exe_sim.html)
+* Tutorials for how to work with verilator:
+[part1](https://www.itsembedded.com/dhd/verilator_1/),
+[part2](https://www.itsembedded.com/dhd/verilator_2/)
-After installing ghdl-yosys-plugin use schroot to switch environment
+Useful links (Libre-SOC):
- schroot -c bullseye_ghdl
- git clone https://git.libre-soc.org/git/microwatt.git
- git checkout -b verilator_trace
- export FPGA_TARGET=verilator
- make microwatt-verilator
+* Libre-SOC page covering our workflow: [[HDL_workflow]]
+* Devscripts Libre-SOC page: [[devscripts]]
+* Original Microwatt Libre-SOC page: [[microwatt]]
+* [Libre-SOC Microwatt repo branch](https://git.libre-soc.org/?p=microwatt.git;a=tree;hb=refs/heads/verilator_trace)
+* [Libre-SOC devscripts repo](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree)
+
+Other Tutorials (Libre-SOC):
+
+* First steps for working with PowerISA instructions Libre-SOC page:
+[[/docs/firststeps]]
+
+## Video Tutorial
+
+[43min tutorial](https://youtu.be/02LCl3ang8g) was made and uploaded to
+Youtube, covering some of the material you'll find on this page.
+
+## Development environment scripts
+
+If you haven't already, clone Libre-SOC's development environment setup scripts.
+These are bash scripts, and greatly simplify the time it takes to create a:
+
+- Stable environment
+- With all software and libraries at specific versions
+(which are known to work).
+
+These attributes are absolutely critical, and no support will be
+provided, unless you use these scripts to setup a development environment. This
+helps us fix any bugs in the scripts, and make sure everyone runs on the same
+page.
+
+ $ git clone https://git.libre-soc.org/git/dev-env-setup.git
+
+
+Do *look through* the
+[code](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) before running
+any of those scripts.
+This is your own legal responsibility (to not run
+arbitrary code off of the internet) and we take no responsibility or accept
+any liability whatsoever for your use or misuse of them.
+
+It is expected for you to use Debian for the host OS (anything else
+is unsupported: many contributors have repeatedly gotten into trouble by not
+following this advice), while all the chroots - which are developed
+very specifically for "reproducible builds" - run Debian 10 (Buster).
+
+## Setting up chroot
+
+Scripts we will be using for the setup are:
+
+* `mk-deb-chroot`, `cp-scripts-to-chroot` for chroot setup
+* `install-hdl-apt-reqs`, `verilator-install`, `hdl-tools-yosys` for working
+with Microwatt
+
+(*Current limitation for `mk-deb-chroot`, is that you must be the first user on
+the host machine, having user ID 1000.*)
+
+Commands to run in terminal to setup a new chroot environment for microwatt
+simulations.
+
+ $ cd dev-env-setup
+ $ sudo bash
+ # ./mk-deb-chroot microwatt
+ # ./cp-scripts-to-chroot microwatt
+ # exit
+ $ schroot -c microwatt
+ (microwatt):$ cd dev-env-setup
+ (microwatt):$ sudo bash
+ (microwatt):# ./install-hdl-apt-reqs
+ (microwatt):# ./verilator-install
+ (microwatt):# ./hdl-tools-yosys
+ (microwatt):# exit
+ (microwatt):$ cd ~/src/
+ (microwatt):$ git clone https://git.libre-soc.org/git/microwatt.git
+ (microwatt):$ cd microwatt
+ (microwatt):$ git checkout verilator_trace
+
+Make sure verilator binaries in $PATH:
+
+ (microwatt):$ export PATH=/usr/local/verilator/bin:$PATH
+ (microwatt):$ export GHDLSYNTH=ghdl
+
+(GHDLSYNTH needs to be redefined because the Makefile has default `ghdl.so`,
+but somewhere else '.so' gets appended. You may see the following error if you
+don't redefine:
+`ERROR: Can't load module
+./ghdl.so':/usr/local/bin/../share/yosys/plugins/**ghdl.so.so**`)
+[IRC](https://libre-soc.org/irclog/%23libre-soc.2023-01-25.log.html#t2023-01-25T11:10:47)
+
+## Compiling the verilator sim for Microwatt
+
+* [Libre-SOC Microwatt repo branch, Makefile](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=Makefile;hb=refs/heads/verilator_trace)
+
+Verilator creates a fairly fast simulation by converting the HDL design to C++,
+and then compiling a binary which the user runs.
+
+To compile the verilator simulation, first set verilator as the target for the
+makefile:
+
+ (microwatt):$ export FPGA_TARGET=verilator
+
+Before compiling, you can change the `THREADS` variable in the makefile, which
+will allow the compiled verilator simulation binary to use more than 1 thread
+(*make sure to check how many CPU threads you have before changing this!*)
+
+To compile the verilator simulation binary, call make with the
+`microwatt-verilator` rule.
+
+ (microwatt):$ make microwatt-verilator
+
+## Compiling hello world code
+
+We need some code to actually run on the core, so start with the 'hello world'.
+Instructions assume you're still in the microwatt directory.
+
+ (microwatt):$ cd hello_world
+ (microwatt):$ make
+
+A `hello_world.bin` should be generated (the final binary to be loaded), as
+well as an
+[.elf file](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format), and
+.hex (representing the binary data as hex text strings).
+
+To view the symbol table (useful to see where various sections of the binary
+begin):
+
+ (microwatt):$ powerpc64le-linux-gnu-objdump -h hello_world.elf
+ (microwatt):$ powerpc64le-linux-gnu-objdump -x hello_world.elf
+
+`-h` shows just the section headers, `-x` shows all headers.
+
+And to view the disassembly (great for learning about the PowerISA instructions,
+and for associating the binary hex with actual instructions), you can view the
+automatically generated `hello_world.as` file.
+
+Command to generate the disassembly:
+
+ (microwatt):$ powerpc64le-linux-gnu-objdump -D hello_world.elf
+
+For more information about `objdump` (common utility, not just for PowerISA),
+see the manual pages.
+
+ (microwatt):$ man powerpc64le-linux-gnu-objdump
+
+The binary is ready to go, now it can be loaded into the simulation.
+
+## Simulation
+
+### Command line args
+
+To find out the `microwatt-verilator` arguments, you can check with `-h` arg:
+
+ (microwatt):$ ./microwatt-verilator -h
+
+Some of the arguments are explained in further sections.
+
+### Running
+
+Run the `microwatt-verilator` binary, with `hello_world/hello_world.bin` as an
+argument:
+
+ (microwatt):$ time ./microwatt-verilator hello_world/hello_world.bin
+
+`time` is a utility you can use to measure how long it takes to run the sim.
+
+A pretty ASCII art of a lightbulb should be printed, and then the user can type
+any characters, which will be echoed back. To end the simulation press Ctrl+C.
+
+If no characters are appearing after about 20 seconds, stop the simulation,
+as there might be other issues.
+
+Single-threaded verilator sim binary, on a 2nd gen intel i5 (sandybridge)
+takes 53 seconds to print the ASCII lightbulb.
+
+On another dev's machine, ASUS KGPE D16, this takes just over a minute.
+
+(*You'll find that uart printout is one of the longer parts of the simulation
+in general.*)
+
+## Analysing results after simulation
+
+The following files will be generated during the sim:
+
+- `bram.dump` - Shows the PC address and instruction being executed. If the sim
+hangs without any printing, view this file, as the processor may have hit an
+exception etc. Grows in size as the sim runs.
+
+- `bram.snapshot.[NUMBER]`, `verilator.save.[NUMBER]` - Snapshot files of the
+contents of bram and verilator model respectively. Can be used to resume the
+simulation. The number on the end corresponds to the tick time (i.e.
+`bram.snapshot.1999990`/`verilator.save.1999990`). First the verilator model is
+loaded, and then the bram contents are loaded. See lines `#65-108` and
+`#189-195` of the
+[microwatt-verilator.cpp file](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=verilator/microwatt-verilator.cpp;h=a226393f6ba74d5e3e1ffdb729d731d2311d53ad;hb=refs/heads/verilator_trace).
+Pass the tick number on the end of the filename with the '-s' flag:
+
+ (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -s 1999990
+
+You'll get a message like this:
+
+ loading hello_world/hello_world.bin at 0x0 size 0x1888
+ loading bram.snapshot.1999990 at 0x0 size 0x10000000
+ restored at 1999990
+
+These snapshots are generated at intervals of every 2,000,000 ticks.
+
+- `microwatt-verilator.vcd` - GTKWave waveform file, allowing you to look at
+processor signals and transitions during simulation.
+Pass `-d` flag to `microwatt-verilator` binary:
+
+ (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -d
+
+**NOTE**: Trace dumping will generate a large VCD file (about 6GB for the hello
+world example)!
-Then run the resulting binary
+If you want GTKWave to load it faster, convert to fst first:
- time ./microwatt-verilator
+ (microwatt):$ vcd2fst --vcdname=microwatt-verilator.vcd --fstname=microwatt-verilator.fst
+ (microwatt):$ gtkwave microwatt-verilator.fst
-If everything works correcly it will print out an light bulb in ascii art
-with the message Microwatt, it works. On my ASUS KGPE D16 this takes
+Fst files are orders-of-magnitude smaller (about 20MB vs 6GB), but are specific
+to the GTKWave tool.
+
+## Micropython
+
+The Microwatt repo comes with a pre-compiled
+[micropython binary](https://git.libre-soc.org/?p=microwatt.git;a=tree;f=micropython;h=18fa078c8145bdaa75667a0ab04eb0b261245665;hb=refs/heads/verilator_trace)
+(version 1.12), which you can try out after confirming 'hello world' works.
+Bear in mind, not all features of python will be available. Such as
+floating-point numbers.
+
+For micropython to work, you'll need to increase the RAM size in the makefile.
+Go to the microwatt-verilator makefile, and comment out the following lines:
+
+ MEMORY_SIZE=8192
+ RAM_INIT_FILE=hello_world/hello_world.hex
+
+And uncomment the following:
+
+ MEMORY_SIZE=393216
+ RAM_INIT_FILE=micropython/firmware.hex
+
+This will increase the RAM size from 8KiB to 384KiB. The `RAM_INIT_FILE` in
+these examples isn't doing anything, however good practice to follow.
+
+Clean up generated files, and recompile:
+
+ (microwatt):$ make clean
+ (microwatt):$ make microwatt-verilator
+
+Once the binary has been built, run the same way as before, but point to the
+micropython firmware binary:
+
+ (microwatt):$ microwatt-verilator micropython/firmware.bin
+
+On the same system as above, with 1 thread, it took 49 seconds to get to the
+micropython shell.
+
+## Verilator runtime commands
+A few examples:
+
+ # Show the version of verilator being used
+ (microwatt):$ ./microwatt-verilator +verilator+version
+
+## Building `microwatt-verilator` using the Libre-SOC core
+
+In the Makefile, you need to set `EXTERNAL_CORE` to true, and copy the
+generated core from soc repo to microwatt. *(If you use a separate chroot to
+generate Libre-SOC cores, then you'll need to copy from that chroot to
+microwatt chroot from host.*
+
+ cd /path/to/soc
+ make microwatt_external_core
+ cp external_core_top.v /path/to/microwatt
+
+Then compile verilator sim binary as before:
+
+ cd ~/src/microwatt/
+ export FPGA_TARGET=verilator
+ export GHDLSYNTH=ghdl
+ make microwatt-verilator
+
+## Running Linux kernel
+
+To run Linux on Microwatt, you'll need two binaries:
+
+- The `sdram_init.bin`, which is easy to compile (no additional software
+required).
+
+- The `dtbImage.microwatt` device tree Linux kernel. This can be compiled (see
+below), or a copy can be downloaded from: <https://ftp.libre-soc.org/dtbImage.microwatt>.
+
+Like with the micropython example, you'll need to increase the RAM size in the
+makefile, and recompile the microwatt-verilator binary.
+
+Uncomment the following:
+
+ MEMORY_SIZE=536870912
+
+Which will change the RAM size to 512KiB.
+
+As there is no `dtbImage.microwatt.hex`, you can leave `RAM_INIT_FILE` unchanged.
+
+### Building the kernel - TODO:
+
+*(Please don't build the kernel yourself, until you've tested with the existing kernel linked
+above!)*
- real 1m4.986s
- user 1m4.373s
- sys 0m0.013s
-
-## Building the kernel
On a POWER9 there is no need to install gcc-powerpc64le-linux-gnu,
you can omit CROSS_COMPILE and ARCH in this case
cd microwatt
wget https://ftp.libre-soc.org/microwatt-linux-5.7.patch
patch -p1 < microwatt-linux-5.7.patch
- wget http://ftp.libre-soc.org/rootfs.cpio
+ wget https://ftp.libre-soc.org/rootfs.cpio
CROSS_COMPILE="ccache powerpc64le-linux-gnu-" ARCH=powerpc make -j8 O=microwatt microwatt_defconfig
CROSS_COMPILE="ccache powerpc64le-linux-gnu-" ARCH=powerpc make -j8 O=microwatt
This will produce a file
microwatt/arch/powerpc/boot/dtbImage.microwatt
-## Building sdram_init.bin
-This needs gcc-powerpc64le-linux-gnu if cross compilation is used.
+### Building `sdram_init.bin`
+This needs gcc-powerpc64le-linux-gnu (already included in the setup step) if
+cross compilation is used. It is assumed you're already in `~/src/microwatt/`
+directory.
+
+ (microwatt):$ cd litedram/gen-src/sdram_init/
+ (microwatt):$ make
+
+The resulting binary will be in the `obj/` directory.
+
+### Running the simulation
+
+Make sure to return back to `src/microwatt/`.
+
+ (microwatt):$ cd ~/src/microwatt/
+ (microwatt):$ cp microwatt/arch/powerpc/boot/dtbImage.microwatt
+ (microwatt):$ ./microwatt-verilator sdram_init.bin dtbImage.microwatt
+
+This will take some time...
+
+### Sysconn information
+
+TODO WIP integrate from <https://libre-soc.org/irclog/%23libre-soc.2022-01-26.log.html>
+
+Sysconn is a module which includes information about the SoC, and the info is
+printed at the start of the simulation.
+
+### Time benchmarks
+
+`microwatt-verilator` was compiled with 3 threads for faster simulation.
+
+- Time to finish printing Sysconn info: about 1min
+- Time to allocate bytes to kernel: ?
+- Time to login prompt: about 1 hour
+- Time to user shell: ?
+
+### TODO: buildroot
+
+* https://github.com/shenki/buildroot/commits/microwatt
+* https://codeconstruct.com.au/docs/microwatt-orangecrab/
+
+## FPGA Development - TODO: Need checking
+### Building the bitstring for OrangeCrab
+
cd microwatt
- cd litedram/gen-src/sdram_init/
- make
+ export FPGA_TARGET=ORANGE-CRAB
+ export GHDLSYNTH=ghdl
+ make microwatt.bit
+
+### flashing the bitstring to the OrangeCrab
+
+ make prog # this will run OpenOCD
+
+### Notes for ulx3s
+
+notes for how to compile for ulx3s
+
+ git clone https://github.com/kost/fujprog
+ (follow build procedure shown in fujprog README)
+ git clone https://git.libre-soc.org/git/microwatt.git
+ git checkout -b verilator_trace
+ export FPGA_TARGET=ulx3s
+ make microwatt.svf
+ fujprog microwatt.svf
+
+
+### Notes for nextpnr-xilinx
+
+superceded: see page [[nextpnr-xilinx]] and devscript
+<https://git.libre-soc.org/?p=dev-env-setup.git;a=blob;f=nextpnr-xilinx-install;hb=HEAD>
+
+for compiling nextpnr-xilinx and making it useable for nmigen
+to compile for the digilent arty-a7-100t, requires a little
+futzing around, using the symbiflow version of prjxray-db
+instead of the one recommended as a submodule
+
+ git clone https://github.com/gatecat/nextpnr-xilinx
+ cd nextpnr-xilinx
+ git checkout cd8b15db6ff5c1a7f10a9e
+ git submodule init
+ git submodule update
+ cd xilinx/external
+ mv prjxray-db prjxray-db-no
+ git clone https://github.com/SymbiFlow/prjxray-db
+ cd prjxray-db
+ git checkout 0a0addedd73e7
+ cp ./artix7/xc7a100t/*.json \
+ ./artix7/xc7a100tcsg324-1
+ cd ../../..
+ cmake -DARCH=xilinx .
+ make
+ make install
+ python3 xilinx/python/bbaexport.py --device xc7a100tcsg324-1 --bba xilinx/xc7a100t.bba
+ ./bbasm --l xilinx/xc7a100t.bba xilinx/xc7a100t.bin
+ mkdir -p /usr/share/nextpnr/xilinx-chipdb
+ cp xilinx/*.bin /usr/share/nextpnr/xilinx-chipdb
+ cp -aux xilinx/external/prjxray-db /usr/share/nextpnr
+
+# Additional Useful Info for UART <-> USB Serial Interface Through OrangeCrab's Built-in USB Interface
+
+This uses OrangeCrab's built-in USB interface, rather than needing a
+separate USB-serial adapter. see the following for further details:
+
+* <https://github.com/antonblanchard/microwatt/pull/347#issuecomment-1058800570>
+* <https://github.com/antonblanchard/microwatt/pull/347#issuecomment-1058834790>
+
+# running orangecrab-examples before flashing microwatt
+
+See <https://github.com/orangecrab-fpga/orangecrab-hardware/blob/main/contrib/10-orangecrab.rules>
+
+If the OrangeCrab is running in DFU mode, lsusb will show:
+
+ 1209:5af0 Generic OrangeCrab r0.2 DFU Bootloader v3.1-6-g62e92e2
+
+OrangeCrab has two DFU devices:
+
+ Found DFU: [1209:5af0] ver=0101, devnum=22, cfg=1, intf=0, path="1-4.2", alt=1, name="0x00100000 RISC-V Firmware", serial="UNKNOWN"
+ Found DFU: [1209:5af0] ver=0101, devnum=22, cfg=1, intf=0, path="1-4.2", alt=0, name="0x00080000 Bitstream", serial="UNKNOWN"
+
+Then clone and patch orangecrab-examples:
+
+ git clone https://github.com/orangecrab-fpga/orangecrab-examples
+ patch -p1 < orangecrab-examples.diff
+
+To build and flash the example:
+
+ pushd orangecrab-examples/nmigen
+ python3 blink.py
+ popd
+ sudo dfu-util -D orangecrab-examples/nmigen/build/top.bit -a 0
projects / libreriscv.git / blobdiff