HDL_workflow/microwatt_tutorial.mdwn

   1 # New tutorial for setting up Microwatt chroot and running simulations
   2
   3 useful links:
   4
   5 * Libre-SOC page covering our workflow: [[HDL_workflow]]
   6 * Devscripts Libre-SOC page: [[devscripts]]
   7 * Original Microwatt Libre-SOC page: [[microwatt]]
   8 * [Libre-SOC Microwatt repo branch](https://git.libre-soc.org/?p=microwatt.git;a=tree;hb=refs/heads/verilator_trace)
   9 * [Libre-SOC devscripts repo](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree)
  10 * [Verilator docs, commands](https://verilator.org/guide/latest/exe_verilator.html)
  11 * [Verilator runtime command documentation](https://verilator.org/guide/latest/exe_sim.html)
  12 * First steps for working with PowerISA instructions Libre-SOC page:
  13 [[/docs/firststeps]]
  14 * Tutorials for how to work with verilator: [part1](https://www.itsembedded.com/dhd/verilator_1/), [part2](https://www.itsembedded.com/dhd/verilator_2/)
  15
  16 ## Development environment scripts
  17
  18 If you haven't already, clone Libre-SOC's development environment setup scripts.
  19 These are bash scripts, and greatly simplify the time it takes to create a:
  20
  21 - Stable environment
  22 - With all software and libraries at specific versions
  23 (which are known to work).
  24
  25 These attributes are absolutely critical, and no support will be
  26 provided, unless you use these scripts to setup a development environment. This
  27 helps us fix any bugs in the scripts, and make sure everyone runs on the same
  28 page.
  29
  30     $ git clone https://git.libre-soc.org/git/dev-env-setup.git
  31
  32 Do *look through* the
  33 [code](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) before running
  34 any of those scripts. They may be confusing, however after reading a few you'll
  35 start to become more familiar with them.
  36
  37 It is expected for you to use Debian, we mostly use 11 (Bullseye) for the host
  38 OS, while all the chroots run Debian 10 (Buster).
  39
  40
  41 ## Setting up chroot
  42
  43 Commands to run in terminal to setup a new chroot environment for microwatt
  44 simulations.
  45
  46     $ cd dev-env-setup
  47     $ sudo bash
  48     # ./mk-deb-chroot microwatt
  49     # ./cp-scripts-to-chroot microwatt
  50     # exit
  51     $ schroot -c microwatt
  52     (microwatt):$ cd dev-env-setup
  53     (microwatt):$ sudo bash
  54     (microwatt):# ./install-hdl-apt-reqs
  55     (microwatt):# ./verilator-install
  56     (microwatt):# ./hdl-tools-yosys
  57     (microwatt):# apt install gtkwave -y
  58     (microwatt):# exit
  59     (microwatt):$ cd ~/src/
  60     (microwatt):$ git clone https://git.libre-soc.org/git/microwatt.git
  61     (microwatt):$ cd microwatt
  62     (microwatt):$ git checkout verilator_trace
  63
  64 Make sure verilator binaries in $PATH:
  65
  66     (microwatt):$ export PATH=/usr/local/verilator/bin:$PATH
  67     (microwatt):$ export GHDLSYNTH=ghdl
  68
  69 (GHDLSYNTH needs to be redefined because the Makefile has default `ghdl.so`,
  70 but somewhere else '.so' gets appended. You may see the following error if you
  71 don't redefine:
  72 `ERROR: Can't load module
  73 ./ghdl.so':/usr/local/bin/../share/yosys/plugins/**ghdl.so.so**`)
  74 [IRC](https://libre-soc.org/irclog/%23libre-soc.2023-01-25.log.html#t2023-01-25T11:10:47)
  75
  76 ## Compiling the verilator sim for Microwatt
  77
  78 * [Libre-SOC Microwatt repo branch, Makefile](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=Makefile;h=b764793fcdf5409be33a01f2440cea3717cb2a32;hb=refs/heads/verilator_trace)
  79
  80 Verilator creates a fairly fast simulation by converting the HDL design to C++,
  81 and then compiling a binary which the user runs.
  82
  83 To compile the verilator simulation, first set verilator as the target for the
  84 makefile:
  85
  86     (microwatt):$ export FPGA_TARGET=verilator
  87
  88 Before compiling, you can change the `THREADS` variable in the makefile, which
  89 will allow the compiled verilator simulation binary to use more than 1 thread
  90 (*make sure to check how many CPU threads you have before changing this!*)
  91
  92 To compile the verilator simulation binary, call make with the
  93 `microwatt-verilator` rule.
  94
  95     (microwatt):$ make microwatt-verilator
  96
  97
  98
  99 ## Compiling hello world code
 100
 101 We need some to actually run on the core, so...?
 102 Instructions assume you're still in the microwatt directory.
 103
 104     (microwatt):$ cd hello_world
 105     (microwatt):$ make
 106
 107 A `hello_world.bin` should be generated (the final binary to be loaded), as
 108 well as an [.elf file](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format),
 109 and .hex (representing the binary data as hex text strings).
 110
 111 To view the symbol table (useful to see where various sections of the binary
 112 begin):
 113
 114     (microwatt):$ powerpc64le-linux-gnu-objdump -h hello_world.elf
 115     (microwatt):$ powerpc64le-linux-gnu-objdump -x hello_world.elf
 116
 117 `-h` shows just the section headers, `-x` shows all headers.
 118
 119 And to view the disassembly (great for learning about the PowerISA instructions,
 120 and for associating the binary hex with actual instructions):
 121
 122     (microwatt):$ powerpc64le-linux-gnu-objdump -d hello_world.elf
 123
 124 For more information about `objdump` (common utility, not just for PowerISA),
 125 see the manual pages.
 126
 127     (microwatt):$ man powerpc64le-linux-gnu-objdump
 128
 129 The binary is ready to go, now it can be loaded into the simulation.
 130
 131 ## Running the simulation
 132
 133 Run the `microwatt-verilator` binary, with `hello_world/hello_world.bin` as an
 134 argument:
 135
 136     (microwatt):$ time ./microwatt-verilator hello_world/hello_world.bin
 137
 138 `time` is a utility you can use to measure how long it takes to run the sim.
 139
 140 A pretty ASCII art of a lightbulb should be printed, and then the user can type any characters, which will be echoed back. To end the simulation press Ctrl+c.
 141
 142 If no characters are appearing after about 20 seconds, stop the simulation,
 143 as there might be other issues.
 144
 145 Single-threaded verilator sim binary, on a 2nd gen intel i5 (sandybridge)
 146 takes 53 seconds to print the ASCII lightbulb.
 147
 148 ## Analysing results after simulation
 149
 150 The following files will be generated during the sim:
 151
 152 - `bram.dump` - Shows the PC address and instruction being executed. If the sim
 153 hangs without any printing, view this file, as the processor may have hit an
 154 exception etc. Grows in size as the sim runs.
 155
 156 - `bram.snapshot.[NUMBER]`, `verilator.save.[NUMBER]` - Snapshot files of the
 157 contents of bram and verilator model respectively. Can be used to resume the
 158 simulation. The number on the end corresponds to the tick time (i.e.
 159 `bram.snapshot.1999990`/`verilator.save.1999990`). First the verilator model is
 160 loaded, and then the bram contents are loaded. See lines `#65-108` and
 161 `#189-195` of the
 162 [microwatt-verilator.cpp file](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=verilator/microwatt-verilator.cpp;h=a226393f6ba74d5e3e1ffdb729d731d2311d53ad;hb=refs/heads/verilator_trace).
 163 Pass the tick number on the end of the filename with the '-s' flag:
 164
 165     (microwatt):$ time ./microwatt-verilator hello_world/hello_world.bin -s 1999990
 166
 167 You'll get a message like this:
 168
 169     loading hello_world/hello_world.bin at 0x0 size 0x1888
 170     loading bram.snapshot.1999990 at 0x0 size 0x10000000
 171     restored at 1999990
 172
 173 These snapshots are generated at intervals of every 2,000,000 ticks.
 174
 175 - `microwatt-verilator.vcd` - GTKWave waveform file, allowing you to look at
 176 processor signals and transitions during simulation.
 177 Pass `-d` flag to `microwatt-verilator` binary:
 178
 179     (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -d
 180
 181 **NOTE**: Trace dumping will generate a large VCD file (about 6GB for the hello
 182 world example)!
 183
 184 If you want GTKWave to load it faster, convert to fst first:
 185
 186     (microwatt):$ vcd2fst --vcdname=microwatt-verilator.vcd --fstname=microwatt-verilator.fst
 187     (microwatt):$ gtkwave microwatt-verilator.fst
 188
 189 ## Micropython
 190
 191 The microwatt repo comes with a pre-compiled micropython binary (version 1.12),
 192 which you can try out after confirming 'hello world' works. Bear in mind, not
 193 all features of python will be available. Such as floating-point numbers.
 194
 195 For micropython to work, you'll need to increase the RAM size in the makefile.
 196 Go to the microwatt-verilator makefile, and comment out the following lines:
 197
 198     MEMORY_SIZE=8192
 199     RAM_INIT_FILE=hello_world/hello_world.hex
 200
 201 And uncomment the following:
 202
 203     MEMORY_SIZE=393216
 204     RAM_INIT_FILE=micropython/firmware.hex
 205
 206 This will increase the RAM size from 8KiB to 384KiB. The `RAM_INIT_FILE` in
 207 these examples isn't doing anything, however good practice to follow.
 208
 209 Clean up generated files, and recompile:
 210
 211     (microwatt):$ make clean
 212     (microwatt):$ make microwatt-verilator
 213
 214 Once the binary has been built, run the same way as before, but point to the
 215 micropython firmware binary:
 216
 217     (microwatt):$ microwatt-verilator micropython/firmware.bin
 218
 219 On the same system as above, with 1 thread, it took 49 seconds to get to the
 220 micropython shell.
 221
 222 ## Verilator runtime commands
 223 A few examples:
 224
 225     # Show the version of verilator being used
 226     (microwatt):$ ./microwatt-verilator +verilator+version
 227