X-Git-Url: https://git.libre-soc.org/?a=blobdiff_plain;f=HDL_workflow%2Fmicrowatt.mdwn;h=3b568763be16b178085a7a2edde37eb5c70d74ef;hb=112d5247fa47aa15e039510263e5a74c9ee856d9;hp=ab233df5c94e19cdf212fc500c362a5710fcd79e;hpb=24935df55467e735a8185549324f4dfb67073df8;p=libreriscv.git diff --git a/HDL_workflow/microwatt.mdwn b/HDL_workflow/microwatt.mdwn index ab233df5c..3b568763b 100644 --- a/HDL_workflow/microwatt.mdwn +++ b/HDL_workflow/microwatt.mdwn @@ -1,31 +1,325 @@ -# Build instructions for Microwatt +# Tutorial for setting up Microwatt chroot and running simulations -WIP integrate from +Useful Links (External): -* install verilator: -* install yosys -* install ghdl-yosys-plugin +* +* +* +* [Verilator docs, commands](https://verilator.org/guide/latest/exe_verilator.html) +* [Verilator runtime command documentation](https://verilator.org/guide/latest/exe_sim.html) +* Tutorials for how to work with verilator: +[part1](https://www.itsembedded.com/dhd/verilator_1/), +[part2](https://www.itsembedded.com/dhd/verilator_2/) -After installing ghdl-yosys-plugin use schroot to switch environment +Useful links (Libre-SOC): - schroot -c bullseye_ghdl - git clone https://git.libre-soc.org/git/microwatt.git - git checkout -b verilator_trace - export FPGA_TARGET=verilator - make microwatt-verilator +* Libre-SOC page covering our workflow: [[HDL_workflow]] +* Devscripts Libre-SOC page: [[devscripts]] +* Original Microwatt Libre-SOC page: [[microwatt]] +* [Libre-SOC Microwatt repo branch](https://git.libre-soc.org/?p=microwatt.git;a=tree;hb=refs/heads/verilator_trace) +* [Libre-SOC devscripts repo](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) + +Other Tutorials (Libre-SOC): + +* First steps for working with PowerISA instructions Libre-SOC page: +[[/docs/firststeps]] + +## Video Tutorial + +[43min tutorial](https://youtu.be/02LCl3ang8g) was made and uploaded to +Youtube, covering some of the material you'll find on this page. + +## Development environment scripts + +If you haven't already, clone Libre-SOC's development environment setup scripts. +These are bash scripts, and greatly simplify the time it takes to create a: + +- Stable environment +- With all software and libraries at specific versions +(which are known to work). + +These attributes are absolutely critical, and no support will be +provided, unless you use these scripts to setup a development environment. This +helps us fix any bugs in the scripts, and make sure everyone runs on the same +page. + + $ git clone https://git.libre-soc.org/git/dev-env-setup.git + + +Do *look through* the +[code](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) before running +any of those scripts. +This is your own legal responsibility (to not run +arbitrary code off of the internet) and we take no responsibility or accept +any liability whatsoever for your use or misuse of them. + +It is expected for you to use Debian for the host OS (anything else +is unsupported: many contributors have repeatedly gotten into trouble by not +following this advice), while all the chroots - which are developed +very specifically for "reproducible builds" - run Debian 10 (Buster). + +## Setting up chroot + +Scripts we will be using for the setup are: + +* `mk-deb-chroot`, `cp-scripts-to-chroot` for chroot setup +* `install-hdl-apt-reqs`, `verilator-install`, `hdl-tools-yosys` for working +with Microwatt + +(*Current limitation for `mk-deb-chroot`, is that you must be the first user on +the host machine, having user ID 1000.*) + +Commands to run in terminal to setup a new chroot environment for microwatt +simulations. + + $ cd dev-env-setup + $ sudo bash + # ./mk-deb-chroot microwatt + # ./cp-scripts-to-chroot microwatt + # exit + $ schroot -c microwatt + (microwatt):$ cd dev-env-setup + (microwatt):$ sudo bash + (microwatt):# ./install-hdl-apt-reqs + (microwatt):# ./verilator-install + (microwatt):# ./hdl-tools-yosys + (microwatt):# exit + (microwatt):$ cd ~/src/ + (microwatt):$ git clone https://git.libre-soc.org/git/microwatt.git + (microwatt):$ cd microwatt + (microwatt):$ git checkout verilator_trace + +Make sure verilator binaries in $PATH: + + (microwatt):$ export PATH=/usr/local/verilator/bin:$PATH + (microwatt):$ export GHDLSYNTH=ghdl + +(GHDLSYNTH needs to be redefined because the Makefile has default `ghdl.so`, +but somewhere else '.so' gets appended. You may see the following error if you +don't redefine: +`ERROR: Can't load module +./ghdl.so':/usr/local/bin/../share/yosys/plugins/**ghdl.so.so**`) +[IRC](https://libre-soc.org/irclog/%23libre-soc.2023-01-25.log.html#t2023-01-25T11:10:47) + +## Compiling the verilator sim for Microwatt + +* [Libre-SOC Microwatt repo branch, Makefile](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=Makefile;hb=refs/heads/verilator_trace) + +Verilator creates a fairly fast simulation by converting the HDL design to C++, +and then compiling a binary which the user runs. + +To compile the verilator simulation, first set verilator as the target for the +makefile: + + (microwatt):$ export FPGA_TARGET=verilator + +Before compiling, you can change the `THREADS` variable in the makefile, which +will allow the compiled verilator simulation binary to use more than 1 thread +(*make sure to check how many CPU threads you have before changing this!*) + +To compile the verilator simulation binary, call make with the +`microwatt-verilator` rule. + + (microwatt):$ make microwatt-verilator + +## Compiling hello world code + +We need some code to actually run on the core, so start with the 'hello world'. +Instructions assume you're still in the microwatt directory. + + (microwatt):$ cd hello_world + (microwatt):$ make + +A `hello_world.bin` should be generated (the final binary to be loaded), as +well as an +[.elf file](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format), and +.hex (representing the binary data as hex text strings). + +To view the symbol table (useful to see where various sections of the binary +begin): + + (microwatt):$ powerpc64le-linux-gnu-objdump -h hello_world.elf + (microwatt):$ powerpc64le-linux-gnu-objdump -x hello_world.elf + +`-h` shows just the section headers, `-x` shows all headers. + +And to view the disassembly (great for learning about the PowerISA instructions, +and for associating the binary hex with actual instructions), you can view the +automatically generated `hello_world.as` file. + +Command to generate the disassembly: + + (microwatt):$ powerpc64le-linux-gnu-objdump -D hello_world.elf + +For more information about `objdump` (common utility, not just for PowerISA), +see the manual pages. + + (microwatt):$ man powerpc64le-linux-gnu-objdump + +The binary is ready to go, now it can be loaded into the simulation. + +## Simulation + +### Command line args + +To find out the `microwatt-verilator` arguments, you can check with `-h` arg: + + (microwatt):$ ./microwatt-verilator -h + +Some of the arguments are explained in further sections. + +### Running + +Run the `microwatt-verilator` binary, with `hello_world/hello_world.bin` as an +argument: + + (microwatt):$ time ./microwatt-verilator hello_world/hello_world.bin + +`time` is a utility you can use to measure how long it takes to run the sim. + +A pretty ASCII art of a lightbulb should be printed, and then the user can type +any characters, which will be echoed back. To end the simulation press Ctrl+C. + +If no characters are appearing after about 20 seconds, stop the simulation, +as there might be other issues. + +Single-threaded verilator sim binary, on a 2nd gen intel i5 (sandybridge) +takes 53 seconds to print the ASCII lightbulb. + +On another dev's machine, ASUS KGPE D16, this takes just over a minute. + +(*You'll find that uart printout is one of the longer parts of the simulation +in general.*) + +## Analysing results after simulation + +The following files will be generated during the sim: + +- `bram.dump` - Shows the PC address and instruction being executed. If the sim +hangs without any printing, view this file, as the processor may have hit an +exception etc. Grows in size as the sim runs. + +- `bram.snapshot.[NUMBER]`, `verilator.save.[NUMBER]` - Snapshot files of the +contents of bram and verilator model respectively. Can be used to resume the +simulation. The number on the end corresponds to the tick time (i.e. +`bram.snapshot.1999990`/`verilator.save.1999990`). First the verilator model is +loaded, and then the bram contents are loaded. See lines `#65-108` and +`#189-195` of the +[microwatt-verilator.cpp file](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=verilator/microwatt-verilator.cpp;h=a226393f6ba74d5e3e1ffdb729d731d2311d53ad;hb=refs/heads/verilator_trace). +Pass the tick number on the end of the filename with the '-s' flag: + + (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -s 1999990 + +You'll get a message like this: + + loading hello_world/hello_world.bin at 0x0 size 0x1888 + loading bram.snapshot.1999990 at 0x0 size 0x10000000 + restored at 1999990 + +These snapshots are generated at intervals of every 2,000,000 ticks. + +- `microwatt-verilator.vcd` - GTKWave waveform file, allowing you to look at +processor signals and transitions during simulation. +Pass `-d` flag to `microwatt-verilator` binary: + + (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -d + +**NOTE**: Trace dumping will generate a large VCD file (about 6GB for the hello +world example)! + +If you want GTKWave to load it faster, convert to fst first: + + (microwatt):$ vcd2fst --vcdname=microwatt-verilator.vcd --fstname=microwatt-verilator.fst + (microwatt):$ gtkwave microwatt-verilator.fst + +Fst files are orders-of-magnitude smaller (about 20MB vs 6GB), but are specific +to the GTKWave tool. + +## Micropython + +The Microwatt repo comes with a pre-compiled +[micropython binary](https://git.libre-soc.org/?p=microwatt.git;a=tree;f=micropython;h=18fa078c8145bdaa75667a0ab04eb0b261245665;hb=refs/heads/verilator_trace) +(version 1.12), which you can try out after confirming 'hello world' works. +Bear in mind, not all features of python will be available. Such as +floating-point numbers. -Then run the resulting binary +For micropython to work, you'll need to increase the RAM size in the makefile. +Go to the microwatt-verilator makefile, and comment out the following lines: - time ./microwatt-verilator + MEMORY_SIZE=8192 + RAM_INIT_FILE=hello_world/hello_world.hex -If everything works correcly it will print out an light bulb in ascii art -with the message Microwatt, it works. On my ASUS KGPE D16 this takes +And uncomment the following: + + MEMORY_SIZE=393216 + RAM_INIT_FILE=micropython/firmware.hex + +This will increase the RAM size from 8KiB to 384KiB. The `RAM_INIT_FILE` in +these examples isn't doing anything, however good practice to follow. + +Clean up generated files, and recompile: + + (microwatt):$ make clean + (microwatt):$ make microwatt-verilator + +Once the binary has been built, run the same way as before, but point to the +micropython firmware binary: + + (microwatt):$ microwatt-verilator micropython/firmware.bin + +On the same system as above, with 1 thread, it took 49 seconds to get to the +micropython shell. + +## Verilator runtime commands +A few examples: + + # Show the version of verilator being used + (microwatt):$ ./microwatt-verilator +verilator+version + +## Building `microwatt-verilator` using the Libre-SOC core + +In the Makefile, you need to set `EXTERNAL_CORE` to true, and copy the +generated core from soc repo to microwatt. *(If you use a separate chroot to +generate Libre-SOC cores, then you'll need to copy from that chroot to +microwatt chroot from host.* + + cd /path/to/soc + make microwatt_external_core + cp external_core_top.v /path/to/microwatt + +Then compile verilator sim binary as before: + + cd ~/src/microwatt/ + export FPGA_TARGET=verilator + export GHDLSYNTH=ghdl + make microwatt-verilator + +## Running Linux kernel + +To run Linux on Microwatt, you'll need two binaries: + +- The `sdram_init.bin`, which is easy to compile (no additional software +required). + +- The `dtbImage.microwatt` device tree Linux kernel. This can be compiled (see +below), or a copy can be downloaded from: . + +Like with the micropython example, you'll need to increase the RAM size in the +makefile, and recompile the microwatt-verilator binary. + +Uncomment the following: + + MEMORY_SIZE=536870912 + +Which will change the RAM size to 512KiB. + +As there is no `dtbImage.microwatt.hex`, you can leave `RAM_INIT_FILE` unchanged. + +### Building the kernel - TODO: + +*(Please don't build the kernel yourself, until you've tested with the existing kernel linked +above!)* - real 1m4.986s - user 1m4.373s - sys 0m0.013s - -## Building the kernel On a POWER9 there is no need to install gcc-powerpc64le-linux-gnu, you can omit CROSS_COMPILE and ARCH in this case @@ -35,28 +329,140 @@ you can omit CROSS_COMPILE and ARCH in this case cd microwatt wget https://ftp.libre-soc.org/microwatt-linux-5.7.patch patch -p1 < microwatt-linux-5.7.patch - wget http://ftp.libre-soc.org/rootfs.cpio + wget https://ftp.libre-soc.org/rootfs.cpio CROSS_COMPILE="ccache powerpc64le-linux-gnu-" ARCH=powerpc make -j8 O=microwatt microwatt_defconfig CROSS_COMPILE="ccache powerpc64le-linux-gnu-" ARCH=powerpc make -j8 O=microwatt This will produce a file microwatt/arch/powerpc/boot/dtbImage.microwatt -## Building sdram_init.bin -This needs gcc-powerpc64le-linux-gnu if cross compilation is used. +### Building `sdram_init.bin` +This needs gcc-powerpc64le-linux-gnu (already included in the setup step) if +cross compilation is used. It is assumed you're already in `~/src/microwatt/` +directory. + + (microwatt):$ cd litedram/gen-src/sdram_init/ + (microwatt):$ make + +The resulting binary will be in the `obj/` directory. + +### Running the simulation + +Make sure to return back to `src/microwatt/`. + + (microwatt):$ cd ~/src/microwatt/ + (microwatt):$ cp microwatt/arch/powerpc/boot/dtbImage.microwatt + (microwatt):$ ./microwatt-verilator sdram_init.bin dtbImage.microwatt + +This will take some time... + +### Sysconn information + +TODO WIP integrate from + +Sysconn is a module which includes information about the SoC, and the info is +printed at the start of the simulation. + +### Time benchmarks + +`microwatt-verilator` was compiled with 3 threads for faster simulation. + +- Time to finish printing Sysconn info: about 1min +- Time to allocate bytes to kernel: ? +- Time to login prompt: about 1 hour +- Time to user shell: ? + +### TODO: buildroot + +* https://github.com/shenki/buildroot/commits/microwatt +* https://codeconstruct.com.au/docs/microwatt-orangecrab/ + +## FPGA Development - TODO: Need checking +### Building the bitstring for OrangeCrab + cd microwatt - cd litedram/gen-src/sdram_init/ - make - -## Building microwatt-verilator using the libre-soc core - cd /path/to/soc - make microwatt_external_core - cp external_core_top.v /path/to/microwatt - cd /path/to/microwatt - export FPGA_TARGET=verilator + export FPGA_TARGET=ORANGE-CRAB export GHDLSYNTH=ghdl - make microwatt-verilator + make microwatt.bit + +### flashing the bitstring to the OrangeCrab + + make prog # this will run OpenOCD + +### Notes for ulx3s + +notes for how to compile for ulx3s + + git clone https://github.com/kost/fujprog + (follow build procedure shown in fujprog README) + git clone https://git.libre-soc.org/git/microwatt.git + git checkout -b verilator_trace + export FPGA_TARGET=ulx3s + make microwatt.svf + fujprog microwatt.svf + + +### Notes for nextpnr-xilinx + +superceded: see page [[nextpnr-xilinx]] and devscript + + +for compiling nextpnr-xilinx and making it useable for nmigen +to compile for the digilent arty-a7-100t, requires a little +futzing around, using the symbiflow version of prjxray-db +instead of the one recommended as a submodule + + git clone https://github.com/gatecat/nextpnr-xilinx + cd nextpnr-xilinx + git checkout cd8b15db6ff5c1a7f10a9e + git submodule init + git submodule update + cd xilinx/external + mv prjxray-db prjxray-db-no + git clone https://github.com/SymbiFlow/prjxray-db + cd prjxray-db + git checkout 0a0addedd73e7 + cp ./artix7/xc7a100t/*.json \ + ./artix7/xc7a100tcsg324-1 + cd ../../.. + cmake -DARCH=xilinx . + make + make install + python3 xilinx/python/bbaexport.py --device xc7a100tcsg324-1 --bba xilinx/xc7a100t.bba + ./bbasm --l xilinx/xc7a100t.bba xilinx/xc7a100t.bin + mkdir -p /usr/share/nextpnr/xilinx-chipdb + cp xilinx/*.bin /usr/share/nextpnr/xilinx-chipdb + cp -aux xilinx/external/prjxray-db /usr/share/nextpnr + +# Additional Useful Info for UART <-> USB Serial Interface Through OrangeCrab's Built-in USB Interface + +This uses OrangeCrab's built-in USB interface, rather than needing a +separate USB-serial adapter. see the following for further details: + +* +* + +# running orangecrab-examples before flashing microwatt + +See + +If the OrangeCrab is running in DFU mode, lsusb will show: + + 1209:5af0 Generic OrangeCrab r0.2 DFU Bootloader v3.1-6-g62e92e2 + +OrangeCrab has two DFU devices: + + Found DFU: [1209:5af0] ver=0101, devnum=22, cfg=1, intf=0, path="1-4.2", alt=1, name="0x00100000 RISC-V Firmware", serial="UNKNOWN" + Found DFU: [1209:5af0] ver=0101, devnum=22, cfg=1, intf=0, path="1-4.2", alt=0, name="0x00080000 Bitstream", serial="UNKNOWN" + +Then clone and patch orangecrab-examples: + + git clone https://github.com/orangecrab-fpga/orangecrab-examples + patch -p1 < orangecrab-examples.diff + +To build and flash the example: -## Running the simulation -Copy microwatt/arch/powerpc/boot/dtbImage.microwatt and sdram_init.bin into the microwatt directory. -./microwatt-verilator sdram_init.bin dtbImage.microwatt + pushd orangecrab-examples/nmigen + python3 blink.py + popd + sudo dfu-util -D orangecrab-examples/nmigen/build/top.bit -a 0