From 8b116bebce826656da5c110a2e7d4e8f7fa4bfef Mon Sep 17 00:00:00 2001 From: Andrey Miroshnikov Date: Tue, 9 May 2023 13:17:37 +0100 Subject: [PATCH] microwatt: Merged the tutorial documentation from recent dev (bug #1073) microwatt_tutorial: Removed as merged with old page. --- HDL_workflow/microwatt.mdwn | 325 +++++++++++++++++++++++---- HDL_workflow/microwatt_tutorial.mdwn | 227 ------------------- 2 files changed, 277 insertions(+), 275 deletions(-) delete mode 100644 HDL_workflow/microwatt_tutorial.mdwn diff --git a/HDL_workflow/microwatt.mdwn b/HDL_workflow/microwatt.mdwn index 0d36c0ce0..f08578743 100644 --- a/HDL_workflow/microwatt.mdwn +++ b/HDL_workflow/microwatt.mdwn @@ -1,40 +1,280 @@ -# Build instructions for Microwatt +# Tutorial for setting up Microwatt chroot and running simulations TODO WIP integrate from -useful links: - +Useful Links (External): * * * -* New tutorial for Microwatt page (IN-PROGRESS): [[microwatt_tutorial]] +* [Verilator docs, commands](https://verilator.org/guide/latest/exe_verilator.html) +* [Verilator runtime command documentation](https://verilator.org/guide/latest/exe_sim.html) +* Tutorials for how to work with verilator: +[part1](https://www.itsembedded.com/dhd/verilator_1/), +[part2](https://www.itsembedded.com/dhd/verilator_2/) -prerequisite tools: +Useful links (Libre-SOC): +* Libre-SOC page covering our workflow: [[HDL_workflow]] +* Devscripts Libre-SOC page: [[devscripts]] +* Original Microwatt Libre-SOC page: [[microwatt]] +* [Libre-SOC Microwatt repo branch](https://git.libre-soc.org/?p=microwatt.git;a=tree;hb=refs/heads/verilator_trace) +* [Libre-SOC devscripts repo](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) -* install verilator: -* install yosys -* install ghdl-yosys-plugin +Other Tutorials (Libre-SOC): +* First steps for working with PowerISA instructions Libre-SOC page: +[[/docs/firststeps]] -After installing ghdl-yosys-plugin use schroot to switch environment +## Development environment scripts - schroot -c bullseye_ghdl - git clone https://git.libre-soc.org/git/microwatt.git - git checkout verilator_trace - export FPGA_TARGET=verilator - make microwatt-verilator +If you haven't already, clone Libre-SOC's development environment setup scripts. +These are bash scripts, and greatly simplify the time it takes to create a: + +- Stable environment +- With all software and libraries at specific versions +(which are known to work). + +These attributes are absolutely critical, and no support will be +provided, unless you use these scripts to setup a development environment. This +helps us fix any bugs in the scripts, and make sure everyone runs on the same +page. + + $ git clone https://git.libre-soc.org/git/dev-env-setup.git + +Do *look through* the +[code](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) before running +any of those scripts. They may be confusing, however after reading a few you'll +start to become more familiar with them. + +It is expected for you to use Debian, we mostly use 11 (Bullseye) for the host +OS, while all the chroots run Debian 10 (Buster). + +## Setting up chroot + +Scripts we will be using for the setup are: + +* `mk-deb-chroot`, `cp-scripts-to-chroot` for chroot setup +* `install-hdl-apt-reqs`, `verilator-install`, `hdl-tools-yosys` for working +with Microwatt + +(*Current limitation for `mk-deb-chroot`, is that you must be the first user on +the host machine, having user ID 1000.*) + +Commands to run in terminal to setup a new chroot environment for microwatt +simulations. + + $ cd dev-env-setup + $ sudo bash + # ./mk-deb-chroot microwatt + # ./cp-scripts-to-chroot microwatt + # exit + $ schroot -c microwatt + (microwatt):$ cd dev-env-setup + (microwatt):$ sudo bash + (microwatt):# ./install-hdl-apt-reqs + (microwatt):# ./verilator-install + (microwatt):# ./hdl-tools-yosys + (microwatt):# exit + (microwatt):$ cd ~/src/ + (microwatt):$ git clone https://git.libre-soc.org/git/microwatt.git + (microwatt):$ cd microwatt + (microwatt):$ git checkout verilator_trace + +Make sure verilator binaries in $PATH: + + (microwatt):$ export PATH=/usr/local/verilator/bin:$PATH + (microwatt):$ export GHDLSYNTH=ghdl + +(GHDLSYNTH needs to be redefined because the Makefile has default `ghdl.so`, +but somewhere else '.so' gets appended. You may see the following error if you +don't redefine: +`ERROR: Can't load module +./ghdl.so':/usr/local/bin/../share/yosys/plugins/**ghdl.so.so**`) +[IRC](https://libre-soc.org/irclog/%23libre-soc.2023-01-25.log.html#t2023-01-25T11:10:47) + +## Compiling the verilator sim for Microwatt + +* [Libre-SOC Microwatt repo branch, Makefile](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=Makefile;hb=refs/heads/verilator_trace) + +Verilator creates a fairly fast simulation by converting the HDL design to C++, +and then compiling a binary which the user runs. + +To compile the verilator simulation, first set verilator as the target for the +makefile: + + (microwatt):$ export FPGA_TARGET=verilator + +Before compiling, you can change the `THREADS` variable in the makefile, which +will allow the compiled verilator simulation binary to use more than 1 thread +(*make sure to check how many CPU threads you have before changing this!*) + +To compile the verilator simulation binary, call make with the +`microwatt-verilator` rule. + + (microwatt):$ make microwatt-verilator + +## Compiling hello world code + +We need some to actually run on the core, so...? +Instructions assume you're still in the microwatt directory. + + (microwatt):$ cd hello_world + (microwatt):$ make + +A `hello_world.bin` should be generated (the final binary to be loaded), as +well as an +[.elf file](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format), and +.hex (representing the binary data as hex text strings). + +To view the symbol table (useful to see where various sections of the binary +begin): + + (microwatt):$ powerpc64le-linux-gnu-objdump -h hello_world.elf + (microwatt):$ powerpc64le-linux-gnu-objdump -x hello_world.elf + +`-h` shows just the section headers, `-x` shows all headers. + +And to view the disassembly (great for learning about the PowerISA instructions, +and for associating the binary hex with actual instructions): + + (microwatt):$ powerpc64le-linux-gnu-objdump -d hello_world.elf + +For more information about `objdump` (common utility, not just for PowerISA), +see the manual pages. + + (microwatt):$ man powerpc64le-linux-gnu-objdump + +The binary is ready to go, now it can be loaded into the simulation. + +## Simulation + +### Command line args + +To find out the `microwatt-verilator` arguments, you can check with `-h` arg: + + (microwatt):$ ./microwatt-verilator -h + +Some of the arguments are explained in further sections. + +### Running + +Run the `microwatt-verilator` binary, with `hello_world/hello_world.bin` as an +argument: + + (microwatt):$ time ./microwatt-verilator hello_world/hello_world.bin + +`time` is a utility you can use to measure how long it takes to run the sim. + +A pretty ASCII art of a lightbulb should be printed, and then the user can type +any characters, which will be echoed back. To end the simulation press Ctrl+C. + +If no characters are appearing after about 20 seconds, stop the simulation, +as there might be other issues. + +Single-threaded verilator sim binary, on a 2nd gen intel i5 (sandybridge) +takes 53 seconds to print the ASCII lightbulb. + +On another dev's machine, ASUS KGPE D16, this takes just over a minute. + +(*You'll find that uart printout is one of the longer parts of the simulation +in general.*) + +## Analysing results after simulation + +The following files will be generated during the sim: + +- `bram.dump` - Shows the PC address and instruction being executed. If the sim +hangs without any printing, view this file, as the processor may have hit an +exception etc. Grows in size as the sim runs. -Then run the resulting binary +- `bram.snapshot.[NUMBER]`, `verilator.save.[NUMBER]` - Snapshot files of the +contents of bram and verilator model respectively. Can be used to resume the +simulation. The number on the end corresponds to the tick time (i.e. +`bram.snapshot.1999990`/`verilator.save.1999990`). First the verilator model is +loaded, and then the bram contents are loaded. See lines `#65-108` and +`#189-195` of the +[microwatt-verilator.cpp file](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=verilator/microwatt-verilator.cpp;h=a226393f6ba74d5e3e1ffdb729d731d2311d53ad;hb=refs/heads/verilator_trace). +Pass the tick number on the end of the filename with the '-s' flag: - time ./microwatt-verilator + (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -s 1999990 -If everything works correcly it will print out an light bulb in ascii art -with the message Microwatt, it works. On my ASUS KGPE D16 this takes +You'll get a message like this: - real 1m4.986s - user 1m4.373s - sys 0m0.013s - -## Building the kernel + loading hello_world/hello_world.bin at 0x0 size 0x1888 + loading bram.snapshot.1999990 at 0x0 size 0x10000000 + restored at 1999990 + +These snapshots are generated at intervals of every 2,000,000 ticks. + +- `microwatt-verilator.vcd` - GTKWave waveform file, allowing you to look at +processor signals and transitions during simulation. +Pass `-d` flag to `microwatt-verilator` binary: + + (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -d + +**NOTE**: Trace dumping will generate a large VCD file (about 6GB for the hello +world example)! + +If you want GTKWave to load it faster, convert to fst first: + + (microwatt):$ vcd2fst --vcdname=microwatt-verilator.vcd --fstname=microwatt-verilator.fst + (microwatt):$ gtkwave microwatt-verilator.fst + +Fst files are orders-of-magnitude smaller (about 20MB vs 6GB), but are specific +to the GTKWave tool. + +## Micropython + +The Microwatt repo comes with a pre-compiled +[micropython binary](https://git.libre-soc.org/?p=microwatt.git;a=tree;f=micropython;h=18fa078c8145bdaa75667a0ab04eb0b261245665;hb=refs/heads/verilator_trace) +(version 1.12), which you can try out after confirming 'hello world' works. +Bear in mind, not all features of python will be available. Such as +floating-point numbers. + +For micropython to work, you'll need to increase the RAM size in the makefile. +Go to the microwatt-verilator makefile, and comment out the following lines: + + MEMORY_SIZE=8192 + RAM_INIT_FILE=hello_world/hello_world.hex + +And uncomment the following: + + MEMORY_SIZE=393216 + RAM_INIT_FILE=micropython/firmware.hex + +This will increase the RAM size from 8KiB to 384KiB. The `RAM_INIT_FILE` in +these examples isn't doing anything, however good practice to follow. + +Clean up generated files, and recompile: + + (microwatt):$ make clean + (microwatt):$ make microwatt-verilator + +Once the binary has been built, run the same way as before, but point to the +micropython firmware binary: + + (microwatt):$ microwatt-verilator micropython/firmware.bin + +On the same system as above, with 1 thread, it took 49 seconds to get to the +micropython shell. + +## Verilator runtime commands +A few examples: + + # Show the version of verilator being used + (microwatt):$ ./microwatt-verilator +verilator+version + +## Building `microwatt-verilator` using the Libre-SOC core + + cd /path/to/soc + make microwatt_external_core + cp external_core_top.v /path/to/microwatt + cd /path/to/microwatt + export FPGA_TARGET=verilator + export GHDLSYNTH=ghdl + make microwatt-verilator + +## Running Linux kernel - TODO: Need to check + +### Building the kernel On a POWER9 there is no need to install gcc-powerpc64le-linux-gnu, you can omit CROSS_COMPILE and ARCH in this case @@ -51,47 +291,36 @@ you can omit CROSS_COMPILE and ARCH in this case This will produce a file microwatt/arch/powerpc/boot/dtbImage.microwatt -## Building sdram_init.bin +### Building sdram_init.bin This needs gcc-powerpc64le-linux-gnu if cross compilation is used. cd microwatt cd litedram/gen-src/sdram_init/ - make - -## Building microwatt-verilator using the libre-soc core - - cd /path/to/soc - make microwatt_external_core - cp external_core_top.v /path/to/microwatt - cd /path/to/microwatt - export FPGA_TARGET=verilator - export GHDLSYNTH=ghdl - make microwatt-verilator + make -## Running the simulation +### Running the simulation cp microwatt/arch/powerpc/boot/dtbImage.microwatt ./microwatt-verilator sdram_init.bin dtbImage.microwatt - -## Building the bitstring for OrangeCrab + +### TODO: buildroot + +* https://github.com/shenki/buildroot/commits/microwatt +* https://codeconstruct.com.au/docs/microwatt-orangecrab/ + +## FPGA Development - TODO: Need checking +### Building the bitstring for OrangeCrab cd microwatt export FPGA_TARGET=ORANGE-CRAB export GHDLSYNTH=ghdl make microwatt.bit -## flashing the bitstring to the OrangeCrab +### flashing the bitstring to the OrangeCrab make prog # this will run OpenOCD - - -## TODO: buildroot - -* https://github.com/shenki/buildroot/commits/microwatt -* https://codeconstruct.com.au/docs/microwatt-orangecrab/ - -# Notes for ulx3s +### Notes for ulx3s notes for how to compile for ulx3s @@ -104,7 +333,7 @@ notes for how to compile for ulx3s fujprog microwatt.svf -# Notes for nextpnr-xilinx +### Notes for nextpnr-xilinx superceded: see page [[nextpnr-xilinx]] and devscript diff --git a/HDL_workflow/microwatt_tutorial.mdwn b/HDL_workflow/microwatt_tutorial.mdwn deleted file mode 100644 index 0ce23ee2f..000000000 --- a/HDL_workflow/microwatt_tutorial.mdwn +++ /dev/null @@ -1,227 +0,0 @@ -# New tutorial for setting up Microwatt chroot and running simulations - -useful links: - -* Libre-SOC page covering our workflow: [[HDL_workflow]] -* Devscripts Libre-SOC page: [[devscripts]] -* Original Microwatt Libre-SOC page: [[microwatt]] -* [Libre-SOC Microwatt repo branch](https://git.libre-soc.org/?p=microwatt.git;a=tree;hb=refs/heads/verilator_trace) -* [Libre-SOC devscripts repo](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) -* [Verilator docs, commands](https://verilator.org/guide/latest/exe_verilator.html) -* [Verilator runtime command documentation](https://verilator.org/guide/latest/exe_sim.html) -* First steps for working with PowerISA instructions Libre-SOC page: -[[/docs/firststeps]] -* Tutorials for how to work with verilator: [part1](https://www.itsembedded.com/dhd/verilator_1/), [part2](https://www.itsembedded.com/dhd/verilator_2/) - -## Development environment scripts - -If you haven't already, clone Libre-SOC's development environment setup scripts. -These are bash scripts, and greatly simplify the time it takes to create a: - -- Stable environment -- With all software and libraries at specific versions -(which are known to work). - -These attributes are absolutely critical, and no support will be -provided, unless you use these scripts to setup a development environment. This -helps us fix any bugs in the scripts, and make sure everyone runs on the same -page. - - $ git clone https://git.libre-soc.org/git/dev-env-setup.git - -Do *look through* the -[code](https://git.libre-soc.org/?p=dev-env-setup.git;a=tree) before running -any of those scripts. They may be confusing, however after reading a few you'll -start to become more familiar with them. - -It is expected for you to use Debian, we mostly use 11 (Bullseye) for the host -OS, while all the chroots run Debian 10 (Buster). - - -## Setting up chroot - -Commands to run in terminal to setup a new chroot environment for microwatt -simulations. - - $ cd dev-env-setup - $ sudo bash - # ./mk-deb-chroot microwatt - # ./cp-scripts-to-chroot microwatt - # exit - $ schroot -c microwatt - (microwatt):$ cd dev-env-setup - (microwatt):$ sudo bash - (microwatt):# ./install-hdl-apt-reqs - (microwatt):# ./verilator-install - (microwatt):# ./hdl-tools-yosys - (microwatt):# apt install gtkwave -y - (microwatt):# exit - (microwatt):$ cd ~/src/ - (microwatt):$ git clone https://git.libre-soc.org/git/microwatt.git - (microwatt):$ cd microwatt - (microwatt):$ git checkout verilator_trace - -Make sure verilator binaries in $PATH: - - (microwatt):$ export PATH=/usr/local/verilator/bin:$PATH - (microwatt):$ export GHDLSYNTH=ghdl - -(GHDLSYNTH needs to be redefined because the Makefile has default `ghdl.so`, -but somewhere else '.so' gets appended. You may see the following error if you -don't redefine: -`ERROR: Can't load module -./ghdl.so':/usr/local/bin/../share/yosys/plugins/**ghdl.so.so**`) -[IRC](https://libre-soc.org/irclog/%23libre-soc.2023-01-25.log.html#t2023-01-25T11:10:47) - -## Compiling the verilator sim for Microwatt - -* [Libre-SOC Microwatt repo branch, Makefile](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=Makefile;h=b764793fcdf5409be33a01f2440cea3717cb2a32;hb=refs/heads/verilator_trace) - -Verilator creates a fairly fast simulation by converting the HDL design to C++, -and then compiling a binary which the user runs. - -To compile the verilator simulation, first set verilator as the target for the -makefile: - - (microwatt):$ export FPGA_TARGET=verilator - -Before compiling, you can change the `THREADS` variable in the makefile, which -will allow the compiled verilator simulation binary to use more than 1 thread -(*make sure to check how many CPU threads you have before changing this!*) - -To compile the verilator simulation binary, call make with the -`microwatt-verilator` rule. - - (microwatt):$ make microwatt-verilator - - - -## Compiling hello world code - -We need some to actually run on the core, so...? -Instructions assume you're still in the microwatt directory. - - (microwatt):$ cd hello_world - (microwatt):$ make - -A `hello_world.bin` should be generated (the final binary to be loaded), as -well as an [.elf file](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format), -and .hex (representing the binary data as hex text strings). - -To view the symbol table (useful to see where various sections of the binary -begin): - - (microwatt):$ powerpc64le-linux-gnu-objdump -h hello_world.elf - (microwatt):$ powerpc64le-linux-gnu-objdump -x hello_world.elf - -`-h` shows just the section headers, `-x` shows all headers. - -And to view the disassembly (great for learning about the PowerISA instructions, -and for associating the binary hex with actual instructions): - - (microwatt):$ powerpc64le-linux-gnu-objdump -d hello_world.elf - -For more information about `objdump` (common utility, not just for PowerISA), -see the manual pages. - - (microwatt):$ man powerpc64le-linux-gnu-objdump - -The binary is ready to go, now it can be loaded into the simulation. - -## Running the simulation - -Run the `microwatt-verilator` binary, with `hello_world/hello_world.bin` as an -argument: - - (microwatt):$ time ./microwatt-verilator hello_world/hello_world.bin - -`time` is a utility you can use to measure how long it takes to run the sim. - -A pretty ASCII art of a lightbulb should be printed, and then the user can type any characters, which will be echoed back. To end the simulation press Ctrl+c. - -If no characters are appearing after about 20 seconds, stop the simulation, -as there might be other issues. - -Single-threaded verilator sim binary, on a 2nd gen intel i5 (sandybridge) -takes 53 seconds to print the ASCII lightbulb. - -## Analysing results after simulation - -The following files will be generated during the sim: - -- `bram.dump` - Shows the PC address and instruction being executed. If the sim -hangs without any printing, view this file, as the processor may have hit an -exception etc. Grows in size as the sim runs. - -- `bram.snapshot.[NUMBER]`, `verilator.save.[NUMBER]` - Snapshot files of the -contents of bram and verilator model respectively. Can be used to resume the -simulation. The number on the end corresponds to the tick time (i.e. -`bram.snapshot.1999990`/`verilator.save.1999990`). First the verilator model is -loaded, and then the bram contents are loaded. See lines `#65-108` and -`#189-195` of the -[microwatt-verilator.cpp file](https://git.libre-soc.org/?p=microwatt.git;a=blob;f=verilator/microwatt-verilator.cpp;h=a226393f6ba74d5e3e1ffdb729d731d2311d53ad;hb=refs/heads/verilator_trace). -Pass the tick number on the end of the filename with the '-s' flag: - - (microwatt):$ time ./microwatt-verilator hello_world/hello_world.bin -s 1999990 - -You'll get a message like this: - - loading hello_world/hello_world.bin at 0x0 size 0x1888 - loading bram.snapshot.1999990 at 0x0 size 0x10000000 - restored at 1999990 - -These snapshots are generated at intervals of every 2,000,000 ticks. - -- `microwatt-verilator.vcd` - GTKWave waveform file, allowing you to look at -processor signals and transitions during simulation. -Pass `-d` flag to `microwatt-verilator` binary: - - (microwatt):$ ./microwatt-verilator hello_world/hello_world.bin -d - -**NOTE**: Trace dumping will generate a large VCD file (about 6GB for the hello -world example)! - -If you want GTKWave to load it faster, convert to fst first: - - (microwatt):$ vcd2fst --vcdname=microwatt-verilator.vcd --fstname=microwatt-verilator.fst - (microwatt):$ gtkwave microwatt-verilator.fst - -## Micropython - -The microwatt repo comes with a pre-compiled micropython binary (version 1.12), -which you can try out after confirming 'hello world' works. Bear in mind, not -all features of python will be available. Such as floating-point numbers. - -For micropython to work, you'll need to increase the RAM size in the makefile. -Go to the microwatt-verilator makefile, and comment out the following lines: - - MEMORY_SIZE=8192 - RAM_INIT_FILE=hello_world/hello_world.hex - -And uncomment the following: - - MEMORY_SIZE=393216 - RAM_INIT_FILE=micropython/firmware.hex - -This will increase the RAM size from 8KiB to 384KiB. The `RAM_INIT_FILE` in -these examples isn't doing anything, however good practice to follow. - -Clean up generated files, and recompile: - - (microwatt):$ make clean - (microwatt):$ make microwatt-verilator - -Once the binary has been built, run the same way as before, but point to the -micropython firmware binary: - - (microwatt):$ microwatt-verilator micropython/firmware.bin - -On the same system as above, with 1 thread, it took 49 seconds to get to the -micropython shell. - -## Verilator runtime commands -A few examples: - - # Show the version of verilator being used - (microwatt):$ ./microwatt-verilator +verilator+version - -- 2.30.2