# Resources and Specifications

This page aims to collect all the resources and specifications we need
in one place for quick access. We will try our best to keep links here
up-to-date. Feel free to add more links here.

[[!toc  ]]

# Getting Started

This section is primarily a series of useful links found online

* [FSiC2019](https://wiki.f-si.org/index.php/FSiC2019)
* Fundamentals to learn to get started [[3d_gpu/tutorial]]

## Is Open Source Hardware Profitable?
[RaptorCS on FOSS Hardware Interview](https://www.youtube.com/watch?v=o5Ihqg72T3c&feature=youtu.be)

# OpenPOWER ISA

* [3.0 PDF](https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0)
* [2.07 PDF](https://openpowerfoundation.org/?resource_lib=ibm-power-isa-version-2-07-b)

## Overview of the user ISA:

[Raymond Chen's PowerPC series](https://devblogs.microsoft.com/oldnewthing/20180806-00/?p=99425)

## OpenPOWER OpenFSI Spec (2016)

* [OpenPOWER OpenFSI Spec](http://openpowerfoundation.org/wp-content/uploads/resources/OpenFSI-spec-100/OpenFSI-spec-20161212.pdf)

* [OpenPOWER OpenFSI Compliance Spec](http://openpowerfoundation.org/wp-content/uploads/resources/openpower-fsi-thts-1.0/openpower-fsi-thts-20180130.pdf)

# Communities

* <https://www.reddit.com/r/OpenPOWER/>
* <http://lists.mailinglist.openpowerfoundation.org/pipermail/openpower-hdl-cores/>
* <http://lists.mailinglist.openpowerfoundation.org/pipermail/openpower-community-dev/>


# JTAG

* [Useful JTAG implementation reference: Design Of IEEE 1149.1 TAP Controller IP Core by Shelja, Nandakumar and Muruganantham, DOI:10.5121/csit.2016.60910](https://web.archive.org/web/20201021174944/https://airccj.org/CSCP/vol6/csit65610.pdf)

    Abstract

    "The objective of this work is to design and implement a TAP controller IP core compatible with IEEE 1149.1-2013 revision of the standard. The test logic architecture also includes the Test Mode Persistence controller and its associated logic. This work is expected to serve as a ready to use module that can be directly inserted in to a new digital IC designs with little modifications."

# RISC-V Instruction Set Architecture

**PLEASE UPDATE** - we are no longer implementing full RISCV, only user-space
RISCV

The Libre RISC-V Project is building a hybrid CPU/GPU SoC. As the name
of the project implies, we will be following the RISC-V ISA I due to it
being open-source and also because of the huge software and hardware
ecosystem building around it. There are other open-source ISAs but none
of them have the same momentum and energy behind it as RISC-V.

To fully take advantage of the RISC-V ecosystem, it is important to be
compliant with the RISC-V standards. Doing so will allow us to to reuse
most software as-is and avoid major forks.

* [Official compiled PDFs of RISC-V ISA Manual]
 (https://github.com/riscv/riscv-isa-manual/releases/latest)
* [Working draft of the proposed RISC-V Bitmanipulation extension](https://github.com/riscv/riscv-bitmanip/blob/master/bitmanip-draft.pdf)
* [RISC-V "V" Vector Extension](https://riscv.github.io/documents/riscv-v-spec/)
* [RISC-V Supervisor Binary Interface Specification](https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md)

Note: As far as I know, we aren't using the RISC-V V Extension directly
at the moment (correction: we were never going to). However, there are many wiki pages that make a reference
to the V extension so it would be good to include it here as a reference
for comparative/informative purposes with regard to Simple-V.
<https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc#vsetvlivsetvl-instructions>

# Radix MMU
 - [Qemu emulation](https://github.com/qemu/qemu/commit/d5fee0bbe68d5e61e2d2beb5ff6de0b9c1cfd182)

# D-Cache

## D-Cache Possible Optimizations papers and links
- [ACDC: Small, Predictable and High-Performance Data Cache](https://dl.acm.org/doi/10.1145/2677093)

# BW Enhancing Shared L1 Cache Design research done in cooperation with AMD
- [Youtube video PACT 2020 - Analyzing and Leveraging Shared L1 Caches in GPUs](https://m.youtube.com/watch?v=CGIhOnt7F6s)
- [Url to PDF of paper on author's website (clicking will download the pdf)](https://adwaitjog.github.io/docs/pdf/sharedl1-pact20.pdf)


# RTL Arithmetic SQRT, FPU etc.

## Wallace vs Dadda Multipliers

* [Paper comparing efficiency of Wallace and Dadda Multipliers in RTL implementations (clicking will download the pdf from archive.org)](https://web.archive.org/web/20180717013227/http://ieeemilestones.ethw.org/images/d/db/A_comparison_of_Dadda_and_Wallace_multiplier_delays.pdf)

## Sqrt
* [Fast Floating Point Square Root](https://pdfs.semanticscholar.org/5060/4e9aff0e37089c4ab9a376c3f35761ffe28b.pdf)
* [Reciprocal Square Root Algorithm](http://www.acsel-lab.com/arithmetic/arith15/papers/ARITH15_Takagi.pdf)

## CORDIC and related algorithms

* <https://bugs.libre-soc.org/show_bug.cgi?id=127> research into CORDIC
* <https://bugs.libre-soc.org/show_bug.cgi?id=208>
* [BKM (log(x) and e^x)](https://en.wikipedia.org/wiki/BKM_algorithm)
* [CORDIC](http://www.andraka.com/files/crdcsrvy.pdf)
 - Does not have an easy way of computing tan(x)
* [zipcpu CORDIC](https://zipcpu.com/dsp/2017/08/30/cordic.html)
* [Low latency and Low error floating point TCORDIC](https://ieeexplore.ieee.org/document/7784797) (email Michael or Cole if you don't have IEEE access)
* <http://www.myhdl.org/docs/examples/sinecomp/> MyHDL version of CORDIC
* <https://dspguru.com/dsp/faqs/cordic/>

## IEEE Standard for Floating-Point Arithmetic (IEEE 754)

Almost all modern computers follow the IEEE Floating-Point Standard. Of
course, we will follow it as well for interoperability.

* IEEE 754-2019: <https://standards.ieee.org/standard/754-2019.html>

Note: Even though this is such an important standard used by everyone,
it is unfortunately not freely available and requires a payment to
access. However, each of the Libre RISC-V members already have access
to the document.

* [Lecture notes - Floating Point Appreciation](http://pages.cs.wisc.edu/~markhill/cs354/Fall2008/notes/flpt.apprec.html)

Among other things, has a nice explanation on arithmetic, rounding modes and the sticky bit.

* [What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)

Nice resource on rounding errors (ulps and epsilon) and the "table maker's dilemma".

## Past FPU Mistakes to learn from

* [Intel Underestimates Error Bounds by 1.3 quintillion on 
Random ASCII – tech blog of Bruce Dawson ](https://randomascii.wordpress.com/2014/10/09/intel-underestimates-error-bounds-by-1-3-quintillion/)
* [Intel overstates FPU accuracy 06/01/2013](http://notabs.org/fpuaccuracy)
* How not to design an ISA
 <https://player.vimeo.com/video/450406346>
  Meester Forsyth <http://eelpi.gotdns.org/>
# Khronos Standards

The Khronos Group creates open standards for authoring and acceleration
of graphics, media, and computation. It is a requirement for our hybrid
CPU/GPU to be compliant with these standards *as well* as with IEEE754,
in order to be commercially-competitive in both areas: especially Vulkan
and OpenCL being the most important. SPIR-V is also important for the
Kazan driver.

Thus the [[zfpacc_proposal]] has been created which permits runtime dynamic
switching between different accuracy levels, in userspace applications.

[**SPIR-V Main Page Link**](https://www.khronos.org/registry/spir-v/)

* [SPIR-V 1.5 Specification Revision 1](https://www.khronos.org/registry/spir-v/specs/unified1/SPIRV.html)
* [SPIR-V OpenCL Extended Instruction Set](https://www.khronos.org/registry/spir-v/specs/unified1/OpenCL.ExtendedInstructionSet.100.html)
* [SPIR-V GLSL Extended Instruction Set](https://www.khronos.org/registry/spir-v/specs/unified1/GLSL.std.450.html)

[**Vulkan Main Page Link**](https://www.khronos.org/registry/vulkan/)

* [Vulkan 1.1.122](https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/index.html)

[**OpenCL Main Page**](https://www.khronos.org/registry/OpenCL/)

* [OpenCL 2.2 API Specification](https://www.khronos.org/registry/OpenCL/specs/2.2/html/OpenCL_API.html)
* [OpenCL 2.2 Extension Specification](https://www.khronos.org/registry/OpenCL/specs/2.2/html/OpenCL_Ext.html)
* [OpenCL 2.2 SPIR-V Environment Specification](https://www.khronos.org/registry/OpenCL/specs/2.2/html/OpenCL_Env.html)

* OpenCL released the proposed OpenCL 3.0 spec for comments in april 2020

* [Announcement video](https://youtu.be/h0_syTg6TtY)
* [Announcement video slides (PDF)](https://www.khronos.org/assets/uploads/apis/OpenCL-3.0-Launch-Apr20.pdf)

Note: We are implementing hardware accelerated Vulkan and
OpenCL while relying on other software projects to translate APIs to
Vulkan. E.g. Zink allows for OpenGL-to-Vulkan in software.

# Graphics and Compute API Stack

I found this informative post that mentions Kazan and a whole bunch of
other stuff. It looks like *many* APIs can be emulated on top of Vulkan,
although performance is not evaluated.

<https://synappsis.wordpress.com/2017/06/03/opengl-over-vulkan-dev/>

* Pixilica is heading up an initiative to create a RISC-V graphical ISA

* [Pixilica 3D Graphical ISA Slides](https://b5792ddd-543e-4dd4-9b97-fe259caf375d.filesusr.com/ugd/841f2a_c8685ced353b4c3ea20dbb993c4d4d18.pdf)

# 3D Graphics Texture compression software and hardware

* [Proprietary Rad Game Tools Oddle Texture Software Compression](https://web.archive.org/web/20200913122043/http://www.radgametools.com/oodle.htm)

* [Blog post by one of the engineers who developed the proprietary Rad Game Tools Oddle Texture Software Compression and the Oodle Kraken decompression software and hardware decoder used in the ps5 ssd](https://archive.vn/oz0pG)

# Various POWER Communities
 - [An effort to make a 100% Libre POWER Laptop](https://www.powerpc-notebook.org/en/)
   The T2080 is a POWER8 chip.
 - [Power Progress Community](https://www.powerprogress.org/campaigns/donations-to-all-the-power-progress-community-projects/)
   Supporting/Raising awareness of various POWER related open projects on the FOSS
   community
 - [OpenPOWER](https://openpowerfoundation.org)
   Promotes and ensure compliance with the Power ISA amongst members.
 - [OpenCapi](https://opencapi.org)
   High performance interconnect for POWER machines. One of the big advantages
   of the POWER architecture. Notably more performant than PCIE Gen4, and is
   designed to be layered on top of the physical PCIE link.
 - [OpenPOWER “Virtual Coffee” Calls](https://openpowerfoundation.org/openpower-virtual-coffee-calls/)
   Truly open bi-weekly teleconference lines for anybody interested in helping
   advance or adopting the POWER architecture.

# Conferences

## Free Silicon Conference

The conference brought together experts and enthusiasts who want to build
a complete Free and Open Source CAD ecosystem for designing analog and
digital integrated circuits.  The conference covered the full spectrum of
the design process, from system architecture, to layout and verification.

* <https://wiki.f-si.org/index.php/FSiC2019#Foundries.2C_PDKs_and_cell_libraries>

* LIP6's Coriolis - a set of backend design tools:
  <https://www-soc.lip6.fr/equipe-cian/logiciels/coriolis/>

Note: The rest of LIP6's website is in French, but there is a UK flag
in the corner that gives the English version.

* KLayout - Layout viewer and editor: <https://www.klayout.de/>

# The OpenROAD Project

OpenROAD seeks to develop and foster an autonomous, 24-hour, open-source
layout generation flow (RTL-to-GDS).

* <https://theopenroadproject.org/>

# Other RISC-V GPU attempts

* <https://fossi-foundation.org/2019/09/03/gsoc-64b-pointers-in-rv32>

* <http://bjump.org/manycore/>

* <https://resharma.github.io/RISCV32-GPU/>

TODO: Get in touch and discuss collaboration

# Tests, Benchmarks, Conformance, Compliance, Verification, etc.

## RISC-V Tests

RISC-V Foundation is in the process of creating an official conformance
test. It's still in development as far as I can tell.

* //TODO LINK TO RISC-V CONFORMANCE TEST

## IEEE 754 Testing/Emulation

IEEE 754 has no official tests for floating-point but there are
well-known third party tools to check such as John Hauser's TestFloat.

There is also his SoftFloat library, which is a software emulation
library for IEEE 754.

* <http://www.jhauser.us/arithmetic/>

Jacob is also working on an IEEE 754 software emulation library written
in Rust which also has Python bindings:

* Source: <https://salsa.debian.org/Kazan-team/simple-soft-float>
* Crate: <https://crates.io/crates/simple-soft-float>
* Autogenerated Docs: <https://docs.rs/simple-soft-float/>

A cool paper I came across in my research is "IeeeCC754++ : An Advanced
Set of Tools to Check IEEE 754-2008 Conformity" by Dr. Matthias Hüsken.

* Direct link to PDF:
  <http://elpub.bib.uni-wuppertal.de/servlets/DerivateServlet/Derivate-7505/dc1735.pdf>

## Khronos Tests

OpenCL Conformance Tests

* <https://github.com/KhronosGroup/OpenCL-CTS>

Vulkan Conformance Tests

* <https://github.com/KhronosGroup/VK-GL-CTS>

MAJOR NOTE: We are **not** allowed to say we are compliant with any of
the Khronos standards until we actually make an official submission,
do the paperwork, and pay the relevant fees.

## Formal Verification

Formal verification of Libre RISC-V ensures that it is bug-free in
regards to what we specify.  Of course, it is important to do the formal
verification as a final step in the development process before we produce
thousands or millions of silicon.

* Possible way to speed up our solvers for our formal proofs <https://web.archive.org/web/20201029205507/https://github.com/eth-sri/fastsmt>

* Algorithms (papers) submitted for 2018 International SAT Competition <https://web.archive.org/web/20201029205239/https://helda.helsinki.fi/bitstream/handle/10138/237063/sc2018_proceedings.pdf> <https://web.archive.org/web/20201029205637/http://www.satcompetition.org/>

Some learning resources I found in the community:

* ZipCPU: <http://zipcpu.com/> ZipCPU provides a comprehensive
  tutorial for beginners and many exercises/quizzes/slides:
  <http://zipcpu.com/tutorial/>
* Western Digital's SweRV CPU blog (I recommend looking at all their
  posts): <https://tomverbeure.github.io/>
* <https://tomverbeure.github.io/risc-v/2018/11/19/A-Bug-Free-RISC-V-Core-without-Simulation.html>
* <https://tomverbeure.github.io/rtl/2019/01/04/Under-the-Hood-of-Formal-Verification.html>

## Automation

* <https://www.ohwr.org/project/wishbone-gen>

# LLVM

## Adding new instructions:

* <https://archive.fosdem.org/2015/schedule/event/llvm_internal_asm/>

# Branch Prediction

* <https://danluu.com/branch-prediction/>

# Python RTL Tools

* [Migen - a Python RTL](https://jeffrey.co.in/blog/2014/01/d-flip-flop-using-migen/)
* [LiTeX](https://github.com/timvideos/litex-buildenv/wiki/LiteX-for-Hardware-Engineers)
  An SOC builder written in Python Migen DSL. Allows you to generate functional
  RTL for a SOC configured with cache, a RISCV core, ethernet, DRAM support,
  and parameterizeable CSRs.
* [Migen Tutorial](http://blog.lambdaconcept.com/doku.php?id=migen:tutorial>)
* There is a great guy, Robert Baruch, who has a good
  [tutorial](https://github.com/RobertBaruch/nmigen-tutorial) on nMigen.
  He also build an FPGA-proven Motorola 6800 CPU clone with nMigen and put
  [the code](https://github.com/RobertBaruch/n6800) and
  [instructional videos](https://www.youtube.com/playlist?list=PLEeZWGE3PwbbjxV7_XnPSR7ouLR2zjktw)
  online.
* [Minerva](https://github.com/lambdaconcept/minerva)
  An SOC written in Python nMigen DSL
* Minerva example using nmigen-soc
  <https://github.com/jfng/minerva-examples/blob/master/hello/core.py>
* [Using our Python Unit Tests(old)](http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2019-March/000705.html)
* <https://chisel.eecs.berkeley.edu/api/latest/chisel3/util/DecoupledIO.html>
* <http://www.clifford.at/papers/2016/yosys-synth-formal/slides.pdf>

# Other

* <https://debugger.medium.com/why-is-apples-m1-chip-so-fast-3262b158cba2> N1
* <https://codeberg.org/tok/librecell> Libre Cell Library
* <https://wiki.f-si.org/index.php/FSiC2019>
* <https://fusesoc.net>
* <https://www.lowrisc.org/open-silicon/>
* <http://fpgacpu.ca/fpga/Pipeline_Skid_Buffer.html> pipeline skid buffer
* <https://pyvcd.readthedocs.io/en/latest/vcd.gtkw.html> GTKwave
* <http://www.sunburst-design.com/papers/CummingsSNUG2002SJ_Resets.pdf>
  Synchronous Resets? Asynchronous Resets? I am so confused! How will I
  ever know which to use? by Clifford E. Cummings
* <http://www.sunburst-design.com/papers/CummingsSNUG2008Boston_CDC.pdf>
  Clock Domain Crossing (CDC) Design & Verification Techniques Using
  SystemVerilog, by Clifford E. Cummings
  In particular, see section 5.8.2: Multi-bit CDC signal passing using
  1-deep / 2-register FIFO synchronizer.
* <http://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-143.pdf>
  Understanding Latency Hiding on GPUs, by Vasily Volkov
* Efabless "Openlane" <https://github.com/efabless/openlane>
* Co-simulation plugin for verilator, transferring to ECP5
  <https://github.com/vmware/cascade>
* Multi-read/write ported memories
  <https://tomverbeure.github.io/2019/08/03/Multiport-Memories.html>
* Data-dependent fail-on-first aka "Fault-tolerant speculative vectorisation"
  <https://arxiv.org/pdf/1803.06185.pdf>
* OpenPOWER Foundation Membership
  <https://openpowerfoundation.org/membership/how-to-join/membership-kit-9-27-16-4/>
* Clock switching (and formal verification)
  <https://zipcpu.com/formal/2018/05/31/clkswitch.html>
* Circuit of Compunit <http://home.macintosh.garden/~mepy2/libre-soc/comp_unit_req_rel.html>
* Circuitverse 16-bit <https://circuitverse.org/users/17603/projects/54486>
* Nice example model of a Tomasulo-based architecture, with multi-issue, in-order issue, out-of-order execution, in-order commit, with reservation stations and reorder buffers, and hazard avoidance.
<https://www.brown.edu/Departments/Engineering/Courses/En164/Tomasulo_10.pdf> 
# Real/Physical Projects

* [Samuel's KC5 code](http://chiselapp.com/user/kc5tja/repository/kestrel-3/dir?ci=6c559135a301f321&name=cores/cpu)
* <https://chips4makers.io/blog/>
* <https://hackaday.io/project/7817-zynqberry>
* <https://github.com/efabless/raven-picorv32> 
* <https://efabless.com>
* <https://efabless.com/design_catalog/default>
* <https://wiki.f-si.org/index.php/The_Raven_chip:_First-time_silicon_success_with_qflow_and_efabless>
* <https://mshahrad.github.io/openpiton-asplos16.html>

# ASIC tape-out pricing

* <https://europractice-ic.com/wp-content/uploads/2020/05/General-MPW-EUROPRACTICE-200505-v8.pdf>

# Funding

* <https://toyota-ai.ventures/>
* [NLNet Applications](http://bugs.libre-riscv.org/buglist.cgi?columnlist=assigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Ccf_budget&f1=cf_nlnet_milestone&o1=equals&query_format=advanced&resolution=---&v1=NLnet.2019.02)

# Good Programming/Design Practices

* [Liskov Substitution Principle](https://en.wikipedia.org/wiki/Liskov_substitution_principle)
* [Principle of Least Astonishment](https://en.wikipedia.org/wiki/Principle_of_least_astonishment)
* <https://peertube.f-si.org/videos/watch/379ef007-40b7-4a51-ba1a-0db4f48e8b16>
* [Rust-Lang Philosophy and Consensus](http://smallcultfollowing.com/babysteps/blog/2019/04/19/aic-adventures-in-consensus/)

* <https://youtu.be/o5Ihqg72T3c>
* <http://flopoco.gforge.inria.fr/>
* Fundamentals of Modern VLSI Devices
  <https://groups.google.com/a/groups.riscv.org/d/msg/hw-dev/b4pPvlzBzu0/7hDfxArEAgAJ>

# 12 skills summary

* <https://www.crnhq.org/cr-kit/>

# Analog Simulation

* <https://github.com/Isotel/mixedsim>
* <http://www.vlsiacademy.org/open-source-cad-tools.html>
* <http://ngspice.sourceforge.net/adms.html>
* <https://en.wikipedia.org/wiki/Verilog-AMS#Open_Source_Implementations>

# Libre-SOC Standards

This list auto-generated from a page tag "standards":

[[!inline pages="tagged(standards)" actions="no" archive="yes" quick="yes"]]

# Server setup

* [[resources/server-setup/web-server]]
* [[resources/server-setup/git-mirroring]]
* [[resources/server-setup/nagios-monitoring]]

# Testbeds

* <https://www.fed4fire.eu/testbeds/>

# Really Useful Stuff

* <https://github.com/im-tomu/fomu-workshop/blob/master/docs/requirements.txt>
* <https://github.com/im-tomu/fomu-workshop/blob/master/docs/conf.py#L39-L47>

# Digilent Arty

* https://store.digilentinc.com/pmod-sf3-32-mb-serial-nor-flash/
* https://store.digilentinc.com/arty-a7-artix-7-fpga-development-board-for-makers-and-hobbyists/
* https://store.digilentinc.com/pmod-vga-video-graphics-array/
* https://store.digilentinc.com/pmod-microsd-microsd-card-slot/
* https://store.digilentinc.com/pmod-rtcc-real-time-clock-calendar/
* https://store.digilentinc.com/pmod-i2s2-stereo-audio-input-and-output/

# CircuitJS experiments

* [[resources/high-speed-serdes-in-circuitjs]]

# ASIC Timing and Design flow resources

* <https://www.linkedin.com/pulse/asic-design-flow-introduction-timing-constraints-mahmoud-abdellatif/>
* <https://www.icdesigntips.com/2020/10/setup-and-hold-time-explained.html>
* <https://www.vlsiguide.com/2018/07/clock-tree-synthesis-cts.html>
* <https://en.wikipedia.org/wiki/Frequency_divider>

# Geometric Haskell Library

* <https://github.com/julialongtin/hslice/blob/master/Graphics/Slicer/Math/GeometricAlgebra.hs>
* <https://github.com/julialongtin/hslice/blob/master/Graphics/Slicer/Math/PGA.hs>
* <https://arxiv.org/pdf/1501.06511.pdf>
* <https://bivector.net/index.html>