README: Flow

[litex.git] / README

             Migen (Milkymist Generator)
a Python toolbox for building complex digital hardware
======================================================

Background
==========
Even though the Milkymist system-on-chip [1] is technically successful,
it suffers from several limitations stemming from its implementation in
manually written Verilog HDL:

(1) The "event-driven" paradigm of today's dominant hardware descriptions
languages (Verilog and VHDL, collectively referred to as "V*HDL" in the
rest of this document) is often too general. Today's FPGA architectures
are optimized for the implementation of fully synchronous circuits. This
means that the bulk of the code for an efficient FPGA design falls into
three categories:
  (a) Combinatorial statements
  (b) Synchronous statements
  (c) Initialization of registers at reset
V*HDL do not follow this organization. This means that a lot of
repetitive manual coding is needed, which brings sources of human errors,
petty issues, and confusion for beginners:
  - wire vs. reg in Verilog
  - forgetting to initialize a register at reset
  - deciding whether a combinatorial statement must go into a
    process/always block or not
  - simulation mismatches with combinatorial processes/always blocks
  - and more...
A little-known fact about FPGAs is that many of them have to ability to
initialize their registers from the bitstream contents. This can be done
in a portable and standard way using an "initial" block in Verilog, and
by affecting a value at the signal declaration in VHDL. This renders an
explicit reset signal unnecessary in practice in some cases, which opens
the way for further design optimization. However, this form of
initialization is entirely not synthesizable for ASIC targets, and it is
not easy to switch between the two forms of reset using V*HDL.

(2) V*HDL support for composite types is very limited. Signals having a
record type in VHDL are unidirectional, which makes them clumsy to use
e.g. in bus interfaces. There is no record type support in Verilog, which
means that a lot of copy-and-paste has to be done when forwarding grouped
signals.

(3) V*HDL support for procedurally generated logic is extremely limited.
The most advanced forms of procedural generation of synthesizable logic
that V*HDL offers are CPP-style directives in Verilog, combinatorial
functions, and generate statements. Nothing really fancy, and it shows.
To give a few examples:
  - Building highly flexible bus interconnect is not possible. Even
arbitrating any given number of bus masters for commonplace protocols
such as Wishbone cannot be done with the tools at V*HDL puts at our
disposal. This requires manual recoding of parts of the arbiter to add or
remove a master, which is tedious and often cause frustrating errors.
Each occurence of the latter can easily cause one or two hours of lost
productivity when combined with the long compilation times of moderately
complex system-on-chip designs.
  - Building a memory infrastructure (including bus interconnect, bridges
and caches) that can automatically adapt itself at compile-time to any
word size of the SDRAM is clumsy and tedious.
  - Building register banks for control, status and interrupt management
of cores can also largely benefit from automation.
  - Many hardware acceleration problems can fit into the dataflow
programming model. Manual dataflow implementation in V*HDL has, again, a
lot of redundancy and potential for human errors. See the Milkymist
texture mapping unit [3][4] for an example of this. The amount of detail
to deal with manually also makes the design space exploration difficult,
and therefore hinders the design of efficient architectures.
  - Pre-computation of values, such as filter coefficients for DSP or
even simply trigonometric tables, must often be done using external tools
whose results are copy-and-pasted (in the best cases, automatically) into
the V*HDL source.

Enter Migen, a Python toolbox for building complex digital hardware. We
could have designed a brand new programming language, but that would have
been reinventing the wheel instead of being able to benefit from Python's
rich features and immense library. The price to pay is a slightly
cluttered syntax at times when writing descriptions in FHDL, but we
believe this is totally acceptable, particularly when compared to VHDL
;-)

Migen is made up of several related components, which are briefly
described below.

Migen FHDL
==========
The Fragmented Hardware Description Language (FHDL) is the lowest layer
of Migen. It consists of a formal system to describe signals, and
combinatorial and synchronous statements operating on them. The formal
system itself is low level and close to the synthesizable subset of
Verilog, and we then rely on Python algorithms to build complex
structures by combining FHDL elements and encapsulating them in
"fragments".
The FHDL module also contains a back-end to produce synthesizable
Verilog, and some basic analysis functions. It would be possible to
develop a VHDL back-end as well, though more difficult than for Verilog -
we are "cheating" a bit now as Verilog provides most of the FHDL
semantics.

FHDL differs from MyHDL [2] in fundamental ways. MyHDL follows the
event-driven paradigm of traditional HDLs (see Background, #1) while FHDL
separates the code into combinatorial statements, synchronous statements,
and reset values. In MyHDL, the logic is described directly in the Python
AST. The converter to Verilog or VHDL then examines the Python AST and
recognizes a subset of Python that it translates into V*HDL statements.
This seriously impedes the capability of MyHDL to generate logic
procedurally. With FHDL, you manipulate a custom AST from Python, and you
can more easily design algorithms that operate on it.

FHDL is made of several elements, which are briefly explained below.

BV
--
The bit vector (BV) object defines if a constant or signal is signed or
unsigned, and how many bits it has. This is useful e.g. to:
 - determine when to perform sign extension (FHDL uses the same rules as
Verilog).
 - determine the size of registers.
 - determine how many bits should be used by each value in
concatenations.

Constant
--------
This object should be self-explanatory. All constant objects contain a BV
object and a value. If no BV object is specified, one will be made up
using the following rules:
  - If the value is positive, the BV is unsigned and has the minimum
number of bits needed to represent the constant's value in the canonical
base-2 system.
  - If the value is negative, the BV is signed, and has the minimum
number of bits needed to represent the constant's value in the canonical
two's complement, base-2 system.

Signal
------
The signal object represents a value that is expected to change in the
circuit. It does exactly what Verilog's "wire" and "reg" and VHDL's
"signal" and "variable" do.

The main point of the signal object is that it is identified by its
Python ID (as returned by the id() function), and nothing else. It is the
responsibility of the V*HDL back-end to establish an injective mapping
between Python IDs and the V*HDL namespace. It should perform name
mangling to ensure this. The consequence of this is that signal objects
can safely become members of arbitrary Python classes, or be passed as
parameters to functions or methods that generate logic involving them.

The properties of a signal object are:
  - a bit vector description
  - a name, used as a hint for the V*HDL back-end name mangler.
  - a boolean "variable". If true, the signal will behave like a VHDL
variable, or a Verilog reg that uses blocking assignment. This parameter
only has an effect when the signal's value is modified in a synchronous
statement.
  - the signal's reset value. It must be an integer, and defaults to 0.
When the signal's value is modified with a synchronous statement, the
reset value is the initialization value of the associated register.
When the signal is assigned to in a conditional combinatorial statement
(If or Case), the reset value is the value that the signal has when no
condition that causes the signal to be driven is verified. This enforces
the absence of latches in designs. If the signal is permanently driven
using a combinatorial statement, the reset value has no effect.
  
The sole purpose of the name property is to make the generated V*HDL code
easier to understand and debug. From a purely functional point of view,
it is perfectly OK to have several signals with the same name property.
The back-end will generate a unique name for each object. If no name
property is specified, Migen will analyze the code that created the
signal object, and try to extract the variable or member name from there.
It then uses the module name that created the signal, a underscore, and
the variable name. For example, if we are in module "foo", the following
statements will create one or several signal(s) named "foo_bar":
  bar = Signal()
  self.bar = Signal()
  self.baz.bar = Signal()
  bar = [Signal() for x in range(42)]

Operators
---------
Operators are represented by the _Operator object, which generally should
not be used directly. Instead, most FHDL objects overload the usual
Python logic and arithmetic operators, which allows a much lighter syntax
to be used. For example, the expression:
  a * b + c
is equivalent to:
  _Operator('+', [_Operator('*', [a, b]), c])
  
Slices
------
Likewise, slices are represented by the _Slice object, which often should
not be used in favor of the Python slice operation [x:y].
Implicit indices using the forms [x], [x:] and [:y] are supported.
Beware! Slices work like Python slices, not like VHDL or Verilog slices.
The first bound is the index of the LSB and is inclusive. The second
bound is the index of MSB and is exclusive. In V*HDL, bounds are MSB:LSB
and both are inclusive.

Concatenations
--------------
Concatenations are done using the Cat object. To make the syntax lighter,
its constructor takes a variable number of arguments, which are the
signals to be concatenated together (you can use the Python '*' operator
to pass a list instead).
To be consistent with slices, the first signal is connected to the bits
with the lowest indices in the result. This is the opposite of the way
the '{}' construct works in Verilog.

Replications
------------
The Replicate object represents the equivalent of {count{expression}} in
Verilog.

Assignments
-----------
Assignments are represented with the _Assign object. Since using it
directly would result in a cluttered syntax, the preferred technique for
assignments is to use the be() method provided by objects that can have a
value assigned to them. They are signals, and their combinations with the
slice and concatenation operators.
As an example, the statement:
  a[0].be(b)
is equivalent to:
  _Assign(_Slice(a, 0, 1), b)

If statement
------------
The If object takes a first parameter which must be an expression
(combination of the Constant, Signal, _Operator, _Slice, etc. objects)
representing the condition, then a variable number of parameters
representing the statements (_Assign, If, Case, etc. objects) to be
executed when the condition is verified.

The If object defines a Else() method, which when called defines the
statements to be executed when the condition is not true. Those
statements are passed as parameters to the variadic method.

For convenience, there is also a Elif() method.

Example:
If(tx_count16 == 0,
    tx_bitcount.be(tx_bitcount + 1),
    If(tx_bitcount == 8,
        self.tx.be(1)
    ).Elif(tx_bitcount == 9,
        self.tx.be(1),
        tx_busy.be(0)
    ).Else(
        self.tx.be(tx_reg[0]),
        tx_reg.be(Cat(tx_reg[1:], 0))
    )
)

Case statement
--------------
The Case object constructor takes as first parameter the expression to be
tested, then a variable number of lists describing the various cases.

Each list contains an expression (typically a constant) describing the
value to be matched, followed by the statements to be executed when there
is a match. The head of the list can be the an instance of the Default
object.

Instances
---------
Instance objects represent the parametrized instantiation of a V*HDL
module, and the connection of its ports to FHDL signals. They are useful
in a number of cases:
  - reusing legacy or third-party V*HDL code.
  - using special FPGA features (DCM, ICAP, ...).
  - implementing logic that cannot be expressed with FHDL (asynchronous
    circuits, ...).
  - breaking down a Migen system into multiple sub-systems, possibly
    using different clock domains.

The properties of the instance object are:
  - the type of the instance (i.e. name of the instantiated module).
  - a list of output ports of the instantiated module. Each element of
    the list is a pair containing a string, which is the name of the
    module's port, and either an existing signal (on which the port will
    be connected to) or a BV (which will cause the creation of a new
    signal).
  - a list of input ports (likewise).
  - a list of (name, value) pairs for the parameters ("generics" in VHDL)
    of the module.
  - the name of the clock port of the module (if any). If this is
    specified, the port will be connected to the system clock.
  - the name of the reset port of the module (likewise).
  - the name of the instance (can be mangled like signal names).

Memories
--------
Memories (on-chip SRAM) are not supported, but will be soon, using a
mechanism similar to instances. (TODO)

Fragments
---------
A "fragment" is a unit of logic, which is composed of:
  - a list of combinatorial statements.
  - a list of synchronous statements.
  - a list of instances.
  - a list of memories.
  - a set of pads, which are signals intended to be connected to
    off-chip devices.

Fragments can reference arbitrary signals, including signals that are
referenced in other fragments. Fragments can be combined using the "+"
operator, which returns a new fragment containing the concatenation of
each pair of lists.

Fragments can be passed to the back-end for conversion to Verilog.

By convention, classes that generate logic implement a method called
"get_fragment". When called, this method builds a new fragment
implementing the desired functionality of the class, and returns it. This
convention allows fragments to be built automatically by combining the
fragments from all relevant objects in the local scope, by using the
autofragment module.

Migen Core Logic
================
Migen Core Logic is a convenience library of common logic circuits
implemented using FHDL:
  - a multi-cycle integer divider.
  - a round-robin arbiter, useful to build bus arbiters.
  - a multiplexer bank (multimux), useful to multiplex composite
    (grouped) signals.
  - a condition-triggered static scheduler of FHDL synchronous statements
    (timeline).

Migen Bus
=========
Migen Bus contains classes providing a common structure for master and
slave interfaces of the following buses:
  - Wishbone [5], the general purpose bus recommended by Opencores.
  - CSR-NG, a low-bandwidth, resource-sensitive bus designed for
    accessing the configuration and status registers of cores from
    software.
  - FastMemoryLink-NG, a split-transaction bus optimized for use with a
    high-performance, out-of-order SDRAM controller. (TODO)

It also provides interconnect components for these buses, such as
arbiters and address decoders. The strength of the Migen procedurally
generated logic can be illustrated by the following example:
  wbcon = wishbone.InterconnectShared(
      [cpu.ibus, cpu.dbus, ethernet.dma, audio.dma],
      [(0, norflash.bus), (1, wishbone2fml.wishbone),
      (3, wishbone2csr.wishbone)])
In this example, the interconnect component generates a 4-way round-robin
arbiter, multiplexes the master bus signals into a shared bus, determines
that the address decoding must occur on 2 bits, and connects all slave
interfaces to the shared bus, inserting the address decoder logic in the
bus cycle qualification signals and multiplexing the data return path. It
can recognize the signals in each core's bus interface thanks to the
common structure mandated by Migen Bus. All this happens automatically,
using only that much user code. The resulting interconnect logic can be
retrieved using wbcon.get_fragment(), and combined with the fragments
from the rest of the system.

Migen Bank
==========
Migen Bank is a system comparable to wishbone-gen [6], which automates
the creation of configuration and status register banks and
(TODO) interrupt/event managers implemented in cores.

Bank takes a description made up of a list of registers and generates
logic implementing it with a slave interface compatible with Migen Bus.

A register can be "raw", which means that the core has direct access to
it. It also means that the register width must be less or equal to the
bus word width. In that case, the register object provides the following
signals:
  - dev_r, which contains the data written from the bus interface.
  - dev_re, which is the strobe signal for dev_r. It is active for one
    cycle, after or during a write from the bus. dev_r is only valid when
    dev_re is high.
  - dev_w, which must provide at all times the value to be read from the
    bus.

Registers that are not raw are managed by Bank and contain fields. If the
sum of the widths of all fields attached to a register exceeds the bus
word width, the register will automatically be sliced into words of the
maximum size and implemented at consecutive bus addresses, MSB first.
Field objects have two parameters, access_bus and access_dev, determining
respectively the access policies for the bus and core sides. They can
take the values READ_ONLY, WRITE_ONLY and READ_WRITE.
If the device can read, the field object provides the dev_r signal, which
contains at all times the current value of the field (kept by the logic
generated by Bank).
If the device can write, the field object provides the following signals:
  - dev_w, which provides the value to be written into the field.
  - dev_we, which strobes the value into the field.

Migen Flow (TODO)
==========
Many hardware acceleration problems can be expressed in the dataflow
paradigm, that is, using a directed graph representing the flow of data
between actors.

Actors in Migen are written directly in FHDL. This maximizes the
flexibility: for example, an actor can implement a DMA master to read
data from system memory. It is conceivable that a CAL [7] to FHDL
compiler be implemented at some point, to support higher level
descriptions of some actors and reuse of third-party RVC-CAL
applications. [8] [9] [10]

Actors communicate by exchanging tokens, whose flow is typically
controlled using handshake signals (strobe/ack).

Each actor has a "scheduling model". It can be:
  - N-sequential: the actor fires when tokens are available at all its
    inputs, and it produces one output token after N cycles. It cannot
    accept new input tokens until it has produced its output. A
    multicycle integer divider would use this model.
  - N-pipelined: similar to the sequential model, but the actor can
    always accept new input tokens. It produces an output token N cycles
    of latency after accepting input tokens. A pipelined multiplier would
    use this model.
  - Dynamic: the general case, when no simple hypothesis can be made on
    the token flow behaviour of the actor. An actor accessing system
    memory on a shared bus would use this model.

Migen Flow automatically generates handshake logic for the first two
scheduling models. In the third case, the FHDL descriptions for the logic
driving the handshake signals must be provided by the actor.

If sequential or pipelined actors are connected together, Migen Flow will
attempt to find a static schedule, remove the handshake signals, optimize
away the control logic in each actor and replace it with a centralized
FSM implementing the static schedule.

An actor can be a composition of other actors.

Actor graphs are managed using the NetworkX [11] library.


References:
[ 1] http://milkymist.org
[ 2] http://www.myhdl.org
[ 3] http://milkymist.org/thesis/thesis.pdf
[ 4] http://www.xilinx.com/publications/archives/xcell/Xcell77.pdf p30-35
[ 5] http://cdn.opencores.org/downloads/wbspec_b4.pdf
[ 6] http://www.ohwr.org/projects/wishbone-gen
[ 7] http://opendf.svn.sourceforge.net/viewvc/opendf/trunk/doc/
     GentleIntro/GentleIntro.pdf
[ 8] http://orcc.sourceforge.net/
[ 9] http://orc-apps.sourceforge.net/
[10] http://opendf.sourceforge.net/
[11] http://networkx.lanl.gov/

Practical information
=====================
Code repository:
https://github.com/milkymist/migen
Experimental version of the Milkymist SoC based on Migen:
https://github.com/milkymist/milkymist-ng

Migen is designed for Python 3.2.

Send questions, comments and patches to devel [AT] lists.milkymist.org
We are also on IRC: #milkymist on the Freenode network.

Migen is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation, version 3 of the License. This program is
distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.
Unless otherwise noted, Migen's source code is copyright (C) 2011
Sebastien Bourdeauducq. Authors retain ownership of their contributions.