microwatt.git
2 years agoUse alternative count-leading-zeroes algorithm in the FPU and LSU
Paul Mackerras [Mon, 21 Feb 2022 01:06:11 +0000 (12:06 +1100)]
Use alternative count-leading-zeroes algorithm in the FPU and LSU

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years agocountzero: Use alternative algorithm for higher bits
Paul Mackerras [Sun, 20 Feb 2022 22:58:07 +0000 (09:58 +1100)]
countzero: Use alternative algorithm for higher bits

This implements an alternative count-leading-zeroes algorithm which
uses less LUTs to generate the higher-order bits (2..5) of the
result.

By doing (v | -v) rather than (v & -v), we get a value which has ones
from the MSB down to the rightmost 1 bit in v and then zeroes down to
the LSB.  This means that we can generate the MSB of the result (the
index of the rightmost 1 bit in v) just by looking at bits 63 and 31
of (v | -v), assuming that v is 64 bits.  Bit 4 of the result requires
looking at bits 63, 47, 31 and 15.  In contrast, each bit of the
result using (v & -v), which has a single 1, requires ORing together
32 bits.

It turns out that the minimum LUT usage comes from using (v & -v) to
generate bits 0 and 1 of the result, and using (v | -v) to generate
bits 2 to 5.  This saves almost 60 6-input LUTs on the Artix-7.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years agosoc: Re-do peripheral address decode to improve timing
Paul Mackerras [Mon, 11 Oct 2021 06:46:44 +0000 (17:46 +1100)]
soc: Re-do peripheral address decode to improve timing

This generates a series of io_cycle_* signals which are clean latches
and which become the 'cyc' signals of the wishbone buses going to
various peripherals (syscon, uarts, XICS, GPIO, etc.).  Effectively
this is done by moving the address decoding into the slave_io_latch
process.  The slave_io_type, which drives the multiplexer which
selects which wishbone to look for a response on, is reduced to just 8
values in the expectation that an 8-way multiplexer will use less
logic than one with more than 8 inputs.

With this timing is considerably better on the A7-100T.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years agofetch1/icache1: Remove the use_previous logic
Paul Mackerras [Mon, 11 Oct 2021 06:23:08 +0000 (17:23 +1100)]
fetch1/icache1: Remove the use_previous logic

This removes logic that I added some time ago with the thought that it
would enable us to do prefetching in the icache.  This logic detects
when the fetch address is an odd multiple of 4 and the next address in
sequence from the previous cycle.  In that case the instruction we
want is in the output register of the icache RAM already so there is
no need to do another read or any icache tag or TLB lookup.

However, this logic adds complexity, and removing it improves timing,
so this removes it.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years agoMerge pull request #345 from antonblanchard/popcnt-go-fast
Paul Mackerras [Fri, 4 Feb 2022 00:43:42 +0000 (11:43 +1100)]
Merge pull request #345 from antonblanchard/popcnt-go-fast

popcnt* timing improvements from Paul

2 years agocore: Make popcnt* take two cycles
Paul Mackerras [Tue, 19 Oct 2021 01:22:10 +0000 (12:22 +1100)]
core: Make popcnt* take two cycles

This moves the calculation of the result for popcnt* into the
countbits unit, renamed from countzero, so that we can take two cycles
to get the result.  The motivation for this is that the popcnt*
calculation was showing up as a critical path.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2 years agoMerge pull request #342 from mkj/orangecrab-merge
Michael Neuling [Tue, 18 Jan 2022 02:27:27 +0000 (13:27 +1100)]
Merge pull request #342 from mkj/orangecrab-merge

Orangecrab working with litedram

Fixed up a few simple merge conflicts in the Makefile.

2 years agoMerge branch 'master' into orangecrab-merge
Michael Neuling [Tue, 18 Jan 2022 01:03:46 +0000 (12:03 +1100)]
Merge branch 'master' into orangecrab-merge

2 years agoMerge pull request #341 from mkj/progtools
Michael Neuling [Tue, 18 Jan 2022 00:51:54 +0000 (11:51 +1100)]
Merge pull request #341 from mkj/progtools

orangecrab programming targets

2 years agoMerge pull request #340 from mkj/orangecrab-ghdl-plugin
Michael Neuling [Tue, 18 Jan 2022 00:50:22 +0000 (11:50 +1100)]
Merge pull request #340 from mkj/orangecrab-ghdl-plugin

Makefile: detect when ghdl is a yosys plugin

2 years agoorangecrab: Fix sdcard wishbone addressing
Matt Johnston [Fri, 19 Nov 2021 05:13:15 +0000 (13:13 +0800)]
orangecrab: Fix sdcard wishbone addressing

Orangecrab missed out on:

Make wishbone addresses be in units of doublewords or words
Author: Paul Mackerras <paulus@ozlabs.org>
Date:   Wed Sep 15 18:18:09 2021 +1000

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoorangecrab: use litesdcard
Matt Johnston [Fri, 14 Jan 2022 00:04:18 +0000 (08:04 +0800)]
orangecrab: use litesdcard

Currently not working (tested in Linux)

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agolitesdcard: add lattice, regenerate
Matt Johnston [Fri, 14 Jan 2022 00:08:09 +0000 (08:08 +0800)]
litesdcard: add lattice, regenerate

Modifies litescard generate script to take a clock speed.

Regenerated verilog with latest litesdcard
e52c731 ("Bump year.")

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoorangecrab: No BTC, LOG_LENGTH, dram NUM_LINES
Matt Johnston [Mon, 23 Aug 2021 02:30:40 +0000 (10:30 +0800)]
orangecrab: No BTC, LOG_LENGTH, dram NUM_LINES

Reduce litedram NUM_LINES 64->8
This allows us to meet timing. Can probably
be improved in future with better BRAM usage.

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoorangecrab: Use litedram
Matt Johnston [Thu, 13 Jan 2022 08:51:57 +0000 (16:51 +0800)]
orangecrab: Use litedram

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoorangecrab: set HAS_SHORT_MULT
Matt Johnston [Wed, 24 Nov 2021 08:47:16 +0000 (16:47 +0800)]
orangecrab: set HAS_SHORT_MULT

It seems free, generated as a single MULT18X18D

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoorangecrab: add Orange Crab r0.2 target
Matt Johnston [Wed, 11 Aug 2021 05:11:57 +0000 (13:11 +0800)]
orangecrab: add Orange Crab r0.2 target

top-orangecrab0.2 is a copy of top-arty with various changes.
USRMCLK is added for the SPI clock
ethernet is removed

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agolitedram: Add orangecrab-85-0.2 target
Matt Johnston [Fri, 13 Aug 2021 02:07:15 +0000 (10:07 +0800)]
litedram: Add orangecrab-85-0.2 target

Parameters are based on
https://github.com/gregdavill/OrangeCrab-test-sw/blob/main/hw/OrangeCrab-bitstream.py
and litex-boards orangecrab.py

rtt_nom and cmd_delay are overridden for OrangeCrab, we do the same here.

Generated with litedram and litex
62abf9c ("litedram_gen: Add block_until_ready port parameter to control blocking behaviour.")
add2746a ("tools/litex_cli: Rename wb to bus.")

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agolitedram: set Makefile -Werror
Matt Johnston [Fri, 24 Sep 2021 04:24:29 +0000 (12:24 +0800)]
litedram: set Makefile -Werror

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agolitedram: disable block_until_ready, regenerate
Matt Johnston [Thu, 13 Jan 2022 23:23:35 +0000 (07:23 +0800)]
litedram: disable block_until_ready, regenerate

Recent litedram gets stuck at memtest unless block_until_ready=False.
(discussion in https://github.com/enjoy-digital/litedram/pull/292)

This change regenerates with latest litedram and litex
62abf9c ("litedram_gen: Add block_until_ready port parameter to control blocking behaviour.")
add2746a ("tools/litex_cli: Rename wb to bus.")

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoMakefile: add ecpprog targets
Matt Johnston [Fri, 26 Nov 2021 02:33:55 +0000 (10:33 +0800)]
Makefile: add ecpprog targets

The 0x80000 offset is specific to the OrangeCrab bootloader.

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoMakefile: Add DFU programming
Matt Johnston [Wed, 11 Aug 2021 05:07:34 +0000 (13:07 +0800)]
Makefile: Add DFU programming

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoMakefile: detect when ghdl is a yosys plugin
Matt Johnston [Wed, 11 Aug 2021 07:17:39 +0000 (15:17 +0800)]
Makefile: detect when ghdl is a yosys plugin

oss-cad-suite builds it as a plugin, some other toolchains
have it built in.

Signed-off-by: Matt Johnston <matt@codeconstruct.com.au>
2 years agoMerge pull request #338 from shenki/yosys-read-verilog
Anton Blanchard [Sat, 8 Jan 2022 21:08:48 +0000 (08:08 +1100)]
Merge pull request #338 from shenki/yosys-read-verilog

Makefile: Use read_verilog with yosys

2 years agoMakefile: Use read_verilog with yosys
Joel Stanley [Mon, 20 Dec 2021 22:32:24 +0000 (09:02 +1030)]
Makefile: Use read_verilog with yosys

Yosys changed command line behaviour following the v0.12 release.  Work
around this by using read_verilog, which maintains the old behaviour.

This should work fine for current yosys and be compatible with
future releases.

See https://github.com/YosysHQ/yosys/issues/3109

Signed-off-by: Joel Stanley <joel@jms.id.au>
3 years agoMerge pull request #337 from paulusmack/fixes
Michael Neuling [Mon, 25 Oct 2021 05:49:19 +0000 (16:49 +1100)]
Merge pull request #337 from paulusmack/fixes

ECP5: Adjust PLL constants so the PLL lock indication works

3 years agoECP5: Adjust PLL constants so the PLL lock indication works
Paul Mackerras [Sat, 16 Oct 2021 08:24:14 +0000 (19:24 +1100)]
ECP5: Adjust PLL constants so the PLL lock indication works

At present, code (such as simple_random) which produces serial port
output during the first few milliseconds of operation produces garbled
output.  The reason is that the clock has not yet stabilized and is
running slow, resulting in the bit time of the serial characters being
too long.

The ECP5 data sheet says that the phase detector should be operated
between 10 and 400 MHz.  The current code operates it at 2MHz.
Consequently, the PLL lock indication doesn't work, i.e. it is always
zero.  The current code works around that by inverting it, i.e. taking
the "not locked" indication to mean "locked".

Instead, we now run it at 12MHz, chosen because the common external
clock inputs on ECP5 boards are 12MHz and 48MHz.  Normally this would
mean that the available system clock frequencies would be multiples of
12MHz, but this is a little inconvenient as we use 40MHz on the Orange
Crab v0.21 boards.  Instead, by using the secondary clock output for
feedback, we can have any divisor of the PLL frequency as the system
clock frequency.

The ECP5 data sheet says the PLL oscillator can run at 400 to 800
MHz.  Here we choose 480MHz since that allows us to generate 40MHz and
48MHz easily and is a multiple of 12MHz.

With this, the lock signal works correctly, and the inversion can be
removed.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #336 from paulusmack/fixes
Michael Neuling [Wed, 13 Oct 2021 06:44:47 +0000 (17:44 +1100)]
Merge pull request #336 from paulusmack/fixes

Makefile: Correct parameters for the Orange Crab 85F

3 years agoMakefile: Add a target for the Orange Crab v0.21 with LFE5U-85F
Paul Mackerras [Tue, 12 Oct 2021 07:30:36 +0000 (18:30 +1100)]
Makefile: Add a target for the Orange Crab v0.21 with LFE5U-85F

The existing orange crab target is for an older board with a
LFE5UM5G-85F device.  Newer orange crab boards (v0.21) have a
LFE5U-85F device in the -8 speed grade, so make a new target for them
called ORANGE-CRAB-0.21.

Also add flags to ecppack to indicate that the bitstream should be
compressed and can be loaded at 38.8MHz.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #334 from antonblanchard/icbi-issue
Michael Neuling [Mon, 27 Sep 2021 23:06:18 +0000 (09:06 +1000)]
Merge pull request #334 from antonblanchard/icbi-issue

Add a test for icbi and dcbz issues

3 years agoMerge pull request #335 from ozbenh/misc
Anton Blanchard [Mon, 27 Sep 2021 20:18:59 +0000 (06:18 +1000)]
Merge pull request #335 from ozbenh/misc

Misc cleanups and icache fix

3 years agoicache: req_laddr becomes req_raddr
Benjamin Herrenschmidt [Mon, 27 Sep 2021 12:03:18 +0000 (22:03 +1000)]
icache: req_laddr becomes req_raddr

Uses real_addr_t and only stores the real address bits

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agoIntroduce addr_to_wb() and wb_to_addr() helpers
Benjamin Herrenschmidt [Mon, 27 Sep 2021 11:53:52 +0000 (21:53 +1000)]
Introduce addr_to_wb() and wb_to_addr() helpers

These convert addresses to/from wishbone addresses, and use them
in parts of the caches, in order to make the code a bit more readable.

Along the way, rename some functions in the caches to make it a bit
clearer what they operate on and fix a bug in the icache STOP_RELOAD state where
the wb address wasn't properly converted.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agoIntroduce real_addr_t and addr_to_real()
Benjamin Herrenschmidt [Mon, 27 Sep 2021 11:50:57 +0000 (21:50 +1000)]
Introduce real_addr_t and addr_to_real()

This moves REAL_ADDR_BITS out of the caches and defines a real_addr_t
type for a real address, along with a addr_to_real() conversion helper.

It makes the vhdl a bit more readable

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agotests/misc: Add a store/dcbz test
Anton Blanchard [Thu, 23 Sep 2021 05:47:52 +0000 (15:47 +1000)]
tests/misc: Add a store/dcbz test

We have a bug where an store near a dcbz can cause the dcbz to only zero
8 bytes. Add a test case for this.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agotests/misc: Add an icbi test
Anton Blanchard [Thu, 23 Sep 2021 02:23:22 +0000 (12:23 +1000)]
tests/misc: Add an icbi test

We have a bug where an icbi can cause an instruction to execute twice.
Add a test case for this.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoMerge pull request #333 from ozbenh/wukong
Anton Blanchard [Mon, 27 Sep 2021 03:41:37 +0000 (13:41 +1000)]
Merge pull request #333 from ozbenh/wukong

Add support for  QMTech Wukong v2 board

3 years agoRegenerate litedram and liteeth
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:24:37 +0000 (14:24 +1000)]
Regenerate litedram and liteeth

Note: There are a few patches to upstream to fix an upstream breakage
of litedram standalone generator, and fix some issues with liteeth
in the way it's used on Wukong. All these have pending pull requests.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agoAdd support for QMTech Wukong v2 board
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:24:37 +0000 (14:24 +1000)]
Add support for QMTech Wukong v2 board

For now only the V2 of the board (slightly different pinout)
and only the A100T variant. I also haven't added GPIOs or anything
else on the PMODs really.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agofpga/clk_gen_plle2: Add support for 50Mhz->100Mhz
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:24:37 +0000 (14:24 +1000)]
fpga/clk_gen_plle2: Add support for 50Mhz->100Mhz

50Mhz clkin, 100Mhz sys_clk, as needed for Wukon v2

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agoAdd support for more spansion flash
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:24:37 +0000 (14:24 +1000)]
Add support for more spansion flash

That's the one on the Wukong v2

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agoMerge pull request #332 from paulusmack/fixes
Anton Blanchard [Sat, 25 Sep 2021 05:15:24 +0000 (15:15 +1000)]
Merge pull request #332 from paulusmack/fixes

Bug fixes

3 years agodcache: Fix bug with dcbz closely following stores with the same tag
Paul Mackerras [Sat, 25 Sep 2021 03:18:59 +0000 (13:18 +1000)]
dcache: Fix bug with dcbz closely following stores with the same tag

This fixes a bug where a dcbz can get incorrectly handled as an
ordinary 8-byte store if it arrives while the dcache state machine is
handling other stores with the same tag value (i.e. within the same
set-sized area of memory).  The logic that says whether to include a
new store in the current wishbone cycle didn't take into account
whether the new store was a dcbz.  This adds a "req.dcbz = '0'" factor
so that it does.  This is necessary because dcbz is handled more like
a cache line refill (but writing to memory rather than reading) than
an ordinary store.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoicache: Fix icache invalidation
Paul Mackerras [Sat, 25 Sep 2021 01:34:38 +0000 (11:34 +1000)]
icache: Fix icache invalidation

This fixes two bugs in the flash invalidation of the icache.

The first is that an instruction could get executed twice.  The
i-cache RAM is 2 instructions (64 bits) wide, so one read can supply
results for 2 cycles.  The fetch1 stage tells icache when the address
is equal to the address of the previous cycle plus 4, and in cases
where that is true, bit 2 of the address is 1, and the previous cycle
was a cache hit, we just use the second word of the doubleword read
from the cache RAM.  However, the cache hit/miss logic also continues
to operate, so in the case where the first word hits but the second
word misses (because of an icache invalidation or a snoop occurring in
the first cycle), we supply the instruction from the data previously
read from the icache RAM but also stall fetch1 and start a cache
reload sequence, and subsequently supply the second instruction
again.  This fixes the issue by inhibiting req_is_miss and stall_out
when use_previous is true.

The second bug is that if an icache invalidation occurs while
reloading a line, we continue to reload the line, and make it valid
when the reload finishes, even though some of the data may have been
read before the invalidation occurred.  This adds a new state
STOP_RELOAD which we go to if an invalidation happens while we are in
CLR_TAG or WAIT_ACK state.  In STOP_RELOAD state we don't request any
more reads from memory and wait for the reads we have previously
requested to be acked, and then go to IDLE state.  Data returned is
still written to the icache RAM, but that doesn't matter because the
line is invalid and is never made valid.

Note that we don't have to worry about invalidations due to snooped
writes while reloading a line, because the wishbone arbiter won't
switch to another master once it has started sending our reload
requests to memory.  Thus a store to memory will either happen before
any of our reads have got to memory, or after we have finished the
reload (in which case we will no longer be in WAIT_ACK state).

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agodecode1: Conditional trap instructions don't need to be single-issue
Paul Mackerras [Sat, 25 Sep 2021 03:26:09 +0000 (13:26 +1000)]
decode1: Conditional trap instructions don't need to be single-issue

They can generate interrupts, but that doesn't mean they have to
single-issue.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #330 from antonblanchard/orange-crab-freq
Paul Mackerras [Fri, 24 Sep 2021 23:35:01 +0000 (09:35 +1000)]
Merge pull request #330 from antonblanchard/orange-crab-freq

Orange Crab is 48MHz not 50MHz, bump PLL frequency

3 years agoMerge pull request #331 from ozbenh/misc
Anton Blanchard [Fri, 24 Sep 2021 05:30:22 +0000 (15:30 +1000)]
Merge pull request #331 from ozbenh/misc

jtag tooling improvements & gitignore fix

3 years agoAdd liteeth/build to gitignore
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:23:06 +0000 (14:23 +1000)]
Add liteeth/build to gitignore

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agomw_debug: Default to jtag backend if unspecified
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:23:06 +0000 (14:23 +1000)]
mw_debug: Default to jtag backend if unspecified

It avoids typing it all the time

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agomw_debug: Probe cable if unspecified
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:23:06 +0000 (14:23 +1000)]
mw_debug: Probe cable if unspecified

Instead of defaulting to DigilentHS1

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agoflash-arty: Add cable argument
Benjamin Herrenschmidt [Fri, 24 Sep 2021 04:23:06 +0000 (14:23 +1000)]
flash-arty: Add cable argument

To select the cable config. Defaults to arty

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
3 years agoMerge pull request #329 from paulusmack/wb-fix
Anton Blanchard [Fri, 24 Sep 2021 04:19:26 +0000 (14:19 +1000)]
Merge pull request #329 from paulusmack/wb-fix

Wishbone addressing fix

3 years agoOrange Crab is 48MHz not 50MHz, bump PLL frequency
Anton Blanchard [Fri, 24 Sep 2021 02:43:33 +0000 (12:43 +1000)]
Orange Crab is 48MHz not 50MHz, bump PLL frequency

I'm not sure why I set the input frequency for the Orange Crab to 50MHz.
Since we easily make timing now, bump our output frequency to 48MHz as
well.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoMerge pull request #328 from paulusmack/shortmult
Michael Neuling [Thu, 16 Sep 2021 06:07:05 +0000 (16:07 +1000)]
Merge pull request #328 from paulusmack/shortmult

core: Add a short multiplier

3 years agoMerge pull request #327 from paulusmack/master
Michael Neuling [Thu, 16 Sep 2021 05:24:36 +0000 (15:24 +1000)]
Merge pull request #327 from paulusmack/master

loadstore1 timing improvement

3 years agoMake wishbone addresses be in units of doublewords or words
Paul Mackerras [Wed, 15 Sep 2021 08:18:09 +0000 (18:18 +1000)]
Make wishbone addresses be in units of doublewords or words

This makes the 64-bit wishbone buses have the address expressed in
units of doublewords (64 bits), and similarly for the 32-bit buses the
address is in units of words (32 bits).  This is to comply with the
wishbone spec.  Previously the addresses on the wishbone buses were in
units of bytes regardless of the bus data width, which is not correct
and caused problems with interfacing with externally-generated logic.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agocore: Add a short multiplier
Paul Mackerras [Sat, 11 Sep 2021 07:10:20 +0000 (17:10 +1000)]
core: Add a short multiplier

This adds an optional 16 bit x 16 bit signed multiplier and uses it
for multiply instructions that return the low 64 bits of the product
(mull[dw][o] and mulli, but not maddld) when the operands are both in
the range -2^15 .. 2^15 - 1.   The "short" 16-bit multiplier produces
its result combinatorially, so a multiply that uses it executes in one
cycle.  This improves the coremark result by about 4%, since coremark
does quite a lot of multiplies and they almost all have operands that
fit into 16 bits.

The presence of the short multiplier is controlled by a generic at the
execute1, SOC, core and top levels.  For now, it defaults to off for
all platforms, and can be enabled using the --has_short_mult flag to
fusesoc.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoloadstore1: Make r1.req.addr not depend on l_in.valid
Paul Mackerras [Mon, 13 Sep 2021 07:25:52 +0000 (17:25 +1000)]
loadstore1: Make r1.req.addr not depend on l_in.valid

Some critical path reports showed r1.req.addr depending on l_in.valid,
which then depended ultimately on the dcache's r1.ls_valid.  In fact
we can update r1.req.addr (and other fields of r1.req, except for
r1.req.valid) independently of l_in.valid as long as busy = 0.
We do also need to preserve r1.req.addr0 when l_in.valid = 0, so we
pull it out of r1.req and store it separately in r1.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #324 from paulusmack/master
Michael Neuling [Mon, 13 Sep 2021 02:11:24 +0000 (12:11 +1000)]
Merge pull request #324 from paulusmack/master

Performance and timing improvements

3 years agocore: Predict not-taken conditional branches using BTC
Paul Mackerras [Tue, 7 Sep 2021 10:38:00 +0000 (20:38 +1000)]
core: Predict not-taken conditional branches using BTC

This adds a bit to the BTC to store whether the corresponding branch
instruction was taken last time it was encountered.  That lets us pass
a not-taken prediction down to decode1, which for backwards direct
branches inhibits it from redirecting fetch to the target of the
branch.  This increases coremark by about 2%.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoxilinx-mult: Move some registers later in the data flow
Paul Mackerras [Mon, 6 Sep 2021 23:32:30 +0000 (09:32 +1000)]
xilinx-mult: Move some registers later in the data flow

This changes s0 to use the P register rather than the A/B/C input
registers, thus improving the timing of the multiplier output.  The
m00, m02 and m03 multipliers now use their P registers rather than the
M registers, moving the addition they do from the second cycle to the
first.

Also, the XOR that inverts the 32 LSBs is moved before the output
register.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #326 from antonblanchard/dcache-nc-fix
Paul Mackerras [Sat, 11 Sep 2021 01:57:04 +0000 (11:57 +1000)]
Merge pull request #326 from antonblanchard/dcache-nc-fix

dcache: Loads from non-cacheable PTEs load entire 64 bits

3 years agoMerge pull request #325 from paulusmack/fixes
Anton Blanchard [Fri, 10 Sep 2021 11:04:08 +0000 (21:04 +1000)]
Merge pull request #325 from paulusmack/fixes

decode1: Fix form of isel marked as single-issue

3 years agodcache: Loads from non-cacheable PTEs load entire 64 bits
Anton Blanchard [Fri, 10 Sep 2021 10:51:53 +0000 (20:51 +1000)]
dcache: Loads from non-cacheable PTEs load entire 64 bits

A non-cacheable load should only load the data requested and no more. We
do the right thing for real mode cache inhibited storage instructions,
but when loading through a non-cacheable PTE we load the entire 64 bits
regardless of the size.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agodecode1: Fix form of isel marked as single-issue
Paul Mackerras [Thu, 13 May 2021 22:41:59 +0000 (08:41 +1000)]
decode1: Fix form of isel marked as single-issue

The row in the decode table for isel with BC=0 was inadvertently left
marked as single-issue by commit 813f8340127f ("Add CR hazard
detection", 2019-10-15).  Fix it.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #323 from paulusmack/fixes
Michael Neuling [Mon, 6 Sep 2021 07:17:37 +0000 (17:17 +1000)]
Merge pull request #323 from paulusmack/fixes

More instruction fixes

3 years agodecode1: Fix maddld and maddhdu to not set CR0
Paul Mackerras [Sat, 4 Sep 2021 09:36:02 +0000 (19:36 +1000)]
decode1: Fix maddld and maddhdu to not set CR0

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agocore: Fix mcrxrx, addpcis and bpermd
Paul Mackerras [Sat, 4 Sep 2021 08:20:54 +0000 (18:20 +1000)]
core: Fix mcrxrx, addpcis and bpermd

- mcrxrx put the bits in the wrong order

- addpcis was setting CR0 if the instruction bit 0 = 1, which it
  shouldn't

- bpermd was producing 0 always and additionally had the wrong bit
  numbering

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #322 from paulusmack/fixes
Michael Neuling [Fri, 3 Sep 2021 02:07:06 +0000 (12:07 +1000)]
Merge pull request #322 from paulusmack/fixes

Fix bug with load hitting two previous stores

3 years agotests/misc: Add a test for a load that hits two preceding stores
Paul Mackerras [Tue, 31 Aug 2021 01:45:57 +0000 (11:45 +1000)]
tests/misc: Add a test for a load that hits two preceding stores

This checks that the store forwarding machinery in the dcache
correctly combines forwarded stores when they are partial stores
(i.e. only writing part of the doubleword, as for a byte store).

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agodcache: Simplify data input to improve timing
Paul Mackerras [Mon, 30 Aug 2021 08:24:49 +0000 (18:24 +1000)]
dcache: Simplify data input to improve timing

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agodcache: Fix bug with forwarding of stores
Paul Mackerras [Tue, 31 Aug 2021 09:47:14 +0000 (19:47 +1000)]
dcache: Fix bug with forwarding of stores

We have two stages of forwarding to cover the two cycles of latency
between when something is written to BRAM and when that new data can
be read from BRAM.  When the writes to BRAM result from store
instructions, the write may write only some bytes of a row (8 bytes)
and not others, so we have a mask to enable only the written bytes to
be forwarded.  However, we only forward written data from either the
first stage of forwarding or the second, not both.  So if we have
two stores in succession that write different bytes of the same row,
and then a load from the row, we will only forward the data from the
second store, and miss the data from the first store; thus the load
will get the wrong value.

To fix this, we make the decision on which forward stage to use for
each byte individually.  This results in a 4-input multiplexer feeding
r1.data_out, with its inputs being the BRAM, the wishbone, the current
write data, and the 2nd-stage forwarding register.  Each byte of the
multiplexer is separately controlled.  The code for this multiplexer
is moved to the dcache_fast_hit process since it is used for cache
hits as well as cache misses.

This also simplifies the BRAM code by ensuring that we can use the
same source for the BRAM address and way selection for writes, whether
we are writing store data or cache line refill data from memory.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agodcache: Move way selection and forwarding earlier
Paul Mackerras [Sat, 28 Aug 2021 10:11:55 +0000 (20:11 +1000)]
dcache: Move way selection and forwarding earlier

This moves the way multiplexer for the data from the BRAM, and the
multiplexers for forwarding data from earlier stores or refills,
before a clock edge rather than after, so that now the data output
from the dcache comes from a clean latch.  To do this we remove the
extra latch on the output of the data BRAM (i.e. ADD_BUF=false) and
rearrange the logic.  The choice whether to forward or not now depends
not on way comparisons but rather on a tag comparisons, for the sake
of timing.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #319 from antonblanchard/verilator-ci
Michael Neuling [Tue, 17 Aug 2021 02:31:12 +0000 (12:31 +1000)]
Merge pull request #319 from antonblanchard/verilator-ci

Add some Verilator CI tests

3 years agoMerge pull request #318 from paulusmack/pmu
Michael Neuling [Mon, 16 Aug 2021 03:08:24 +0000 (13:08 +1000)]
Merge pull request #318 from paulusmack/pmu

PMU enhancements

3 years agoMerge pull request #320 from antonblanchard/litedram-regenerate
Michael Neuling [Sun, 15 Aug 2021 23:23:27 +0000 (09:23 +1000)]
Merge pull request #320 from antonblanchard/litedram-regenerate

litedram: Regenerate from upstream litex

3 years agolitedram: Regenerate from upstream litex
Anton Blanchard [Sat, 14 Aug 2021 20:20:22 +0000 (06:20 +1000)]
litedram: Regenerate from upstream litex

This adds Joel's sdcard feature reporting to the firmware.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoci: Add verilator tests
Anton Blanchard [Sat, 14 Aug 2021 08:25:15 +0000 (18:25 +1000)]
ci: Add verilator tests

Now we have some verilator tests, add them to the CI.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agomakefile: Check environment for MEMORY_SIZE/RAM_INIT_FILE
Anton Blanchard [Sat, 14 Aug 2021 09:34:12 +0000 (19:34 +1000)]
makefile: Check environment for MEMORY_SIZE/RAM_INIT_FILE

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agomakefile: Add some verilator micropython tests
Anton Blanchard [Sat, 14 Aug 2021 06:59:50 +0000 (16:59 +1000)]
makefile: Add some verilator micropython tests

These are the same micropython tests we use against the ghdl
simulation.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoverilator: Specify top level module
Anton Blanchard [Sat, 14 Aug 2021 05:03:16 +0000 (15:03 +1000)]
verilator: Specify top level module

While verilator finds the correct top level module with the current
setup, if we start adding simulation models it can get confused.

Explicitly specify the top level module.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agomakefile: Simplify microwatt-verilator target, add Docker image
Anton Blanchard [Fri, 13 Aug 2021 05:04:06 +0000 (15:04 +1000)]
makefile: Simplify microwatt-verilator target, add Docker image

Recent versions of verilator support the --build option, allowing
us to remove a step.

Also add a Docker image for verilator.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoPMU: Add several more events
Paul Mackerras [Fri, 13 Aug 2021 09:29:48 +0000 (19:29 +1000)]
PMU: Add several more events

This implements most of the architected PMU events.  The ones missing
are mostly the ones that depend on which level of the cache hierarchy
data is fetched from.  The events implemented here, and their raw
event codes, are:

    Floating-point operation completed (100f4)
    Load completed (100fc)
    Store completed (200f0)
    Icache miss (200fc)
    ITLB miss (100f6)
    ITLB miss resolved (400fc)
    Dcache load miss (400f0)
    Dcache load miss resolved (300f8)
    Dcache store miss (300f0)
    DTLB miss (300fc)
    DTLB miss resolved (200f6)
    No instruction available and none being executed (100f8)
    Instruction dispatched (200f2, 300f2, 400f2)
    Taken branch instruction completed (200fa)
    Branch mispredicted (400f6)
    External interrupt taken (200f8)

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoMerge pull request #316 from antonblanchard/verilator-fix
Paul Mackerras [Sat, 14 Aug 2021 00:18:42 +0000 (10:18 +1000)]
Merge pull request #316 from antonblanchard/verilator-fix

Rename 'do' signal to avoid verilator System Verilog warning

3 years agoMerge pull request #317 from antonblanchard/gpio-fix
Paul Mackerras [Sat, 14 Aug 2021 00:18:12 +0000 (10:18 +1000)]
Merge pull request #317 from antonblanchard/gpio-fix

gpio: Add HAS_GPIO to avoid verilator build errors

3 years agoPMU: Fix PMC5/6 behaviour when MMCR0[PMCC] = 11
Paul Mackerras [Wed, 11 Aug 2021 07:17:25 +0000 (17:17 +1000)]
PMU: Fix PMC5/6 behaviour when MMCR0[PMCC] = 11

The architecture states that when MMCR0[PMCC] = 0b11, PMC5 and PMC6
are not part of the Performance Monitor, meaning that they are not
controlled by bits in MMCRs, and counter negative conditions in PMCs 5
and 6 don't generate Performance Monitor alerts, exceptions or
interrupts.  It doesn't say that PMC5 and PMC6 are frozen in this
case, so presumably they should continue to count run instructions and
run cycles.

This implements that behaviour.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agogpio: Add HAS_GPIO to avoid verilator build errors
Anton Blanchard [Fri, 13 Aug 2021 04:44:04 +0000 (14:44 +1000)]
gpio: Add HAS_GPIO to avoid verilator build errors

The verilator build fails with warnings and errors, because NGPIO
is 0 and we do things like:

        gpio_out : out std_ulogic_vector(NGPIO - 1 downto 0);

Set NGPIO to something reasonable (eg 32) and add HAS_GPIO to avoid
building the macro entirely if it isn't in use.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoRename 'do' signal to avoid verilator System Verilog warning
Anton Blanchard [Fri, 13 Aug 2021 03:52:51 +0000 (13:52 +1000)]
Rename 'do' signal to avoid verilator System Verilog warning

Experimenting with using ghdl to do VHDL to Verilog conversion (instead
of ghdl+yosys), verilator complains that a signal is a SystemVerilog
keyword:

%Error: microwatt.v:15013:18: Unexpected 'do': 'do' is a SystemVerilog keyword misused as an identifier.
        ... Suggest modify the Verilog-2001 code to avoid SV keywords, or use `begin_keywords or --language.

We could probably make this go away by disabling SystemVerilog, but
it's easy to rename the signal in question. Rename di at the same
time.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoMerge pull request #315 from paulusmack/pmu
Michael Neuling [Wed, 11 Aug 2021 22:24:05 +0000 (08:24 +1000)]
Merge pull request #315 from paulusmack/pmu

Add basic PMU implementation

3 years agoMerge pull request #314 from antonblanchard/yosys-go-fast-bits
Paul Mackerras [Wed, 11 Aug 2021 06:26:29 +0000 (16:26 +1000)]
Merge pull request #314 from antonblanchard/yosys-go-fast-bits

Reduce Yosys ECP5 cell usage by 30% with -abc9 -nowidelut

3 years agoMerge pull request #313 from paulusmack/fixes
Michael Neuling [Wed, 11 Aug 2021 06:25:35 +0000 (16:25 +1000)]
Merge pull request #313 from paulusmack/fixes

Fix bug causing FP unavailable interrupts to be missed

3 years agocore: Add a basic performance monitor unit (PMU) implementation
Paul Mackerras [Mon, 26 Apr 2021 08:09:48 +0000 (18:09 +1000)]
core: Add a basic performance monitor unit (PMU) implementation

This is the start of an implementation of a PMU according to PowerISA
v3.0B.  Things not implemented yet include most architected events,
the BHRB, event-based branches, thresholding, MMCR0[TBCC] field, etc.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agoReduce Yosys ECP5 cell usage by 30% with -abc9 -nowidelut
Anton Blanchard [Wed, 11 Aug 2021 04:34:41 +0000 (14:34 +1000)]
Reduce Yosys ECP5 cell usage by 30% with -abc9 -nowidelut

We've been investigating why the barrel rotator uses an enormous
number of cells on the yosys ECP5 target. Eventually it was narrowed
down to the -abc9 -nowidelut options, which see the cell count go from
4985 cells to 841 cells.

Using the same options on an Orange Crab build reduces the cell count
from 50864 to 36085. The main differences:

     LUT4                        31040 -> 25270
     PFUMX                        6956 ->     0
     L6MUX21                      1746 ->     0
     CCU2C                        2066 ->  1759

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agoMerge pull request #311 from antonblanchard/litesdcard-nexys-video
Michael Neuling [Tue, 10 Aug 2021 23:48:07 +0000 (09:48 +1000)]
Merge pull request #311 from antonblanchard/litesdcard-nexys-video

Update litesdcard from upstream and add Nexys Video support

3 years agoMerge pull request #312 from shenki/sdcard-soc-features
Michael Neuling [Tue, 10 Aug 2021 23:33:36 +0000 (09:33 +1000)]
Merge pull request #312 from shenki/sdcard-soc-features

litedram: Add sdcard to soc features

3 years agotests/fpu: Test FPU unavailable interrupt following a load
Paul Mackerras [Wed, 26 May 2021 07:44:19 +0000 (17:44 +1000)]
tests/fpu: Test FPU unavailable interrupt following a load

This adds a load before a floating-point load which should generate a
floating-point unavailable interrupt, to test for the bug where
unavailability interrupts can get dropped while loadstore1 is
executing instructions.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agolitedram: Add sdcard to soc features
Joel Stanley [Tue, 10 Aug 2021 09:19:29 +0000 (18:49 +0930)]
litedram: Add sdcard to soc features

Signed-off-by: Joel Stanley <joel@jms.id.au>
3 years agoexecute1: Handle interrupts during sequences of load/store operations
Paul Mackerras [Wed, 26 May 2021 07:34:12 +0000 (17:34 +1000)]
execute1: Handle interrupts during sequences of load/store operations

At present the logic prevents any interrupts from being handled while
there is a load/store instruction (one that has unit=LDST) being
executed.  However, load/store instructions can still get sent to
loadstore1.  Thus an instruction which should generate an interrupt
such as a floating-point unavailable interrupt will instead get
executed.

To fix this, when we detect that an interrupt should be generated but
loadstore1 is still executing a previous instruction, we don't execute
any new instructions, and set a new r.intr_pending flag.  That results
in busy_out being asserted (meaning that no further instructions will
come in from decode2).  When loadstore1 has finished the instructions
it has, the interrupt gets sent to writeback.  If one of the
instructions in loadstore1 generates an interrupt in the meantime, the
l_in.interrupt signal gets asserted and that clears r.intr_pending, so
the interrupt we detected gets discarded.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
3 years agolitesdcard: Add Nexys Video support
Anton Blanchard [Mon, 9 Aug 2021 20:41:45 +0000 (06:41 +1000)]
litesdcard: Add Nexys Video support

This board has a reset line that needs to be held low to power up the
SD card hardware.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>
3 years agolitesdcard: Use vendor not board type
Anton Blanchard [Mon, 9 Aug 2021 20:37:09 +0000 (06:37 +1000)]
litesdcard: Use vendor not board type

litesdcard provides a macro per vendor (eg xilinx, lattice) and not per
board, so modify the fusesoc generator to take a vendor. This will make
it easier to add litesdcard to more boards.

Signed-off-by: Anton Blanchard <anton@linux.ibm.com>