Discussion about "Getting started with Debian Linux on RISC-V with SymbiFlow tutorial"

34 views
Skip to first unread message

Karsten Merker

unread,
May 11, 2019, 8:27:38 PM5/11/19
to linux...@googlegroups.com, Michael Gielda, Tim Ansell, mer...@debian.org, ma...@debian.org
Hello,

there has been some discussion by email between a number of
interested people about the ideas outlined at
https://github.com/SymbiFlow/ideas/issues/35 ("Getting started with
Debian Linux on RISC-V with SymbiFlow tutorial") as not everybody
involved has a github account and contributing to issues on github
is impossible for people without a github account. During this
email exchange it has been proposed to move the discussion to the
linux-litex list, therefore I'd hereby like to post a summary of
what has been previously discussed:

- To implement usable Linux-capable SoCs on small- to
medium-sized FPGAs the only reasonable base ISA is RV32;
RV64 is not practical there.

- Debian currently only supports RV64.

A Debian port for RV32 is possible in principle once we have a
stable ABI for RV32 (which will hopefully be the case with the
release of glibc 2.30) and there are people who are willing to
work on a bootstrap, but there are a number of things to
consider:

- Debian is a general-purpose Linux distribution and a
significant part of its users are in the desktop and server
space. As such, Debian builds its packages with a lot of
optional functionality enabled and therefore the memory
footprint of Debian packages is usually significantly larger
than the memory footprint of code built by build-systems
specifically targeting low-memory fixed-function embedded
devices such as e.g. buildroot.

Most FPGA boards come with very small amounts of RAM; the ECP5
board with the largest amount of memory appears to be the ECP5
versa board, and that has only 128MB of DRAM. Running Debian
in 128MB of RAM is technically possible, but it isn't fun.
Anything even remotely practically usable requires at least
256MB of RAM. The typical amount of memory on today's
single-board computers (which define the lower end of what
general-purpose Linux distributions usually target) is 1GB of
RAM (2GB on the better-equipped models) and compiling/linking
big packages nowadays often requires 4GB of RAM or even more.

The only commonly available and hobbyist-affordable FPGA board
with more than 128MB of RAM appears to be the Digilent Arty-A7
which comes with 256MB of RAM, but isn't yet supported by the
open-source FPGA toolchains. Tim is rather confident though
that Project X-Ray will have made enough progress by the end of
the year to make building VexRiscV for the Artix-7 possible.

- If we are to start a Debian port for RV32, we have to define an
appropriate hardware baseline. The Debian riscv64 port uses
RV64GC=RV64IMAFDC as its baseline, but for FPGA implementations,
having the F and D extensions doesn't really make sense as they
would probably require vastly more logic ressources than the
complete rest of the SoC together, so RV32GC doesn't make much
sense when also targeting FPGAs. The work-in-progress RISC-V
unix platform specification will probably declare the C
extension as mandatory, the A extention is mandatory for Linux,
so that leaves RV32IMAC or RV32IAC as potential hardware
baselines for anything that would also target FPGA
implementations. As has been pointed out in the previous
discussion, implementing hardware multiply doesn't seem to be
much of a problem, but hardware division appears to be. ISTR
that an older version of the RISC-V ISA spec explicitly allowed
claming support for the M extension if only multiply was
implemented in hardware and divide was emulated via an M-mode
trap handler, but I cannot find that anymore in the current
spec. Does anybody have further information on this?

I don't have any access to the closed-door discussions of the
RISC-V foundation's unix platform spec working group, but based
on previous public discussions I would guess that the WIP
RISC-V unix platform spec will probably define RV32IMAC as its
baseline for RV32.

- An RV32IMAC port would play roughly in the same league as the
existing Debian armel (armv5 without FPU) port and there have
already been multiple attempts at phasing out the armel port
and only keeping the armhf port (armv7 + vfp3d16 FPU) in
Debian. For the current release cycle (Buster), armel will be
kept, but for how long afterwards has to be seen.

There are a number of reasons for dropping the armel port. Some
of them potentially also apply to an RV32IMAC port, some don't:

- Devices targeted by armel are usually severely
memory-constrained. This is something they share with most
off-the-shelf FPGA boards.

- There are more and more toolchain issues and many upstream
projects are dropping support for anything before armv7.
For RV32, the toolchain is actively maintained, so that's
currently not an issue. Last time I looked, there were quite
a number of upstream projects which had support for RV64,
but not for RV32, though. How things develop there will have
to be seen.

- Lack of appropriate build hardware. Most systems targeted
by the armel port aren't "beefy" enough both in terms of
memory and CPU power to keep up with building the archive.
Running armv5 code on modern arm64 systems doesn't work as
ARM has done some incompatible changes in the ISA after v5.
For RÍSC-V we are in a similar situation as RV32 and RV64 are
incompatible ISAs, i.e. we cannot just run an RV32 builder
in a chroot on a beefy RV64 system. For a second-tier port
(i.e. one in the "Debian-Ports" archive), running builders
in qemu VMs is acceptable, but for first-tier ports it's not.

Bootstrapping a Debian port for RV32 is therefore only useful if
there are systems with enough memory to make use of it, and
regarding off-the-shelf-hardware things look rather dire at the
moment. What would give the whole topic of running Linux/RISC-V
on FPGAs a big boost would be the availability of a suitable FPGA
board aimed at being used as a single-board computer, i.e. with
a large FPGA such as an ECP5-85k, enough DRAM, an SD-card slot,
an ethernet PHY and if possible some further SBC-typical I/O
interfaces. Until recently, the lack of a working open-source
DDR DRAM controller for the ECP5 was a big hindrance for using
DDR2/DDR3 RAM on OSHW FPGA boards, but since David Shah has
ported LiteDRAM to the ECP5, this has become possible now.

Is there perhaps somebody around who is interested and has the
necessary skills to tackle such a hardware design? In principle
designing such a board should be possible with an all-open-source
workflow if one looks e.g. at the Olimex A64-OLinuXino, an arm64
single-board computer with 2GB of DDR3 memory, gigabit ethernet and
everything else that one usually expects from an SBC, which has
been designed completely with only open-source tools:
https://www.olimex.com/Products/OLinuXino/A64/A64-OLinuXino/open-source-hardware

Regards,
Karsten
--
Ich widerspreche hiermit ausdrücklich der Nutzung sowie der
Weitergabe meiner personenbezogenen Daten für Zwecke der Werbung
sowie der Markt- oder Meinungsforschung.

Ewen McNeill

unread,
May 11, 2019, 11:15:15 PM5/11/19
to linux...@googlegroups.com, Michael Gielda, Tim Ansell, mer...@debian.org, ma...@debian.org
On 2019-05-12 12:27, Karsten Merker wrote:
> [...]
> Debian Linux on RISC-V with SymbiFlow tutorial") [...]
> I'd hereby like to post a summary of what has been previously discussed:
>
> - To implement usable Linux-capable SoCs on small- to
> medium-sized FPGAs the only reasonable base ISA is RV32;
> RV64 is not practical there.
>
> - Debian currently only supports RV64.
>
> A Debian port for RV32 is possible in principle once we have a
> stable ABI for RV32 (which will hopefully be the case with the
> release of glibc 2.30)

Thanks for posting a summary of the discussion, it's useful to have all
the details in one place.

As a long term user of Debian Linux (20+ years, over lots of platforms),
and someone interested in seeing Linux running on FPGA Soft CPUs, I'm
quite interested in the idea of seeing a Debian derived Linux runable on
a FPGA SoftCPU -- and Risc V 32-bit seems the best possible modern
option, due to the hardware constraints you outlined.

I think there's an important distinction here, between:

- creating a Debian-package derived RV32 distribution for FPGA soft CPU
usage, and

- creating a full Debian port to RV32.

The first (Debian-package derived for FPGA soft CPU) seems to me a great
idea -- pull together a hardware baseline, including a RISC V feature
set to target, and get enough Debian packages building for it so that we
can bring up a small Debian Linux derived system with serial console,
etc, and end up with something that can be self-hosting. To my mind
this is like, eg, Raspbian for the Raspberry Pi -- Debian derived, but
not trying to be Everything Debian.

The second (full Debian port), strikes me as both a difficult target
(porting *everything* in Debian is a lot of work), and not particularly
useful. Anyone who *actually* wants to run "Real Debian" on a RISC V
platform is almost certainly going to target RV64. And building
everything would probably require a lot more RAM that is likely to be
common on any reasonably affordable FPGA platform.

To my mind the ideal sizing target would be to aim to fit onto:
- Digilent Arty A7 (256MB RAM)

- Lattice Versa ECP5 5G dev board (128MB RAM)

with enough to be useful at the command line. These are actually
available now hardware targets, and 128MB-256MB RAM is a common sort of
level on higher end FPGA dev boards. (IIRC the Arty A7 board is about
US$120, and the Versa ECP5 5G dev board is about US$250.)

While it might be nice to dream of making a custom FPGA board with lots
more RAM, by the time you get to, eg, 1GB/2GB of RAM, it probably makes
sense to run RV64 anyway, for more software options. I think any custom
FPGA board like that would be sufficiently uncommon that it wouldn't
solve many users problems (as most of them wouldn't have it). And
probably cost sufficiently much that users might as well, eg, buy a Risc
V board from SiFive or similar. Whereas something that could run on
Digilent Arty A7 or Lattice Versa ECP5 5G, stands a good chance of
running on other mid-range FPGA dev boards as they come out, and thus
accessible to users for US$100-US$300.

> - If we are to start a Debian port for RV32, we have to define an
> appropriate hardware baseline. [...] The work-in-progress RISC-V
> unix platform specification will probably declare the C
> extension as mandatory, the A extention is mandatory for Linux,
> so that leaves RV32IMAC or RV32IAC as potential hardware
> baselines for anything that would also target FPGA
> implementations.

I suspect RV32IMAC is probably the more useful platform to target if
possible. With possibly multiply implemented directly in the FPGA, and
divide emulated.
> that an older version of the RISC-V ISA spec explicitly allowed
> claming support for the M extension if only multiply was
> implemented in hardware and divide was emulated via an M-mode
> trap handler, but I cannot find that anymore in the current
> spec. Does anybody have further information on this?

The RISC V architecture design paper:

https://people.eecs.berkeley.edu/~krste/papers/EECS-2016-1.pdf

talks about systems emulating various harder to implement features. Eg,
misaligned access (p21 / 34 in the PDF), system instructions (p25 / 38),
and in particular division instructions (p32 / 45; s4.1):

"""
On the other hand, for many FPGA implementations, division is
quite a bit more expensive to implement than multiplication,
thanks to the presence of hardened multiplier blocks; these processors
may choose to trap division instructions and emulate them in software.
"""

so it definitely seems *intended* that a CPU could choose to microcode
something like division (ie, trap on that instruction, and do something
behind the scenes to emulate it). That seems like a general design
strategy throughout the CPU.

Division is normally expected to be slow anyway, and should be
relatively uncommon in the instruction stream, so trapping division and
emulating it slowly seems fine. (Software builds without mul/div
instructions generally link in a software version via a compiler
intrinsic, that's going to do the same thing as emulation: replace the
multiply/divide with a software function call.)

I suspect multiply is more common anyway, so it'd be nice to get the
multiply "at hardware speeds" by using RV32IMAC, even if divide is still
relatively slow.

> - An RV32IMAC port would play roughly in the same league as the
> existing Debian armel (armv5 without FPU) port and there have
> already been multiple attempts at phasing out the armel port
> and only keeping the armhf port (armv7 + vfp3d16 FPU) in
> Debian. For the current release cycle (Buster), armel will be
> kept, but for how long afterwards has to be seen.

This is one of the main reasons that I think it makes sense to aim at a
"Debian derived" distribution, riding on the coat tails of the work on
the Debian RV64 port, but not trying to be a "real" Debian port. Linux
in general, and Debian in particular, are generally moving away from
targeting 32-bit platforms (outside of IoT kinds of usage), so trying to
be a real port would be an uphill battle.

> Bootstrapping a Debian port for RV32 is therefore only useful if
> there are systems with enough memory to make use of it, and
> regarding off-the-shelf-hardware things look rather dire at the
> moment.

I think the useful use case here is "FPGA soft CPU that needs more
software functionality than MicroPython, but not enough to justify using
a board with a real CPU / RAM along side the FPGA". Hence the
suggestions above to fit into 128MB/256MB of RAM.

I've got Debian VMs with 256MB of RAM (x86 / x86-64) and they're still
useful for specific purposes (eg, run specific daemons, and some CLI
stuff for maintenance). For "no daemons", "simple CLI", 128MB might
even be sufficient.

FWIW, one of my 256MB VMs currently claims 180MB of "available RAM" --
ie, could be stolen from file buffers / free space, etc -- and only
about 34MB actually actively used.

Ewen

Karsten Merker

unread,
May 12, 2019, 2:02:00 PM5/12/19
to Ewen McNeill, linux...@googlegroups.com, Michael Gielda, Tim Ansell, mer...@debian.org, ma...@debian.org
On Sun, May 12, 2019 at 03:15:06PM +1200, Ewen McNeill wrote:

> As a long term user of Debian Linux (20+ years, over lots of
> platforms), and someone interested in seeing Linux running on
> FPGA Soft CPUs, I'm quite interested in the idea of seeing a
> Debian derived Linux runable on a FPGA SoftCPU -- and Risc V
> 32-bit seems the best possible modern option, due to the
> hardware constraints you outlined.
>
> I think there's an important distinction here, between:
>
> - creating a Debian-package derived RV32 distribution for FPGA
> soft CPU usage, and
>
> - creating a full Debian port to RV32.
>
> The first (Debian-package derived for FPGA soft CPU) seems to
> me a great idea -- pull together a hardware baseline, including
> a RISC V feature set to target, and get enough Debian packages
> building for it so that we can bring up a small Debian Linux
> derived system with serial console, etc, and end up with
> something that can be self-hosting. To my mind this is like,
> eg, Raspbian for the Raspberry Pi -- Debian derived, but not
> trying to be Everything Debian.

Hello,

in fact Raspbian is trying to be "Everything Debian" ;-), just
built with a compiler targeting armv6+vfp2 instead of
armv7+vfp3d16. AFAIK Raspbian runs a standard Debian autobilder
infrastructure and generally builds the whole archive; the only
difference to "regular" Debian autobuilders is that it doesn't
push the build results into Debian's archive but into a separate
one. The trick that is used by Raspbian is overloading the
meaning of "armhf" with something different from what it means in
Debian. For Raspbian that has the advantage that Raspbian
doesn't need any adjustments in Debian infrastructure and key
packages (e.g. dpkg, binutils, kernel, wanna-build, etc) and can
just run build chroots with modified compilers but otherwise
pretend to be the "real" Debian/armhf.

Something like that doesn't work for a potential RV32 port,
though, which in contrast to Raspbians armv6+vfp2 "armhf" is a
distinctly different architecture from all existing ones, i.e.
it needs to be plugged into various pieces of Debian
infrastructure to make it feasible to upload source packages to
the Debian archive which refer to riscv32. There are quite a
number of cases where the control files of Debian packages
contain architecture-specific options, and DAK, the "Debian
Archive Kit" doesn't accept any source package that contains
references to an architecture that it doesn't know about.
Therefore without having riscv32 as an official Debian
architecture (tier-two/"ports architecture", but nonetheless),
one would have to manually keep forks of a significant number of
packages up to date, and that doesn't scale. I'm talking from
practical experience here; we were exactly in that situation at
the beginning of the Debian riscv64 port and I really don't want
to be in that situation for the whole lifetime of a potential
riscv32 port. So if a potential riscv32 port is intended as a
"living" port that follows the development of Debian/unstable
instead of being a one-shot thing, getting it accepted as an
official architecture is effectively a must. As there is a cost
associated with getting a new architecture into Debian (both in
terms of manpower and hardware), doing so requires making
plausible that adding the new port "makes sense" for the Debian
project at large and that there is a realistic use case for it.
I have severe doubts that "we target only tiny system with 128MB
of RAM" would be accepted as a sufficient use case by the teams
that would have to do the necessary infrastructure work within
Debian.

> While it might be nice to dream of making a custom FPGA board
> with lots more RAM, by the time you get to, eg, 1GB/2GB of RAM,
> it probably makes sense to run RV64 anyway, for more software
> options.

Not necessarily. As we are talking about FPGA implementations,
an RV64GC SoC that can run the existing Debian riscv64 port in a
practically usable fashon isn't realistic with the FPGAs that we
have access to and if the argument would be "use an RV64GC ASIC",
we would be on the wrong mailinglist ;-).

From what people have told me, there is definitely interest in
having a practically usable fully-open-source system down to the
digital logic level. People have told me that they can live with
the (compared to ASICs) low clockrates on FPGAs, but they need an
otherwise practically usable system and that means enough memory.
I am rather confident that "we have an FPGA-based open-source
hardware SBC that can run the Debian riscv32 port well" would be
a compelling argument for the relevant teams in Debian to accept
riscv32 as an additional architecture.

> I think any custom FPGA board like that would be sufficiently
> uncommon that it wouldn't solve many users problems (as most of
> them wouldn't have it). And probably cost sufficiently much
> that users might as well, eg, buy a Risc V board from SiFive or
> similar. Whereas something that could run on Digilent Arty A7
> or Lattice Versa ECP5 5G, stands a good chance of running on
> other mid-range FPGA dev boards as they come out, and thus
> accessible to users for US$100-US$300.

I'm not a hardware designer, so I'm unfortunately not qualified
to make cost estimates here, but from a layman's view I ask
myself: if Olimex, a small, "family shop"-style manufacturer
within the EU can produce the A64-OLinuXino-1Gs16M OSHW board in
comparatively small quantities for ~46,00 EUR incl. taxes (arm64
SoC, 1GB RAM, Gigabit Ethernet, 16GB eMMC, SD card slot, various
I/O), wouldn't an FPGA boad with an ECP5 and 1GB of RAM be
possible for a price point comparable to an ECP5 Versa board
(~187 EUR incl. taxes for the non-5G version, ~225 EUR incl.
taxes for the 5G version), i.e. roughly 4 to 5 times the price
of the A64-OLinuXino? I would appreciate very much if somebody
who has experience with these kind of things could chime in.

Ewen McNeill

unread,
May 12, 2019, 6:56:19 PM5/12/19
to Karsten Merker, Ewen McNeill, linux...@googlegroups.com, Michael Gielda, Tim Ansell, ma...@debian.org
Hi Karsten,

On 2019-05-13 06:01, Karsten Merker wrote:
> in fact Raspbian is trying to be "Everything Debian" ;-), just
> built with a compiler targeting armv6+vfp2 instead of
> armv7+vfp3d16. AFAIK Raspbian runs a standard Debian autobilder
> infrastructure and generally builds the whole archive

Wow, I didn't realise Raspbian were building everything! I'd always
assumed it was a cut down distro to a useful core set of packages :-)
(As you say it probably helps that they're mostly just compile flags
different from an existing Debian port.)

> [...] There are quite a
> number of cases where the control files of Debian packages
> contain architecture-specific options, and DAK, the "Debian
> Archive Kit" doesn't accept any source package that contains
> references to an architecture that it doesn't know about.

Ah.

I'd been assuming that we could at least get the RV32 specific bits
included in via the existing RV64 port, without too much confusion
(since it's similar tooling, just with a different RISC-V subset). It
sounds like that might not work for enough packages to be useful.

> one would have to manually keep forks of a significant number of
> packages up to date, and that doesn't scale.

I definitely agree that maintaining an extensive set of "unmerged"
patches in order to be able to build things is an exercise in a lot of
busy work, and to be avoided (or at least only done for as long as
necessary to do the initial bootstrap). So we'd need some way to get
relevant changes "upstream" (be that the Debian archive for just
"compile flags" type changes, or the upstream project for anything else).

Relevant links:

https://wiki.debian.org/Ports
https://www.debian.org/ports/

the first one in particular provides hints on porting some of the
"harder to port" (eg, self hosting) things, like Haskell, Free Pascal,
OpenJDK, Rust, etc. It's basically those harder to port things that I
was hoping it'd be possible to avoid needing to port, by just targeting
a subset of the archive. Because in practice it seems unlikely many
would actually use those tools / packages built with those tools, on a
RV32 port targeted at fairly small FPGA systems.

> I have severe doubts that "we target only tiny system with 128MB
> of RAM" would be accepted as a sufficient use case by the teams
> that would have to do the necessary infrastructure work within
> Debian.

Is it plausible to take the idea of "tier 3" ports back to Debian for
wider discussion? With the idea being that they *don't* have all the
Debian build/release infrastructure set up, but *are* allowed to include
architecture-specific sections options in the control files?

(I've followed Debian for more than long enough to know that could be a
opening can of worms :-) But it seems to me that many of the "second
tier" Debian ports that have struggled to keep up with porting
everything might benefit from an approach like that -- still be able to
have compile flags, etc, for their architecture merged into Debian, but
not be expected to "build everything, all the time".)

> From what people have told me, there is definitely interest in
> having a practically usable fully-open-source system down to the
> digital logic level. People have told me that they can live with
> the (compared to ASICs) low clockrates on FPGAs, but they need an
> otherwise practically usable system and that means enough memory.

Interesting. It's definitely an intellectually attractive idea, of
"open source down to the CPU gates". And "larger DDR[34] memory" isn't
a particularly difficult thing to throw onto a board custom designed for
this purpose (getting from 512MiB to 2GiB is, in theory, just a few more
RAM chips, and some more circuit board routing).

It's unclear to me whether it'd be possible to sell enough of those (say
Lattice ECP5 5G 85K LUTs, 2GiB RAM, maybe SD card) to get the price to a
level where more than a handful would be bought.

The Lattice LFE5UM5G-85F-8BG756C FPGA (eg
https://www.mouser.com/ProductDetail/Lattice/LFE5UM5G-85F-8BG756C?qs=sGAEpiMZZMsg%252Bek5lc7F5tI%2FGQe9yvf%252BZp%252Be%2FGdSypg%3D)
isn't that expensive compared with say an x86 desktop CPU (US$64,
quantity 1), and pretty much all the other requirements (RAM, Ethernet,
flash / storage, etc) are pretty comparable. But there's a non-trivial
amount of design work required for the circuit board, and non-trivial
manufacturing for the sorts of pin packages required for that large a
FPGA. So there's a lot of up front costs, which are difficult to recoup
at a low sale price without decent volume.

> [...] wouldn't an FPGA boad with an ECP5 and 1GB of RAM be
> possible for a price point comparable to an ECP5 Versa board
> (~187 EUR incl. taxes for the non-5G version, ~225 EUR incl.
> taxes for the 5G version), i.e. roughly 4 to 5 times the price
> of the A64-OLinuXino? I would appreciate very much if somebody
> who has experience with these kind of things could chime in.
I think you're right that the 225 EUR/ US$250 sort of price range for
the Versa ECP5 5G board is probably close to the upper limit of what
people would want to pay (given other options with hard CPUs, under 100
EUR / US$100).

As another cost reference, the NeTV2, also fairly low volume custom
design (without the extra RAM, as it's aimed at something else), ends up
being US$350-US$450:

https://www.crowdsupply.com/alphamax/netv2

(The US$450 one is a limited run with a large FPGA -100T; the regular
-35T FPGA is a medium sized one.)

Obviously more people would buy at, say, US$199 versus US$350, and more
still at US$100. But at the end of the day from a Debian port point of
view it's still a "slow expensive CPU".... so the extra money is mostly
buying openness. Would there be, eg, hundreds of buyers?

Ewen

Ewen McNeill

unread,
May 14, 2019, 8:13:37 PM5/14/19
to Manuel A. Fernandez Montecelo, Karsten Merker, linux...@googlegroups.com, Michael Gielda, Tim Ansell, Manuel A. Fernandez Montecelo
Hi Manuel,

On 2019-05-14 12:36, Manuel A. Fernandez Montecelo wrote:
> Em seg, 13 de mai de 2019 às 00:56, Ewen McNeill
> <linux...@ewen.mcneill.gen.nz> escreveu:
>> I'd been assuming that we could at least get the RV32 specific bits
>> included in via the existing RV64 port, without too much confusion
>
> No, it will not work straight away, what has been done for riscv64 is
> not immediately useful to start riscv32 (*).

As you said, the major advantage of Debian is that it's ported to
big-endian and little-endian systems, on 32-bit and 64-bit systems.
Thus there's a lot more portability for likely combinations of issues.
And having ported to RV64 already, I'm assuming there's some "#ifdef
RISCV" style bits in code/Makefiles, which also might already help a
RV32 port if it's still relevant.

Possibly changes to the source code / Makefiles / configure scripts for
RV32 could go in via the RV64 port given cooperative maintainers. But I
understand your point that anything requiring packaging metadata changes
(debian/rules, RV32 only patches, etc) would need dpkg to know about the
architecture.

> OpenJDK is used by db5.3 for example, it needs it to build bindings of
> the library for Java, and db5.3 is a pretty basic library used in tons
> of software

Oh, yes, I'd forgotten that particular challenge (and I've even run into
it myself trying to build Berkeley DB from source on other platforms
without Java). Bootstrapping is hard!

> What it can be done more easily and with almost no requirement of
> effort from maintainers is to keep an up-to-date script which allows
> to bootstrap more easily from other architectures, so for example
> images could be created every now and then for end-users.

This sounds like the most useful interim step (before some eventual
"ideal hardware to justify a full port" emerges). At least being able
to auto-build a useful subset of Debian packages for RV32 would probably
a helpful "Linux on RV32 on FPGA" bootstrapping step.

> Conversely, I don't see many people in Debian gladly accepting patches
> for ports that don't have hardfloat support,

Interesting point. This is another source of "small FPGA" versus
"useful target CPU features" tensions. hardfloat support tends to a
non-trivial consumption of FPGA/silicon space. Possibly having the RV32
CPU "microcode" the floating point instructions when built for a smaller
FPGA target, and implement them directly when built for a larger FPGA
target, might provide a way forward for a common compile target.

Off the top of my head I think most of the open source RISC-V on FGPA
implementations have hardfloat on their "todo" list rather than
implemented. But maybe in the next 6-12 months that "compile time
option" for hardfloat becomes realistic.

> As I mentioned in previous e-mails of the related discussion, I think
> that there's some disconnect and wide gap between using Debian in a
> system and not implementing MUL or DIV in the hardware running it
> because it's too expensive -- I fear that the expectations and
> mindsets might be too far apart to make this work in the long term.

Yes, that's my concern too. That the overlap between "small RV32 on
FPGA" and "sufficiently widely usable CPU to justify a RV32 Debian port"
is just too small. As discussed earlier, the RAM issue is solvable with
a bit of custom hardware. But the generalised nature of FPGAs is
inherently not well suited to really fast CPU clocks.

Especially if a RV32 on FPGA port were to be self-hosting, the slow CPU
clocks (and thus longer build times) could be quite an issue for a full
Debian port. And there's clearly a tension between FPGA size and
software-target feature set.

I'm still interested in the idea of *building* the Debian core packages
for RV32 aimed at a smaller target, as it'd provide something more
interesting than buildroot-with-busybox to run.

But it's less clear how practical it is to make that a full Debian port,
at least without a specific host FPGA board that makes that more
practical and people who want the "Open Source to the CPU
implementation" nature.

Perhaps the newest set of Intel issues
(https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html)
might find more people who want an "open source CPU" though :-)

Ewen

Charles Papon

unread,
May 22, 2019, 3:48:02 PM5/22/19
to Linux for LiteX FPGA SoC
About BOM cost. At very low cost a simple SoC should be feasible in a 5.5$ LFE5U-12F
And for sure, there should be more than enough space in a 9.5$ LFE5U-25F

Hasjim Williams

unread,
May 22, 2019, 11:07:19 PM5/22/19
to linux...@googlegroups.com
The die on those two ICs are the same. 

nextpnr uses --25k
ecppack uses --idcode 0x21111043 for the 12F

Ewen McNeill

unread,
Jun 4, 2019, 1:58:08 AM6/4/19
to linux...@googlegroups.com, Karsten Merker, Michael Gielda, Tim Ansell, ma...@debian.org
On 2019-05-13 10:56, Ewen McNeill wrote:
> Interesting.  It's definitely an intellectually attractive idea, of
> "open source down to the CPU gates".  And "larger DDR[34] memory" isn't
> a particularly difficult thing to throw onto a board custom designed for
> this purpose (getting from 512MiB to 2GiB is, in theory, just a few more
> RAM chips, and some more circuit board routing).

For future reference, Dave Shah has designed a ECP5 FPGA development
board with 1GB of RAM:

https://github.com/daveshah1/TrellisBoard

(in a repository titled "Ultimate ECP5 development board" :-) )

AFAICT it's only a prototype (with only 3 actual hardware items ever
created I believe), but I suppose there's a chance that Dave Shah (or
someone else) might be persuaded to do a (small?) production run. I
suspect the cost would be towards the upper end, given it's got the
largest/fastest ECP5 chip on it, plus a bunch of peripherals. But at
least there's a plausibly close design already existing.

The license seems to be ISC with modifications to make it more
applicable to open hardware
(https://github.com/daveshah1/TrellisBoard/blob/master/COPYING), which
is also a useful start. (Eg, it might be possible to get to 2GB RAM
with either larger chips, or rodoing the layout to get a second pair of
RAM chips on the board.)

There's a bit more discussion from #litex on IRC yesterday/today:

https://logs.timvideos.us/%23litex/%23litex.2019-06-03.log.html

Apparently the full prototype assembly service for 3 cost US$2142, which
is about US$700 per board (that was "about $525 for the bare board, $800
for parts and $790 for assembly", total across the 3).

Even allowing for cheaper circuit boards, and cheaper assembly, and bulk
ordering of parts, etc, that still feels like it'd be a US$350-US$400
assembled board, minimum. Maybe a wee bit cheaper with a smaller FPGA
(that FPGA is US$70 quantity 1).

Ewen

PS: Also apparently getting more than 256MB RAM running on Litex
required a patch, but presumably that'll get incorporated upstream
eventually.

Karsten Merker

unread,
Jun 5, 2019, 5:21:12 PM6/5/19
to Ewen McNeill, linux...@googlegroups.com, Karsten Merker, Michael Gielda, Tim Ansell, ma...@debian.org
On Tue, Jun 04, 2019 at 05:58:02PM +1200, Ewen McNeill wrote:

> For future reference, Dave Shah has designed a ECP5 FPGA
> development board with 1GB of RAM:
>
> https://github.com/daveshah1/TrellisBoard
>
> (in a repository titled "Ultimate ECP5 development board" :-) )

Wow, that's a very impressive piece of work!

> AFAICT it's only a prototype (with only 3 actual hardware items
> ever created I believe), but I suppose there's a chance that
> Dave Shah (or someone else) might be persuaded to do a (small?)
> production run. I suspect the cost would be towards the upper
> end, given it's got the largest/fastest ECP5 chip on it, plus a
> bunch of peripherals. But at least there's a plausibly close
> design already existing.
[...]
> https://logs.timvideos.us/%23litex/%23litex.2019-06-03.log.html
>
> Apparently the full prototype assembly service for 3 cost
> US$2142, which is about US$700 per board (that was "about $525
> for the bare board, $800 for parts and $790 for assembly",
> total across the 3).
>
> Even allowing for cheaper circuit boards, and cheaper assembly,
> and bulk ordering of parts, etc, that still feels like it'd be
> a US$350-US$400 assembled board, minimum. Maybe a wee bit
> cheaper with a smaller FPGA (that FPGA is US$70 quantity 1).

That is probably indeed quite a bit over most people's spending
limit for an FPGA-based OSHW single-board-computer, but probably
still within "enthusiast" territory for a number of people. At
least for myself I can say that I would immediately buy the
TrellisBoard if it were available to me in this price range.
Reply all
Reply to author
Forward
0 new messages