New experimental Gentoo image

[Gabriele Svelto]

Hi all,
I've just finished assembling a new Gentoo image for the CI20. It still
needs a bit of polish but I'd be glad if more people could test it to
see if its stable, especially for those with boards that experienced
freezes and memory-related problems. The XZ compressed image is
available here:

https://drive.google.com/open?id=1Akr0f9dvOrwV5nz7_nPFqTaddOfffOgS

To install it unpack it and dd it to an SD-card and then follow the
usual procedure for flashing an image from an SD-card.

The image is based on the original Gentoo image but all software has
been updated. It contains a 3.18.118 kernel with all CI20-specific
patches rebased on top of it. The default compiler is gcc 6.4.0 but
version 7.3.0 is also available. I've left the minimal graphical
environment intact but I have removed the applications to reduce the
size of the image. More importantly this contains the memory timing
fixes [1] I've mentioned in a previous post. Before these changes my
board was unreliable, freezing at least once a day. With these changes
it's been fully stable, I kept it running continuous compilations for a
week without encountering a single issue.

The root user password is 'root' and the portage directory is empty, so
if you want to install other packages you'll need to download a portage
snapshot [2], unpack it under /usr and then select the mipsel o32
profile with the following command:

eselect profile set 7

The serial ports are configured like on the Debian image, with the
kernel output going to the dedicated UART and U-boot/console available
on both the main expansion header UART and the dedicated one. Enjoy!

Gabriele

[1] https://github.com/gabrielesvelto/CI20_u-boot/tree/fixed-timings
[2] http://distfiles.gentoo.org/snapshots/portage-latest.tar.xz

[Paul Boddie]

On Tuesday, August 21, 2018 at 12:38:39 AM UTC+2, Gabriele Svelto wrote:

Hi all,
I've just finished assembling a new Gentoo image for the CI20. It still
needs a bit of polish but I'd be glad if more people could test it to
see if its stable, especially for those with boards that experienced
freezes and memory-related problems.

I'm not likely to be testing this, or at least not immediately, but I would be interested in knowing what the process was to make Gentoo for the CI20. My reason for wanting to know a bit more about it is that I was trying to build the Parabola GNU/Linux-libre distribution (an Arch Linux derivative) for the CI20 using two different approaches, and neither of them proved convenient or yielded a result in the time I set aside to achieve something.

The first approach I used involved building packages on the CI20 itself under Debian. I chose this to avoid cross-compiling since Arch, like many distributions, doesn't really support cross-building the distribution. Here I experienced problems with getting packages to build against newly-built packages, not things on the system, and I find it astonishing how inconsistent and generally poor things like autotools are for this: one ends up doing all sorts of ...FLAGS hacks in an attempt to direct these tools to the proper resources; things like build, host and target separation are strictly for cross-compiling.

The second approach involves cross-building using knowledge gained about the way Arch packages are configured and built. Here, I didn't get particularly far because glibc wouldn't build, yielding a linker assertion error. (I get the impression that nobody cares about such errors, or not enough to be embarrassed that one effectively experiences the tool crashing in an ungraceful way rather than indicating the cause of the problem in the input.)

Since Gentoo is a build-from-source distribution, I suppose that such issues have to be dealt with, so it would be useful to know the general approach if it isn't too much of a burden for me to ask about it.

Paul

[Mathieu Malaterre]

Slightly off-topic, but you do understand that Debian archive contains mipsel binaries, right ?

As a side note, I have been very impressed by OpenELEC/LibreELEC simple Makefile system to build kernel + userspace, bootstrapping gcc/binutils/glibc for cross building. 

[Paul Boddie]

On Monday 27. August 2018 10.40.39 Mathieu Malaterre wrote:
> On Sunday, August 26, 2018 at 3:39:20 PM UTC+2, Paul Boddie wrote:
> >
> > The first approach I used involved building packages on the CI20 itself
> > under Debian.

[...]

> Slightly off-topic, but you do understand that Debian archive contains
> mipsel binaries, right ?

Yes, it would have to.

> As a side note, I have been very impressed by OpenELEC/LibreELEC simple
> Makefile system to build kernel + userspace, bootstrapping
> gcc/binutils/glibc for cross building.

I haven't tried that system, so it is useful to be reminded of it. Personally,
I have found Buildroot to be decent enough for building toolchains, and I was
looking at how glibc was being built by Buildroot.

My focus was on building more general distributions, though, and here I
perceive a gulf between things like Buildroot, OpenWrt/LibreWrt, and so on
which often emphasise images (even if they may provide some kind of
installable packages) and things like Debian which emphasise packages.

Paul

[Gabriele Svelto]

Hello Paul,

On 26/08/2018 15:39, Paul Boddie wrote:
> The first approach I used involved building packages on the CI20 itself
> under Debian. I chose this to avoid cross-compiling since Arch, like
> many distributions, doesn't really support cross-building the
> distribution. Here I experienced problems with getting packages to build
> against newly-built packages, not things on the system, and I find it
> astonishing how inconsistent and generally poor things like autotools
> are for this: one ends up doing all sorts of ...FLAGS hacks in an
> attempt to direct these tools to the proper resources; things like
> build, host and target separation are strictly for cross-compiling.

Doing that by hand is extremely time consuming and error prone.

> Since Gentoo is a build-from-source distribution, I suppose that such
> issues have to be dealt with, so it would be useful to know the general
> approach if it isn't too much of a burden for me to ask about it.

Since I had a starting stage4 tarball I did most of the work on the
board itself. I cross-compiled a modern kernel, then used the script [1]
to build an image with the new kernel and the old tarball. Once I did
that I slowly upgraded all the packages on the board itself. I did it
this way because I also wanted to test the board stability since it
always gave me problems.

However, building an image on Gentoo is very easy. First of all there's
crossdev [2] available which enables you to build packages locally as if
you were doing it within the target environment. Then the image can be
assembled using catalyst [3]. The trick is that you can always chroot
within an environment that contains the target files but is using the
cross-compiler to build stuff. I don't know if crossdev can be reused
for building other distros but it might be worth a try.

Gabriele

[1]
https://github.com/ZubairLK/ci20_other_files/blob/master/make-flash-card.sh
[2] https://wiki.gentoo.org/wiki/Cross_build_environment
[3] https://wiki.gentoo.org/wiki/Catalyst

[Paul Boddie]

On Monday 27. August 2018 16.05.01 Gabriele Svelto wrote:
>
> On 26/08/2018 15:39, Paul Boddie wrote:
> > The first approach I used involved building packages on the CI20 itself
> > under Debian. I chose this to avoid cross-compiling since Arch, like
> > many distributions, doesn't really support cross-building the
> > distribution. Here I experienced problems with getting packages to build
> > against newly-built packages, not things on the system, and I find it
> > astonishing how inconsistent and generally poor things like autotools
> > are for this: one ends up doing all sorts of ...FLAGS hacks in an
> > attempt to direct these tools to the proper resources; things like
> > build, host and target separation are strictly for cross-compiling.
>
> Doing that by hand is extremely time consuming and error prone.

Indeed. Still, it is incredible that here we are in the year 2018 and the
tools are still as cumbersome as they always have been. It should really be
possible to nominate filesystem locations not only for installation but also
as the sources of headers and libraries.

Maybe newer build tools try and deal with some of these issues, but they
always seem to have their own peculiarities, too.

[...]

> However, building an image on Gentoo is very easy. First of all there's
> crossdev [2] available which enables you to build packages locally as if
> you were doing it within the target environment. Then the image can be
> assembled using catalyst [3]. The trick is that you can always chroot
> within an environment that contains the target files but is using the
> cross-compiler to build stuff. I don't know if crossdev can be reused
> for building other distros but it might be worth a try.

It seems to involve qemu for some things, which always makes me a bit
skeptical. I guess you didn't need to resort to using qemu, though, because
you were able to build everything on the CI20 itself.

I have been told that Debian is much improved for cross-building, even though
doing so for the entire archive has never been an official objective, and that
general distribution bootstrapping is more approachable than ever. It will
soon be impossible for me to not investigate this, I imagine.

Thanks for the information about Gentoo!

Paul

[djordje....@rt-rk.com]

Hi Gabriele,

I've successfully booted gentoo on ci20 system using instructions

you have provided, and successfully test-ran chromium and cobalt

binaries (cross compiled on x86 using debian sysroot) on it.

No issues found during this time.

Great work, thanks!

Djordje

[Gabriele Svelto]

Hi Djordje,

thanks a lot! I should have some time next week to finish polishing it
and then I'll make a proper release with documentation and all. I want
to pull in the latest 3.18 kernel and ensure that GCC 7.3.0 produces
stable code, then rebuild everything using it.

Gabriele