Sage 5.4 on ARM

[Julien Puydt]

Hi,

I finished compiling sage 5.4 on ARM. I'll make a bdist available soon,
and run "make ptestlong" afterwards (and report).

The compilation went mostly fine ; the only caveat is that in libm4rie,
conversion.c took hours to build, and wouldn't build with MAKE="make -j
3" because it was too heavy for the box in that case.

Snark on #sagemath

[Julien Puydt]

Le 16/11/2012 09:10, Julien Puydt a �crit :

> I'll make a bdist available soon,

Here it is:
http://boxen.math.washington.edu/home/jpuydt/sage-5.4-armv7l-Linux.tar.xz

I'd like to know if it works on more recent ARM-based hardware ; for
example, if someone has an android tablet, it should be possible to
install ubuntu in a chroot (I think there's an app for that, something
like "Ubuntu noroot" or some such) and run the bdist from there.

I just started the tests, and as usual it starts by rebuilding the
docs... I'll report when it will be finished.

Snark

[mmarco]

I run sage on my tablet (asus transformer prime) precisely in that
way: over a ubuntu chroot. I have an old version installed though. I
have an ubuntu 10 chroot, where i compiled sage 4.8 (it was a pain:
over a week of comilation time).

Later i tested version 5.1 over a very minimal newer version of
ubuntu. It also runs fine.

Which version of ubuntu did you build it in?

On 16 nov, 12:53, Julien Puydt <julien.pu...@laposte.net> wrote:
> Le 16/11/2012 09:10, Julien Puydt a crit :
>
> > I'll make a bdist available soon,
>

> Here it is:http://boxen.math.washington.edu/home/jpuydt/sage-5.4-armv7l-Linux.ta...

[Julien Puydt]

Le 16/11/2012 16:53, mmarco a �crit :

> I run sage on my tablet (asus transformer prime) precisely in that
> way: over a ubuntu chroot. I have an old version installed though. I
> have an ubuntu 10 chroot, where i compiled sage 4.8 (it was a pain:
> over a week of comilation time).

What!? *A week*!?

If I don't err, this box has:
- 1Go of RAM (vs 512Mo)
- a quad-core nvidia tegra 3 (vs dual-core nvidia tegra 2)

Perhaps you didn't compile with export MAKE="make -j 3" ?

I was pondering buying one of those beasts (more powerful, more ram,
good autonomy) to replace my poor netbook (toshiba AC100, whose hinges
are starting to get bad -- mechanical problem), so I'm really interested
in understanding what happens.

> Which version of ubuntu did you build it in?

Ubuntu precise (12.04).

Snark

[mmarco]

Well, the main problem was overheating. Compilation failed several
times, the device turned off by itself. I even think it got damaged,
since the power button stopped working properly (luckily asus was kind
enough to replace it)

[Julien Puydt]

Le 16/11/2012 18:58, mmarco a �crit :

> Well, the main problem was overheating. Compilation failed several
> times, the device turned off by itself. I even think it got damaged,
> since the power button stopped working properly (luckily asus was kind
> enough to replace it)

Ouch. Bad, very bad, extremely bad!

I wouldn't have asked for a replacement but for a refound...

What model of asus transformer prime was it?

Snark on #sagemath

[Julien Puydt]

Le 16/11/2012 09:10, Julien Puydt a �crit :

> and run "make ptestlong" afterwards (and report).

Ok, now it's done.

The failing tests are the usual ones: the ones related to the gamma
function in the libc, and the maxima("1.7e+17") error.

No regression.

Snark on #sagemath

[Dima Pasechnik]

On 2012-11-16, Julien Puydt <julien...@laposte.net> wrote:

> Le 16/11/2012 16:53, mmarco a écrit :
>> I run sage on my tablet (asus transformer prime) precisely in that
>> way: over a ubuntu chroot. I have an old version installed though. I
>> have an ubuntu 10 chroot, where i compiled sage 4.8 (it was a pain:
>> over a week of comilation time).
>
> What!? *A week*!?
>
> If I don't err, this box has:
> - 1Go of RAM (vs 512Mo)
> - a quad-core nvidia tegra 3 (vs dual-core nvidia tegra 2)
>
> Perhaps you didn't compile with export MAKE="make -j 3" ?
>
> I was pondering buying one of those beasts (more powerful, more ram,
> good autonomy) to replace my poor netbook (toshiba AC100, whose hinges
> are starting to get bad -- mechanical problem), so I'm really interested
> in understanding what happens.

IMHO Samsung's new ARM Chromebook is what you might want; unfortunately
the internal SSD is small, only 16GB (like on AC100), but 1.7GHz
dual-core Cortex A15. (and 12"(?) screen)

http://www.chromium.org/chromium-os/developer-information-for-chrome-os-devices/samsung-arm-chromebook

Best,
Dima

[Julien Puydt]

Le 17/11/2012 10:13, Dima Pasechnik a �crit :

> IMHO Samsung's new ARM Chromebook is what you might want; unfortunately
> the internal SSD is small, only 16GB (like on AC100), but 1.7GHz
> dual-core Cortex A15. (and 12"(?) screen)
>
> http://www.chromium.org/chromium-os/developer-information-for-chrome-os-devices/samsung-arm-chromebook

The internal storage is small, but since it's possible to stick a big SD
in, that could fly (I need about 5Go for a system, 7Go for sage [did I
mention it's too much?] and 20Go [mostly] static storage).

Now I'll only
(1) make sure it can run ubuntu instead of the crappy chrome OS it ships
with
(2) wait until it's available in France (my AC100 isn't dead yet, so no
hurry).

I can't help but notice that you didn't comment on the thread's main
topic: the quite honest status of sage on ARM/ubuntu :-P

Snark on #sagemath

[mmarco]

It was a TF 201

Oh, and btw, i think the real bottleneck with it was not the
processor, but the disk access. The processor is not bad (but again,
it can overheat), but the access to the internal sd disk is too slow.

Definitely, not a machine designed with this purpose in mind. And i
guess that pretty much the same could be said for almost every tablet
or smartphone: no cooling system, usually slow internal drives... they
are thought for another kind of usage.

But if your process is not very disk intensive, it does a decent job.
Foir instance, now that i have sage installed, it takes a long time to
startup, but after that, the speed is decent.

[Julien Puydt]

Le 17/11/2012 12:03, mmarco a �crit :

> It was a TF 201
>
> Oh, and btw, i think the real bottleneck with it was not the
> processor, but the disk access. The processor is not bad (but again,
> it can overheat), but the access to the internal sd disk is too slow.
>
> Definitely, not a machine designed with this purpose in mind. And i
> guess that pretty much the same could be said for almost every tablet
> or smartphone: no cooling system, usually slow internal drives... they
> are thought for another kind of usage.
>
> But if your process is not very disk intensive, it does a decent job.
> Foir instance, now that i have sage installed, it takes a long time to
> startup, but after that, the speed is decent.

Same as my AC100 ; and I suspect it is likewise with the latest ARM
chromebook.

Though I think more ram should help, since the poor AC100 has both a
slow (and small) storage, and needs to swap to it a lot.

Snark

[Dima Pasechnik]

On 2012-11-17, Julien Puydt <julien...@laposte.net> wrote:
>
> The internal storage is small, but since it's possible to stick a big SD
> in, that could fly (I need about 5Go for a system, 7Go for sage [did I
> mention it's too much?] and 20Go [mostly] static storage).
>
> Now I'll only
> (1) make sure it can run ubuntu instead of the crappy chrome OS it ships
> with
> (2) wait until it's available in France (my AC100 isn't dead yet, so no
> hurry).

it might take a while, as they are apparently being sold out very
quickly...

>
> I can't help but notice that you didn't comment on the thread's main
> topic: the quite honest status of sage on ARM/ubuntu :-P

Since Sept. my nappy-changing duties take away a lot of time, so I didn't touch
ARM since. :-)

[mmarco]

I have tried your package in a chroot environment (actually in two, a
full ubuntu and a lubuntu), and it doesn't start in neither of them.

Stranegly, the problem seems to be different: in the lubuntu case, it
just shows the message with the version and quits. In the full ubuntu,
it complains with the message "can't import module sage", but for
example, sage -python works fine

[mmarco]

Nevermind, i found the problem: i had to use an armhf ubuntu.

[jaebond]

mmarco,

I had been working on this exact same issue a few months ago.  Would you mind explaining what you did to fix your issue?

[mmarco]

The problem was that the ubuntu image that i was using was armel
architecture, but the sage distribution made by Julien was compiled in
an armhf environment (a slightly different architecture). So there
were not compatible.


My solution was to get an armhf rootfs from the ubuntu core site
(http://cdimage.ubuntu.com/ubuntu-core/releases/12.04/release/), and
create a disk image with it to use.

The procedure to do so would be the following (on a linux box, maybe
you would need root privileges to mount the image):

1) create an empty file to build the image in. If you just wwant it to
use sage, 3GB should be fine. If you want to install a complete ubuntu
system, you will need more (i used 7GB):
dd -if=/dev/zero of=ubuntu.img -bs=1G count=3

2) format the file
mkfs.ext2 ubuntu.img

3)create a directory and mount the image there
mkdir mount
mount -o loop ubuntu.img mount

4) put the ubuntu core tarball in the directory and unpack it
cd mount
wget http://cdimage.ubuntu.com/ubuntu-core/releases/12.04/release/ubuntu-core-12.04.1-core-armhf.tar.gz
tar xzf ubuntu-core-12.04.1-core-armhf.tar.gz

5) Unpack the sage install inside the directory
6) Unmount the directory
cd ..
umount mount

And that's it, the file ubuntu.img is ready to be used as a chroot
environment with a very minimal ubuntu system (not even graphical
interface, just the very basic system tools), where sage should work
just fine.

I do have a 7gb image with a lightweight desktop, vnc server, and some
applications installed (although for example qt apps don't render
well, and firefox and chromium don't work either, so i have replaced
them by some alternatives).

I can try to upload it somewhere, if people is interested.

[Julien Puydt]

Le 07/12/2012 23:46, jaebond a �crit :

> I had been working on this exact same issue a few months ago. Would you
> mind explaining what you did to fix your issue?

To explain a bit more: there are several arm flavours, the armel and the
armhf. The -el stands for "endian-little" (a pun), the -hf for "hard
float". When I started the sage port to arm, I used the armel flavour
(which was the only available then). I switched to armhf as soon as
possible (as you can guess, using hard floats instead of software floats
tends to be faster).

Snark on #sagemath

[jaebond]

Thank you for that, that will help a lot.  I would be interested in the 7gb file.  How do you chroot into your images, do you use a script or an app?  I have been using an app, but it doesn't seem to like images that weren't made by the app.

[mmarco]

I have made a zip archive with both the image and the scripts that i
use. you can download it from:

http://riemann.unizar.es/algebra/Archive1.zip

It also includes a readme.txt file

> >http://cdimage.ubuntu.com/ubuntu-core/releases/12.04/release/ubuntu-c...

[mmarco]

jaebond, did you try the image i linked? did it work for you?

[tom d]

Hey, yo;

I'm experimenting with building Sage on the Raspberry Pi.  It apparently has an ARM6 processor, so I'm running from scratch.  I ran into problems building libm4rie as well (and also on building conversion.c); it would start running and then after about 20 minutes the device would freeze up and my ssh session (since I'm running headless!) would seize up.  Nothing I can find in the log files; seems like just a straight out crash.  I'm trying again from the beginning in case something went wrong earlier in the build process.

ATLAS took 807 minutes to compile on the first attempt.  I seem to recall hearing that ATLAS has some kind of processor database at some point?  I poked around looking for whether there's a good way to store the timings, etc, and chop down the ATLAS compile time but wasn't immediately able to turn any answers up.  Anyone have a good idea of where to look?

cheers!
-tom

[Julien Puydt]

Le 19/12/2012 21:52, tom d a �crit :

> I'm experimenting with building Sage on the Raspberry Pi. It apparently
> has an ARM6 processor, so I'm running from scratch. I ran into problems
> building libm4rie as well (and also on building conversion.c); it would
> start running and then after about 20 minutes the device would freeze up
> and my ssh session (since I'm running headless!) would seize up. Nothing
> I can find in the log files; seems like just a straight out crash. I'm
> trying again from the beginning in case something went wrong earlier in
> the build process.

Yes, that one file is a problem.

> ATLAS took 807 minutes to compile on the first attempt. I seem to recall
> hearing that ATLAS has some kind of processor database at some point? I
> poked around looking for whether there's a good way to store the
> timings, etc, and chop down the ATLAS compile time but wasn't
> immediately able to turn any answers up. Anyone have a good idea of
> where to look?

apt-get install libatlas-dev (or something like this),
then export SAGE_ATLAS_LIB=/usr
that way atlas won't get compiled in sage.

Snark on #sagemath

[jaebond]

mmarco,

Things got pretty crazy and I haven't had a chance to take a look yet, but I should be able to give it a try in the next day or two.  Thank you again for your help.

[mmarco]

I have seen that there is an android app that runs an ubuntu desktop
without the need of root access. From the web page, i see that it uses
something called fakechroot. I would like to investigate that
possibility. Maybe it could be the open door to some nice android app
that would install sage to be run in the terminal emulator.

Somebody knows anything about this fakechroot thing?


And on a different subject: what happened with the ARM machine that
William bought to serve as a buildbot?

[Volker Braun]

fakechroot is a LD_PRELOAD hack that overrides glibc calls and e.g. redirects file open() calls. Though its not perfect, for example no pseudo-ttys (I think this already rules out Sage). 

[mmarco]

Ah, that's a pity. It was too beautyful to be true: the possibility to
deliver a sage android app that does not require to hack your device.

[jaebond]

I didn't have any luck getting the scripts to work with a basic image I built.  Does the image have to be in /sdcard/ubuntu (I replaced /sdcard/ubuntu with /data/sdext2/ubuntu in the scripts, but I'm not sure if more has to be done)?  Do the scripts need something specific to the image you provided?

On Thursday, December 13, 2012 6:35:52 AM UTC-5, mmarco wrote:

[jaebond]

I think I got the image working with the linux app I've been using (Complete Linux Installer), so I'll give Sage a try tomorrow or Sunday.

[tom d]

Man, still no success in getting through the libm4rie build.  It ran for 38 hours before I had to get ready to head back to North America (which involved cutting power to the Pi).  It looks like the swap (on a connected usb drive) just got so jammed up after a couple hours that the work was happening at a truly glacial pace: cpu usage was basically zero, with 98+% of the cpu time spent 'waiting,' according to top.

So I'm wondering about options for making a more portable Sage, given that we're going to want to build on more kinds of devices in the future.  Things I've thought about:

a) Is it possible to cross-compile some packages, while leaving tuned pacages like ATLAS to compile on the actual device?  This would let us build _most_ of Sage, and then do the remainder on the slow device.

b) Would it be reasonable to build a 'lite' version of Sage, with some of the heavier packages left as optional?  This would probably involve setting up some extra flags when checking for dependencies in the code..........  maybe terrible for doc-testing, and probably a very large project overall.

c) ATLAS has these default settings which one can choose from for different processors with different instruction sets.  Is there a place I could harvest/submit the processor profile of the Raspberry Pi for inclusion in ATLAS?

I think this is pretty important stuff, as the Pi has sold piles and piles of units; if we can have a Sage binary, it becomes available to every 10-20 year old nerd with $35+ to burn on a new machine..........  I'm personally looking at deploying in rural schools in Western Kenya.

Best,
-tom

On Thursday, December 20, 2012 12:47:53 AM UTC+3, Snark wrote:

Le 19/12/2012 21:52, tom d a �crit :

[Julien Puydt]

Le 24/12/2012 11:28, tom d a �crit :

> Man, still no success in getting through the libm4rie build.

I think we should report it as a bug upstream.

Snark on #sagemath

[Martin Albrecht]

On Monday 24 Dec 2012, tom d wrote:
> Man, still no success in getting through the libm4rie build. It ran for 38
> hours before I had to get ready to head back to North America (which
> involved cutting power to the Pi). It looks like the swap (on a connected
> usb drive) just got so jammed up after a couple hours that the work was
> happening at a truly glacial pace: cpu usage was basically zero, with 98+%
> of the cpu time spent 'waiting,' according to top.

Hi, M4RIE developer here.

I am pretty sure you're stuck in conversion.c which is a pretty dumb file
actually which translates bitpacked representations to bitsliced, it's just
bit fiddling but unrolled which probably explains the huge demand for
compiling. How much RAM do you have? I could try to limit my RAM on my machine
and split up the file so GCC compiles it with that RAM limit. Alternatively,
try reducing the -O level.

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
_www: http://martinralbrecht.wordpress.com/
_jab: martinr...@jabber.ccc.de

[Julien Puydt]

Le 24/12/2012 12:53, Martin Albrecht a �crit :

> On Monday 24 Dec 2012, tom d wrote:
>> Man, still no success in getting through the libm4rie build. It ran for 38
>> hours before I had to get ready to head back to North America (which
>> involved cutting power to the Pi). It looks like the swap (on a connected
>> usb drive) just got so jammed up after a couple hours that the work was
>> happening at a truly glacial pace: cpu usage was basically zero, with 98+%
>> of the cpu time spent 'waiting,' according to top.
>
> Hi, M4RIE developer here.
>
> I am pretty sure you're stuck in conversion.c which is a pretty dumb file
> actually which translates bitpacked representations to bitsliced, it's just
> bit fiddling but unrolled which probably explains the huge demand for
> compiling. How much RAM do you have? I could try to limit my RAM on my machine
> and split up the file so GCC compiles it with that RAM limit. Alternatively,
> try reducing the -O level.

RAM: 512M
Swap: 1G

Snark on #sagemath

[tom d]

And I'm on a truly measly 256mb of RAM, but with 2gb swap.

Thanks for taking some time to consider this!

[Julien Puydt]

Le 24/12/2012 12:53, Martin Albrecht a �crit :

> I am pretty sure you're stuck in conversion.c which is a pretty dumb file
> actually which translates bitpacked representations to bitsliced, it's just
> bit fiddling but unrolled which probably explains the huge demand for
> compiling. How much RAM do you have? I could try to limit my RAM on my machine
> and split up the file so GCC compiles it with that RAM limit. Alternatively,
> try reducing the -O level.

It would be nice to split that file indeed, because it makes it quite
painful to compile sage on the box I use (and probably on others, though
I guess one feels it less on more powerful boxes): I have to basically
make the box only run the make command (shutdown X and a few other
things...)!

I will probably make a bdist of sage 5.5 available in a few days ; I'll
try to finish compiling libm4rie (well, conversion.c ...) then I'll need
the box for other things than compiling. Then atlas will come...

Cheers,

Snark on #sagemath