memory leak when building sage-6.6.rc0

[bogda...@gmail.com]

My computer runs out of memory when I attempt to build sage from source. I want to build sage-6.6.rc0.

Installation log (26MB)

I get to the stage when I see a bunch of statements "Cythonizing path/to/some/file.pyx". At this stage some python process consumes more than 1GB RAM and quickly grows in size.

Things slow down to a crawl when swap comes into play. I interrupted make by pressing ^C, then ran make again with the same results (see log).

OS: Linux acer 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt7-1 (2015-03-01) x86_64 GNU/Linux   (Debian Jessie)

PC: 2GB RAM + 4GB swap, CPU: Intel Celeron N2830

Steps to reproduce:

1. Download, unzip sage-6.6.rc0
   
2. cd sage-6.6.rc0
   
3. make
   

[Volker Braun]

Thats probably not enough for a parallel build. Try:

MAKE='make -j1' make

[Jeroen Demeyer]

On 2015-03-26 20:09, Volker Braun wrote:
> Thats probably not enough for a parallel build. Try:
>
> MAKE='make -j1' make

I don't think the original poster was using a parallel build (at least I
see no evidence of this).

[Jeroen Demeyer]

On 2015-03-26 19:23, bogda...@gmail.com wrote:
> PC: 2GB RAM + 4GB swap, CPU: Intel Celeron N2830

If you have *2GB* of RAM and the Python process needs *1GB*, then there
should be no problem. Just make sure you close everything else on that
machine while Sage is building and then things should work again.

[94n...@gmail.com]

I can only agree with the original posting: Sage-6.6 release on gentoo requires more than 1.5GB for compiling, on a 2GB computer about half a GB is needed for running gnome (especially now that polkit has an unfixed memory-leak in gentoo), so during single-threaded builds Swapfile will definitely be used. from what I see up to 5GB per thread is needed for building, inside some python scripts. would be nice if this got reverted in next release. if not, at least warn about it, so people with only 16GB can avoid using more than 4 threads...

as for 1GB being used: there is a script setup.py which ps shows 3 times with the parameter "build", even though I told make to use at most 2 threads and only when load is <1. "ps aux" shows "0.1 400736 2272", "27.8 1675752 568952" and "40.5 1638940 829452" in their "%MEM VSZ RSS" columns, their size shows as 275,612 1,531,976 and 1,495,164 respectively. is it a bug in make that swap-file management doesn't count as load? ps doesn't show any process called "make" in that phase though...

[Andrey Novoseltsev]

Cython was using too much memory, should be fixed in recent beta.