problem with paralell build

[Ralf Hemmecke]

I seem to have a problem with building sage 9.4 via.

MAKE='make -j12' make -j12

It stops during the build of dochtml. Obviously because of too many open
files. I'm now trying without "-j12" and it seems to be building nicely.

Anyhow... any suggestion what to change to prevent my problem and still
be able to build in parallel?

Thanks
Ralf

[dochtml] File "<frozen importlib._bootstrap_external>", line 987, in
get_code
[dochtml] OSError: [Errno 24] Too many open files:
'/zvol/sage/sage-9.4/x86_64/local/lib/python3.9/site-packages/sage/doctest/__init__.py'
[dochtml] File "<frozen importlib._bootstrap_external>", line 1044, in
get_data
[dochtml] OSError: [Errno 24] Too many open files:
'/zvol/sage/sage-9.4/x86_64/local/lib/python3.9/site-packages/sage/rings/finite_rings/all.py'
[dochtml] make[5]: *** [Makefile:20: doc-inventory--reference-algebras]
Error 1
[dochtml] make[5]: *** Waiting for unfinished jobs....
[dochtml] make[5]: *** [Makefile:20:
doc-inventory--reference-polynomial_rings] Error 1

>ulimit -a
real-time non-blocking time (microseconds, -R) unlimited
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 31256
max locked memory (kbytes, -l) 1009691
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 31256
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

[Dima Pasechnik]

On Wed, Sep 29, 2021 at 11:35 AM Ralf Hemmecke <hemm...@gmail.com> wrote:

I seem to have a problem with building sage 9.4 via.

  MAKE='make -j12' make -j12

It stops during the build of dochtml. Obviously because of too many open
files. I'm now trying without "-j12" and it seems to be building nicely.

Anyhow... any suggestion what to change to prevent my problem and still
be able to build in parallel?

try increasing this limit (1024 is very small...)

 

ulimit -n 4096

 

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/14616aec-24cd-3411-a18c-c30a2c8fce1f%40gmail.com.

[John Cremona]

On Wed, 29 Sept 2021 at 11:54, Dima Pasechnik <dim...@gmail.com> wrote:
>
>
>
> On Wed, Sep 29, 2021 at 11:35 AM Ralf Hemmecke <hemm...@gmail.com> wrote:
>>
>> I seem to have a problem with building sage 9.4 via.
>>
>> MAKE='make -j12' make -j12
>>
>> It stops during the build of dochtml. Obviously because of too many open
>> files. I'm now trying without "-j12" and it seems to be building nicely.
>>
>> Anyhow... any suggestion what to change to prevent my problem and still
>> be able to build in parallel?
>>
> try increasing this limit (1024 is very small...)
>
> ulimit -n 4096

Mine is also 1024, so perhaps it is a common default (I have ubuntu
20.04.3 LTS, and the only ulimit thing I set to a non-default value is
'ulimit -c 0'). Perhaps the build instructions could mention this as
something to change for parallel builds.

John

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq0ZWCUw9CCWBz1J1m4Ovze%3DJPSKZe5%3DfhTsX_YXeyivfw%40mail.gmail.com.

[Eric Gourgoulhon]

Le mercredi 29 septembre 2021 à 13:13:34 UTC+2, john.c...@gmail.com a écrit :

Mine is also 1024, so perhaps it is a common default (I have ubuntu
20.04.3 LTS, and the only ulimit thing I set to a non-default value is
'ulimit -c 0'). Perhaps the build instructions could mention this as
something to change for parallel builds.

I confirm: on my Ubuntu 20.04.3 machine, ulimit -n is 1024 as well. However, I could build Sage 9.4 with -j16 without any trouble (CPU: Intel Xeon E5-2623 (8 cores); RAM: 16 GB).

Eric.