While working on #9343, I discovered that some of the doctests (in
particular, related to elliptic curves) use lots of memory (more than 1GB).
I know that Sage has a "# long time" option for tests which take a long
time. Maybe there should be an option "# huge memory" or something to
indicate that.
Is there any policy on how much memory a reasonable doctest should use?
Jeroen.
What particular test is using a lot of RAM? On what operating system?
32-bit or 64-bit?
Although not recently, I've built Sage and it passed all doctests on a
machine with 1.5 GB RAM. Only yesterday I built is on a machine with
only 2 GB RAM. These were all 32-bit Solaris builds on SPARC.
There is some memory leaks on Sage. When I linked Sage on Solaris
using a debugging memory tool, I discovered there are memory leaks
even before we get the
sage:
prompt. Some of the functions called had 'pari in the name, so
probably related to Pari, though I'm not saying Pari is at fault.
Dave
$ ulimit -v 1900000 # Use at most 1,945,600,000 bytes of virtual memory
$ sage -t sage/schemes/elliptic_curves/heegner.py
File "[...]/sage/schemes/elliptic_curves/heegner.py", line 4646:
sage: H.optimal_embeddings(-7, 1, R)
Exception raised:
Traceback (most recent call last):
[...]
MemoryError: Unable to allocate 1024000000 bytes memory for PARI.
> On what operating system? 32-bit or 64-bit?
$ uname -a
Linux arcanis 2.6.32-gentoo-r7 #5 SMP Thu Jun 10 23:07:26 CEST 2010
x86_64 Intel(R) Core(TM)2 Duo CPU T5870 @ 2.00GHz GenuineIntel GNU/Linux
I set that ulimit and it passed the test ok on Solaris 10 (SPARC). The memory
kept climbing, but *only* reached 695 MB. Still a lot I would agree.
I did a test the other day on a 64-bit build of Sage on Solaris, by using a
different malloc - one designed for testing for memory leaks. There is certainly
something that looks to be Pari related there - see below. Note how one of the
calls which is generating a memory leak has 'pari' in the name?
kirkby@t2:[~/sage-4.5-hacked-for-64-bit-solaris] $ mdb core
Loading modules: [ libumem.so.1 libc.so.1 libuutil.so.1 ld.so.1 ]
> > ::findleaks -dv
findleaks: maximum buffers => 38054
findleaks: actual buffers => 37054
findleaks:
findleaks: potential pointers => 10532388
findleaks: dismissals => 3532123 (33.5%)
findleaks: misses => 4985446 (47.3%)
findleaks: dups => 1977768 (18.7%)
findleaks: follows => 37051 ( 0.3%)
findleaks:
findleaks: elapsed wall time => 8 seconds
findleaks:
BYTES LEAKED VMEM_SEG CALLER
65536 1 ffffffff73c60000 MMAP
------------------------------------------------------------------------
Total 1 oversized leak, 65536 bytes
CACHE LEAKED BUFCTL CALLER
000000010015e028 1 00000001026e2020
gen.so`__pyx_f_4sage_4libs_4pari_3gen__new_gen+0x68
0000000100168028 1 00000001039e6e10
integer.so`__pyx_f_4sage_5rings_7integer_fast_tp_new+0xb0
----------------------------------------------------------------------
Total 2 buffers, 24 bytes
mmap(2) leak: [ffffffff73c60000, ffffffff73c70000), 65536 bytes
umem_alloc_8 leak: 1 buffer, 8 bytes
ADDR BUFADDR TIMESTAMP THREAD
CACHE LASTLOG CONTENTS
1026e2020 1014f7fe0 6880239fd06b2d 1
10015e028 0 0
libumem.so.1`umem_cache_alloc+0x148
libumem.so.1`umem_alloc+0x5c
libumem.so.1`malloc+0x40
gen.so`__pyx_f_4sage_4libs_4pari_3gen__new_gen+0x68
gen.so`__pyx_f_4sage_4libs_4pari_3gen_12PariInstance_new_gen+0x28
gen.so`__pyx_pf_4sage_4libs_4pari_3gen_12PariInstance___init__+0x8d0
libpython2.6.so.1.0`type_call+0x88
libpython2.6.so.1.0`PyObject_Call+0x60
gen.so`initgen+0x194c
libpython2.6.so.1.0`_PyImport_LoadDynamicModule+0x94
libpython2.6.so.1.0`import_submodule+0xc8
libpython2.6.so.1.0`load_next+0x98
libpython2.6.so.1.0`import_module_level+0x1f8
libpython2.6.so.1.0`PyImport_ImportModuleLevel+0x28
libpython2.6.so.1.0`builtin___import__+0x8c
umem_alloc_16 leak: 1 buffer, 16 bytes
ADDR BUFADDR TIMESTAMP THREAD
CACHE LASTLOG CONTENTS
1039e6e10 1039df140 6880256324a377 1
100168028 0 0
libumem.so.1`umem_cache_alloc+0x21c
libumem.so.1`umem_alloc+0x5c
libumem.so.1`malloc+0x40
integer.so`__pyx_f_4sage_5rings_7integer_fast_tp_new+0xb0
integer.so`__pyx_f_4sage_5rings_7integer_8int_to_Z__call_+0x68
coerce.so`__pyx_f_4sage_9structure_6coerce_24CoercionModel_cache_maps_canonical_coercion+0x2fc
element.so`__pyx_f_4sage_9structure_7element_7Element__richcmp+0x248
integer.so`__pyx_pf_4sage_5rings_7integer_7Integer___richcmp__+0x20
libpython2.6.so.1.0`try_rich_compare+0x5c
libpython2.6.so.1.0`PyObject_RichCompare+0x64
libpython2.6.so.1.0`PyEval_EvalFrameEx+0x2838
libpython2.6.so.1.0`PyEval_EvalCodeEx+0x988
libpython2.6.so.1.0`function_call+0x158
libpython2.6.so.1.0`PyObject_Call+0x60
libpython2.6.so.1.0`instancemethod_call+0x150
> >
The pari library does its own memory management. Not for the faint-hearted!
John
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>
> The pari library does its own memory management. Not for the faint-hearted!
Actually, it is a very simple-minded memory management. But a
malloc-debugger is pointless to debug PARI memory leaks.
Jeroen.