libgap is not using the available memory

[Simon Brandhorst]

sage: V = libgap.GF(13)^500
sage: S = V.Subspaces(50)
sage: S.List()
gap: cannot extend the workspace any more!

As expected the computation runs out of memory. However, there is still a lot of unused memory.

Before:

                      total        used        free      shared  buff/cache   available
Mem:            15G        1.8G         11G        290M        2.3G         13G
Swap:           39G          0B         39G

Right before termination

Mem:            15G        5.7G        7.6G        287M        2.3G        9.2G
Swap:           39G          0B         39G
 
After
                     total        used        free      shared  buff/cache   available
Mem:            15G        1.6G         11G        287M        2.3G         13G
Swap:           39G          0B         39G

In
https://groups.google.com/forum/#!topic/sage-support/m6R8_NKepyQ
it is suggested to increase the swap space.
However with 40 GB of swap I can only use about 4GB.
On a different machine with 64 GB of swap I can use about 8 GB.
However assigning more swap is impossible as I start to run out of hard disk space.
Can you reproduce this behavior? Suggestions?

[Dima Pasechnik]

The arguments with which libgap is started are set up in
src/sage/libs/gap/util.pyx
in the function initialize()
- setting various members of argv. In particular look for what `-o` is set to.
By digging through the code, it appears that it's set by
set_gap_memory_pool_size()
from interfaces/gap.py

So if you call set_gap_memory_pool_size with a big value, before
starting libgap, you might get
more (I didn't try)

HTH,
Dima

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[Simon Brandhorst]

Dear Dima,

thank you so much. It works :-). I would suggest to hint at this solution in the error message.
As soon as gap does not take sage with it anymore when running out of memory.
Best
Simon

[Volker Braun]

Is your computer running more than one gap process at a time? There are more states for swap space than free and used, it can also be exclusively reseved (but not necessarily used). This is what gap is doing and probably not reflected in your output.

[Simon Brandhorst]

There should be only one gap process if I run these 3 lines right after startup.
You can find attached the output of top. Maybe there is more information for you.