How much memory can Scilab use?

[arvro395]

Hi!
I am working with very huge matrices and I constantly run out of
memory.
Scilab says:
"stack size exceeded! (Use stacksize function to increase it)"

When I try to increase the stack size, for example using
--> stacksize(150e6)
Scilab says:
"!--error 112"
"Cannot allocate more memory"

But I have plenty of RAM and lots of swap on my machine. Is there a
hard limit somewhare? I noticed that scilab commes with its own malloc
function in malloc.c Why is that? Can't this function handle large
memory allocations?

I am using the latest Scilab source checked out from the SVN archive on
a Linux AMD 64 computer.

Cheers
Arvid

[Francois Vogel]

Well, that's strange. I just tried this:

-->stacksize(150e6)

-->stacksize(1500e6)
stacksize requested size is too big (max < 268435455)

!--error 112
Cannot allocate more memory

-->stacksize(15000e6)
!--error 1504
Out of bounds value for stacksize argument

Indeed, stacksize code in Fortran is in matsys.f.

A first check on the input argument is performed in matsys.f: the upper
limit of the argument given to stacksize is hardcoded to be 2.0d0**31. If it
exceeds it, then the "Out of bounds" message is issued.

Then scimem (routines/default.scimem.c) is called, and another check is
performed:

double dsize = ((double) sizeof(double)) * (*n + 1);
unsigned long ulsize = ((unsigned long)sizeof(double)) * (*n + 1);
if ( dsize != (double) ulsize)
{
unsigned long pos = MAXLONG/sizeof(double);

sciprint("stacksize requested size is too big (max < %lu)\r\n",pos);
}
else
...

IIUC, it checks whether the memory (in doubles) to allocate stored as a
double can be stored (casted) in an unsigned long with the same
representation. If it's not the case, error 112 is thrown with the maximum
size computed from MAXLONG (2147483647).

Now, why stacksize(150e6) works for me and not for you is not crystal clear
but it seems that the above test triggers the error for you while it does
not for me (P4 CPU, 512Mb RAM, swap memory set to 1536Mo max - Windows
prevents me from allowing more for some unclear reason). The Scilab output
above uses the most recent SVN version. Funnily, if I use Scilab4.0
(prebuilt binary), I get error 112 on the same machine with
stacksize(150e6).

Francois

"arvro395" <ar...@softube.se> a écrit dans le message de news:
1147888930....@j33g2000cwa.googlegroups.com...

[arvro395]

Hi Francois,
And thanks for you answer!

Yes, this is strange behaviour. And pretty bad too. Handeling of large
data sets is one of the more important tasks of software like Scilab.

I tried this on my older windows machine too, using Scilab 3.0. This
computer has less RAM than my linux one, but still I managed to
increase the stacksize to at least 100e6 which never worked on my linux
computer. So this looks like mainly a linux issue.

I also noticed that error 112 seems to be triggered when scilab's
malloc (in malloc.c) returns 0. I looked at the code in malloc.c, but I
couldn't really tell what it was doing.

Does the windows version of scilab use this code? Or does it use
win_mem_alloc.c instead? Could this explain the different behaviour on
the two platforms?

/Arvid

[Derek]

Hello,

I have just installed Scilab 4.0 on a Dell WS 620, Pentium III Xeon,
800MHz, 640MB ram.

Can anybody explain the difference between stacksize and gstacksize?

-->gstacksize(29e6)

-->gstacksize()
ans =

29000001. 1192.

-->gstacksize(30e6)

!--error 112
Cannot allocate more memory

-->stacksize(150e6)

-->stacksize()
ans =

1.500D+08 15004.

-->stacksize(160e6)
!--error 112
Cannot allocate more memory.

Is there a system function that tells you how much memory Scilab can
see or use. I tried looking in the usual places without success.

Regards,

Derek O'Connor

[Derek]

I should have added Windows 2000 SP4.

Derek O'Connor

[Francois Vogel]

> Can anybody explain the difference between stacksize and gstacksize?

Well, did you read the help for these keywords?

help stacksize tells it:
Scilab stores "usual" variables in a stack stk (for global variables see
gstacksize ).
stacksize(n) allows the user to increase or decrease the size of this stack

help gstacksize confirms:
Scilab stores gobal variables in a stack
gstacksize(n) allows the user to increase or decrease the size of this
stack.

> Is there a system function that tells you how much memory Scilab can
> see or use. I tried looking in the usual places without success.

See help getmemory perhaps. But relating the figures this command provides
to the stacksize parameter to give is not obvious.

Francois

[Enrico Segre]

Though dated and referring only to 32bit OSs, you might like to have a
look at Lydia's wisdom:

http://groups.google.com/group/comp.soft-sys.math.scilab/browse_thread/thread/45caad00e40c39d4/0db8bd2aef67c979?q=malloc&rnum=8#0db8bd2aef67c979
http://groups.google.com/group/comp.soft-sys.math.scilab/browse_thread/thread/3a17c968a759cd4/c51dc069183f59eb?q=malloc&rnum=6#c51dc069183f59eb

I didn't check into the code, but I have no reason to doubt that
"scilab's own malloc function in malloc.c" just translates into a
wrapper to a normal malloc.

In my case however nothing to complain:

-->stacksize(165e6)

!--error 112
Cannot allocate more memory

-->stacksize(160e6)

-->stacksize()
ans =

1.600D+08 15911.

-->a=rand(10e3,10e3);

-->stacksize() //(ages later, I'm 700Mb out in swap)
ans =

1.0D+08 *

1.6 1.0001591

-->[free, total]=getmemory()
total =

1035728.
free =

13532.

$ free
total used free shared buffers
cached
Mem: 1035728 1021692 14036 0 5000
75800
-/+ buffers/cache: 940892 94836
Swap: 1188768 708040 480728

[arvro395]

Enrico Segre skrev:

> I didn't check into the code, but I have no reason to doubt that
> "scilab's own malloc function in malloc.c" just translates into a
> wrapper to a normal malloc.
>

I did check the code, and malloc.c is NOT a wrapper for the normal
malloc.
I looked at the code, but it is quite hard to grasp what it does.

On a 64-bit system, shouldn't I be able to allocate huge amounts of
memory? I can't understand why this is a problem in the first place,
and I am willing to help to solve it if I can. But I need help here...

[arvro395]

arvro395 skrev:

> Enrico Segre skrev:
>
> > I didn't check into the code, but I have no reason to doubt that
> > "scilab's own malloc function in malloc.c" just translates into a
> > wrapper to a normal malloc.
> >
>
> I did check the code, and malloc.c is NOT a wrapper for the normal
> malloc.
> I looked at the code, but it is quite hard to grasp what it does.

Hm... well it does call sbrk(), so I guess it is just a complicated
wrapper around the C library functions.
It seems like sbrk() returns -1, which we can't blaim scilab for I
guess.

/Arvid