Re: How to kill slave kernel securely?
Oliver Ruebenkoenig
> Oliver Ruebenkoenig <rueb...@wolfram.com> wrote:
>> On Thu, 10 Mar 2011, Alexey Popkov wrote:
>>> 2) My bitter experience with MemoryConstrained clearly shows that it is
> not
>>> a reliable function. It practical cases when I heavily use such
> functions as
>>> NDSolve, NIntegrate etc. it often allows MathKernel to take for example
>>> 500-900 MB more that I have set. And system goes into swapping...
>>
>> You could forward that to the tech support for us to look at.
>>
>> Oliver
>
> I have found today a simple example when MemoryConstrained fails:
>
> In[1]:= MaxMemoryUsed[]
> MemoryConstrained[
> Quiet@Module[{f, n = 0},
> f[x_] := f[x + 1] + Plus @@ Table[Random[], {10^7}];
> Block[{$RecursionLimit = 3000}, f[0]]];, 90000000];
> MaxMemoryUsed[]
>
> Out[1]= 10757752
>
> Out[3]= 291258232
>
> You can see that MathKernel takes more than 260 Mb additional memory in peak
> although only less than 90 Mb additional memory was allowed by
> MemoryConstrained. This behavior is machine-dependent. Try to play with the
> memory constrain if you cannot reproduce.
Actually I do not see that:
Mathematica 8.0 for Linux x86 (64-bit)
Copyright 1988-2011 Wolfram Research, Inc.
In[1]:= MaxMemoryUsed[]
Out[1]= 13993528
MemoryConstrained[
Quiet@Module[{f, n = 0},
f[x_] := f[x + 1] + Plus @@ Table[Random[], {10^7}];
Block[{$RecursionLimit = 3000},
f[0]]];, 90000000]; MaxMemoryUsed[]
Out[2]= 14521744
For version 8 the memory allocator has been rewritten and improved.
(What a small sentence for such a huge endeavor and such a fine execution)
Here is another example with a lower bound
MaxMemoryUsed[]/1024.^2
MemoryConstrained[
Quiet@Module[{f, n = 0},
f[x_] := f[x + 1] + Plus @@ Table[Random[], {10^8}];
Block[{$RecursionLimit = 20000}, f[0]]];, 20000000];
MaxMemoryUsed[]/1024.^2
>
> As I said before, my experience shows that in practice the overshoot can be
> 900 Mb and more! This often happens in complicated computations with heavy
> usage such functions as NDSolve and NIntegrate inside loops.
>
> Alexey
>
>
There may be various issues here, for example, NDSolve calls external
libraries. And M- does not and can not have control over those memory
processes. Similar issues may arise with e.g. MKL.
Could you provide an example, where the overshot is 900 Mb - assuming you
see this in V8.
Oliver
Alexey Popkov
that in v.8 the memory allocator has been improved, but at this moment I
have no access to v.8. May be later... At the same time it is possible that
you just have tried not enough with other (greater) values for the memory
constrain or you should increase the value for $RecursionLimit...
Alexey
Alexey Popkov
> On Thu, 10 Mar 2011, Alexey Popkov wrote:
> > 2) My bitter experience with MemoryConstrained clearly shows that it is
not
> > a reliable function. It practical cases when I heavily use such
functions as
> > NDSolve, NIntegrate etc. it often allows MathKernel to take for example
> > 500-900 MB more that I have set. And system goes into swapping...
>
> You could forward that to the tech support for us to look at.
>
> Oliver
I have found today a simple example when MemoryConstrained fails:
In[1]:= MaxMemoryUsed[]
MemoryConstrained[
Quiet@Module[{f, n = 0},
f[x_] := f[x + 1] + Plus @@ Table[Random[], {10^7}];
Block[{$RecursionLimit = 3000}, f[0]]];, 90000000];
MaxMemoryUsed[]
Out[1]= 10757752
Out[3]= 291258232
You can see that MathKernel takes more than 260 Mb additional memory in peak
although only less than 90 Mb additional memory was allowed by
MemoryConstrained. This behavior is machine-dependent. Try to play with the
memory constrain if you cannot reproduce.
As I said before, my experience shows that in practice the overshoot can be
Daniel Lichtblau
----- Original Message -----
> From: "Alexey Popkov" <leh...@gmail.com>
> To: math...@smc.vnet.net
> Sent: Saturday, March 19, 2011 5:22:07 AM
> Subject: Re: How to kill slave kernel securely?
That MemoryConstrained returns $Aborted. What exactly do you mean by the claim that it fails? That it does not also account for subroutine stack space/ I believe that was already explained.
Daniel Lichtblau
Wolfram Research