How to kill slave kernel securely?
Alexey
LinkClose[link] "does not necessarily terminate the program at the
other end of the connection" as it is said in the Documentation. Is
there a way to kill the process of the slave kernel securely?
Oliver Ruebenkoenig
Alexey,
you could use Quit.
Oliver
Oliver Ruebenkoenig
> Oliver,
>
> Is this a good way to do when the slave kernel freezes because of swapping?
>
> Alexey
>
>
Alexey,
You could try MemoryConstrained and use Quit for the failexpr.
Oliver
John Fultz
> Good day,
>
> LinkClose[link] "does not necessarily terminate the program at the
> other end of the connection" as it is said in the Documentation. Is
> there a way to kill the process of the slave kernel securely?
If the kernel's $ParentLink is the link that got closed, then the kernel should
kill itself after, at most, a few seconds. This is not true for MathLink
programs generally, but it is true for Mathematica kernels. Are you coming
across a case where that isn't happening?
Sincerely,
John Fultz
jfu...@wolfram.com
User Interface Group
Wolfram Research, Inc.
Alexey Popkov
function to protect from swapping securely...
----- Original Message -----
From: "Oliver Ruebenkoenig" <rueb...@wolfram.com>
To: "Alexey Popkov" <leh...@gmail.com>
Cc: "comp.soft-sys.math.mathematica" <math...@smc.vnet.net>
Sent: Wednesday, March 09, 2011 9:46 PM
Subject: Re: How to kill slave kernel securely?
> On Wed, 9 Mar 2011, Alexey Popkov wrote:
>
> > Oliver,
> >
> > Is this a good way to do when the slave kernel freezes because of
swapping?
> >
> > Alexey
> >
> >
>
> Alexey,
>
> You could try MemoryConstrained and use Quit for the failexpr.
>
> Oliver
>
> >
> > ----- Original Message -----
> > From: "Oliver Ruebenkoenig" <rueb...@wolfram.com>
> > Newsgroups: wri.mathgroup
> > To: "Alexey" <leh...@gmail.com>
> > Cc: <math...@smc.vnet.net>
> > Sent: Wednesday, March 09, 2011 5:11 PM
> > Subject: Re: How to kill slave kernel securely?
> >
> >
> >> On Wed, 9 Mar 2011, Alexey wrote:
> >>
> >>> Good day,
> >>>
> >>> LinkClose[link] "does not necessarily terminate the program at the
> >>> other end of the connection" as it is said in the Documentation. Is
> >>> there a way to kill the process of the slave kernel securely?
> >>>
> >>>
> >>
Armand Tamzarian
On occasion I've noticed errors that stemmed from slave kernels not
closing properly and ended up killing the processes using the Activity
Monitor app in Mac. You can do this in the terminal as well.
Mike
Alexey Popkov
kernel I get an "emancipated" slave kernel that even can be used
interactively.
(sorry for typos and mistakes in the original response)
----- Original Message -----
From: "Alexey Popkov" <leh...@gmail.com>
To: <jfu...@wolfram.com>; <math...@smc.vnet.net>
Sent: Thursday, March 10, 2011 1:12 AM
Subject: Re: How to kill slave kernel securely?
> Very often if I Abort (by Alt+",") a program heavily uses slave kernel I
get
> an "emancipated" slave kernel that even can be uses interactively.
>
>
> ----- Original Message -----
> From: "John Fultz" <jfu...@wolfram.com>
> To: <leh...@gmail.com>; <math...@smc.vnet.net>
> Sent: Thursday, March 10, 2011 12:06 AM
> Subject: Re: How to kill slave kernel securely?
>
>
> On Wed, 9 Mar 2011 07:00:02 -0500 (EST), Alexey wrote:
> > Good day,
> >
> > LinkClose[link] "does not necessarily terminate the program at the
> > other end of the connection" as it is said in the Documentation. Is
> > there a way to kill the process of the slave kernel securely?
>
John Fultz
computing functionality or whether you're talking about using some sort of
custom front end to a program (a kernel? your own program?) which is in turn
hooked to a kernel. You've posted on this forum before about writing custom
front ends before, which makes any attempt at guessing your circumstances on my
part difficult.
The statement that I made is precisely true. If a kernel has its $ParentLink
pointing to a link which is closed from the other side, that kernel will kill
itself, even if it's in the middle of a computation. There could, of course, be
a bug in the mechanism, but I have to say that I rely on this fact *very*
frequently (often multiple times daily because of the nature of the work I do),
and I almost never see it fail.
However, it's also possible that you're describing a situation which does not
precisely mirror the circumstances I'm laying out (e.g., the link is not closed,
or the kernel's $ParentLink is not that link at the time the link is closed). I
simply can't tell, as your description is not precise enough.
Also, I should have mentioned that you can send the MLTerminateMessage message
using MLPutMessage(). Any program which respects this message (both the kernel
and the FE do) will quit as quickly as possible. In top-level Mathematica code,
this can be accomplished by executing LinkInterrupt[1].
Sincerely,
John Fultz
jfu...@wolfram.com
User Interface Group
Wolfram Research, Inc.
Alexey Popkov
implement another one with reliable protection from swapping and support for
older versions of kernels. I use standard MathLink methods from the
Documentation for this at the moment.
I wish to terminate the slave kernel(s) securely when 'Alt+.' is pressed in
the FrontEnd. The $ParentLink (the master kernel) in this case will not be
closed.
Todd Gayley
>Good day,
>
>LinkClose[link] "does not necessarily terminate the program at the
>other end of the connection" as it is said in the Documentation. Is
>there a way to kill the process of the slave kernel securely?
Alexey,
As you note, the kernel doesn't always die immediately when the link
to it closes. That requires some cooperation from the kernel, and if
the kernel is deep in some computational library function, it might
not be in a position to respond. A much stronger step than just
closing the link is to send the kernel an MLTerminateMessage. From
top-level code (which is what I know you are using):
LinkInterrupt[link, 1] (* An undocumented form that lets you pick
the message type *)
In C:
MLPutMessage(link, MLTerminateMessage);
In Java using J/Link:
link.terminateKernel();
In .NET using .NET/Link:
link.TerminateKernel();
Todd Gayley
Wolfram Research
Oliver Ruebenkoenig
> Instead of MemoryConstrained I would prefer to have 'FreeMemoryConstrained'
> function to protect from swapping securely...
>
What is holding you from setting that MemoryConstraint to the amount of
physical memory you have?
>
> ----- Original Message -----
> From: "Oliver Ruebenkoenig" <rueb...@wolfram.com>
> To: "Alexey Popkov" <leh...@gmail.com>
> Cc: "comp.soft-sys.math.mathematica" <math...@smc.vnet.net>
> Sent: Wednesday, March 09, 2011 9:46 PM
> Subject: Re: How to kill slave kernel securely?
>
>
>> On Wed, 9 Mar 2011, Alexey Popkov wrote:
>>
>>> Oliver,
>>>
>>> Is this a good way to do when the slave kernel freezes because of
> swapping?
>>>
>>> Alexey
>>>
>>>
>>
>> Alexey,
>>
>> You could try MemoryConstrained and use Quit for the failexpr.
>>
>> Oliver
>>
>>>
>>> ----- Original Message -----
>>> From: "Oliver Ruebenkoenig" <rueb...@wolfram.com>
>>> Newsgroups: wri.mathgroup
>>> To: "Alexey" <leh...@gmail.com>
>>> Cc: <math...@smc.vnet.net>
>>> Sent: Wednesday, March 09, 2011 5:11 PM
>>> Subject: Re: How to kill slave kernel securely?
>>>
>>>
>>>> On Wed, 9 Mar 2011, Alexey wrote:
>>>>
>>>>> Good day,
>>>>>
>>>>> LinkClose[link] "does not necessarily terminate the program at the
>>>>> other end of the connection" as it is said in the Documentation. Is
>>>>> there a way to kill the process of the slave kernel securely?
>>>>>
>>>>>
>>>>
>>>> Alexey,
>>>>
Alexey Popkov
Is this a good way to do when the slave kernel freezes because of swapping?
Alexey
----- Original Message -----
Alexey Popkov
> On Thu, 10 Mar 2011, Alexey Popkov wrote:
>
> > Instead of MemoryConstrained I would prefer to have
'FreeMemoryConstrained'
> > function to protect from swapping securely...
> >
>
> What is holding you from setting that MemoryConstraint to the amount of
> physical memory you have?
There are two problems:
1) I cannot know how much memory I should allow MathKernel to allocate
because other applications can allocate memory too. If I simply set
MemoryConstrained to the amount of physical memory or even to the half of
physical memory the system still can go into swapping. From the other side I
do wish to allow MathKernel to use almost all available free physical memory
with except for about 200 MB which must be always available for other
applications.
2) My bitter experience with MemoryConstrained clearly shows that it is not
a reliable function. It practical cases when I heavily use such functions as
NDSolve, NIntegrate etc. it often allows MathKernel to take for example
500-900 MB more that I have set. And system goes into swapping...
Oliver Ruebenkoenig
>
>
> From: "Oliver Ruebenkoenig" <rueb...@wolfram.com>
> To: "Alexey Popkov" <leh...@gmail.com>
> Cc: "comp.soft-sys.math.mathematica" <math...@smc.vnet.net>
> Sent: Thursday, March 10, 2011 12:39 PM
> Subject: Re: How to kill slave kernel securely?
>
>
>> On Thu, 10 Mar 2011, Alexey Popkov wrote:
>>
>>> Instead of MemoryConstrained I would prefer to have
> 'FreeMemoryConstrained'
>>> function to protect from swapping securely...
>>>
>>
>> What is holding you from setting that MemoryConstraint to the amount of
>> physical memory you have?
>
> There are two problems:
>
> 1) I cannot know how much memory I should allow MathKernel to allocate
> because other applications can allocate memory too. If I simply set
> MemoryConstrained to the amount of physical memory or even to the half of
> physical memory the system still can go into swapping. From the other side I
> do wish to allow MathKernel to use almost all available free physical memory
> with except for about 200 MB which must be always available for other
> applications.
I am not sure I understand that. How would you that know with
FreeMemoryConstrained.
If you do not want your system to go into swap, have you tried switching
it off in the first place?
Assuming you are on Linux you could use ulimit to set memory constraints.
This then would allow other processes to access the rest of the memory. I
have no idea how and if this works, say, on windows.
>
> 2) My bitter experience with MemoryConstrained clearly shows that it is not
> a reliable function. It practical cases when I heavily use such functions as
> NDSolve, NIntegrate etc. it often allows MathKernel to take for example
> 500-900 MB more that I have set. And system goes into swapping...
>
You could forward that to the tech support for us to look at.
Oliver
John Fultz
memory. Whether it's backed by RAM, disk, or some other storage medium is a
detail that the operating system manages, not applications (with the exception
of mechanisms like memory-mapped files). And if an application did have the
ability to lock its memory into RAM, it would be quite unfriendly indeed to
other applications on the system.
Would you really want an app that insisted on keeping 2 gigabytes of RAM in play
for itself (or ten applications that could keep 200 megabytes each), even if the
application didn't happen to be doing any computation right now and other apps
were totally starved for RAM? This could lead to a total failure of the
operating system itself, which is much worse than swapping.
Modern operating systems simply cannot allow apps to behave in that fashion. If
they did, then instead of swap hell, you would end up with routine failures of
the entire operating system itself.
Sincerely,
John Fultz
jfu...@wolfram.com
User Interface Group
Wolfram Research, Inc.
On Thu, 10 Mar 2011 06:12:04 -0500 (EST), Alexey Popkov wrote:
> Instead of MemoryConstrained I would prefer to have
> 'FreeMemoryConstrained'
> function to protect from swapping securely...
>
>
> ----- Original Message -----
> From: "Oliver Ruebenkoenig" <rueb...@wolfram.com>
> To: "Alexey Popkov" <leh...@gmail.com>
> Cc: "comp.soft-sys.math.mathematica" <math...@smc.vnet.net>
> Sent: Wednesday, March 09, 2011 9:46 PM
> Subject: Re: How to kill slave kernel securely?
>
>
>> On Wed, 9 Mar 2011, Alexey Popkov wrote:
>>
>>> Oliver,
>>>
>>> Is this a good way to do when the slave kernel freezes because of
> swapping?
>
>>> Alexey
>>>
>>>
>> Alexey,
>>
>> You could try MemoryConstrained and use Quit for the failexpr.
>>
>> Oliver
>>
>>>
>>> ----- Original Message -----
>>> From: "Oliver Ruebenkoenig" <rueb...@wolfram.com>
>>> Newsgroups: wri.mathgroup
>>> To: "Alexey" <leh...@gmail.com>
>>> Cc: <math...@smc.vnet.net>
>>> Sent: Wednesday, March 09, 2011 5:11 PM
>>> Subject: Re: How to kill slave kernel securely?
>>>
>>>
Alexey Popkov
> > because other applications can allocate memory too. If I simply set
> > MemoryConstrained to the amount of physical memory or even to the half
of
> > physical memory the system still can go into swapping. From the other
side I
> > do wish to allow MathKernel to use almost all available free physical
memory
> > with except for about 200 MB which must be always available for other
> > applications.
>
> I am not sure I understand that. How would you that know with
> FreeMemoryConstrained.
FreeMemoryConstrained will check the current amount of free physical memory
for example 100 times per second. It is not very expensive and should not
take considerable processor time if it is implemented on the level of kernel
functions. For example under Windows the GlobalMemoryStatusEx function of
kernel32.dll is pretty fast. At this moment I am forced to call this
function through NETLink and even in such way it does not take much of
processor time on my machine. But using NETLink requires to be under Windows
and have Microsoft .NET 2 or later to be installed. It also seems that for
64-bit systems I should implement another function.
I think it would be very useful to have such function inside MathKernel and
do not rely on .NET.
As I understand MemoryConstrained works in the same way: it checks the
amount of memory used with some intervals but the intervals depends on what
is happening inside the MathKernel at this moment and this is the reason why
it often fails.
>
> If you do not want your system to go into swap, have you tried switching
> it off in the first place?
>
> Assuming you are on Linux you could use ulimit to set memory constraints.
> This then would allow other processes to access the rest of the memory. I
> have no idea how and if this works, say, on windows.
I did not know about this, thank you. Now I am interested to find such
function for Windows.
Alexey Popkov
I have discovered that in standard cases when using LinkInterrupt[link, 1]
my operating system (Windows 2000 at the moment) releases physical memory
only in 0.05-0.1 second beginning with a moment of execution of
LinkInterrupt[link, 1] while with LinkClose[link] it releases physical
memory in 0.01-0.03 second (both values include the time, spent on execution
of the command itself). Time intervals were measured by using SessionTime[]
under equal conditions and are steadily reproduced.
Is there a way to make LinkInterrupt[link, 1] faster?
Alexey Popkov
----- Original Message -----
From: "Todd Gayley" <tga...@wolfram.com>
To: "Alexey" <leh...@gmail.com>; <math...@smc.vnet.net>
Sent: Thursday, March 10, 2011 4:09 AM
Subject: Re: How to kill slave kernel securely?
> At 06:00 AM 3/9/2011, Alexey wrote:
> >Good day,
> >
> >LinkClose[link] "does not necessarily terminate the program at the
> >other end of the connection" as it is said in the Documentation. Is
> >there a way to kill the process of the slave kernel securely?
>
>
> Alexey,
>
Alexey Popkov
In really I just need a function that kills the slave kernel securely and
returns only when the process of the slave kernel has already killed and its
memory has already released.