Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Updating a running executable ("text file busy")
2,622 views
Skip to first unread message
Kenny McCormack
May 29, 2015, 9:08:43 AM5/29/15
to
I've discovered that, on Linux, if I (try to) "cp" a new version of an
executable over an existing, running one (say, /usr/bin/xyz), I
sometimes/usually get error "text file busy" and the file copy fails. This
is normal and expected.
But, I've found that I can just remove ("rm") the file and that works
(IMLE, of course...). And, of course, once it is removed, you can copy
over a new version.
So my questions are:
1) Why does it let me remove it, but not copy over it? Note that
other, more contentious OSes, wouldn't never let you remove it if
it is open or "in-use".
2) What happens to the running program if/when it finds it needs to
access its "text" and the file has been removed?
Note that if the file is nfs-mounted, then when you remove it but somebody
still has it open, the file gets renamed something like .nfs123456789 and
the system automagically takes care of getting rid of this temporary file
when it can. But the case I am talking about here (in this thread) is when
nfs isn't involved.
--
"That's the eternal flame."[7] The single became another worldwide hit.[8]
Hoffs was actually naked when she recorded the song, after being convinced
by Sigerson that Olivia Newton-John got her amazing performances by
recording everything while naked.[9]
(From: http://en.wikipedia.org/wiki/The_Bangles)
executable over an existing, running one (say, /usr/bin/xyz), I
sometimes/usually get error "text file busy" and the file copy fails. This
is normal and expected.
But, I've found that I can just remove ("rm") the file and that works
(IMLE, of course...). And, of course, once it is removed, you can copy
over a new version.
So my questions are:
1) Why does it let me remove it, but not copy over it? Note that
other, more contentious OSes, wouldn't never let you remove it if
it is open or "in-use".
2) What happens to the running program if/when it finds it needs to
access its "text" and the file has been removed?
Note that if the file is nfs-mounted, then when you remove it but somebody
still has it open, the file gets renamed something like .nfs123456789 and
the system automagically takes care of getting rid of this temporary file
when it can. But the case I am talking about here (in this thread) is when
nfs isn't involved.
--
"That's the eternal flame."[7] The single became another worldwide hit.[8]
Hoffs was actually naked when she recorded the song, after being convinced
by Sigerson that Olivia Newton-John got her amazing performances by
recording everything while naked.[9]
(From: http://en.wikipedia.org/wiki/The_Bangles)
Xavier Roche
May 29, 2015, 9:21:43 AM5/29/15
to
Le 29/05/2015 15:08, Kenny McCormack a écrit :
> 1) Why does it let me remove it, but not copy over it?
These are totally different things.
> 1) Why does it let me remove it, but not copy over it?
* removing the file is a directory issue (ie. xyz is being removed from the directory /usr/bin) and has no impact on the file itself (except decreasing the refcount of the file)
* copying data over the file is a file issue (ie. you need write access to the file contents - access which is probably locked when the executable is mapped with execution access rights ?)
> other, more contentious OSes, wouldn't never let you remove it if
> it is open or "in-use".
> 2) What happens to the running program if/when it finds it needs to
> access its "text" and the file has been removed?
Siri Cruz
May 29, 2015, 9:29:48 AM5/29/15
to
In article <mk9ocm$vns$1...@news.xmission.com>,
until all opens close. rm than copy creates a new directory entry and inode.
The kernel is allowed to page in read only parts from the executable file to the
real memory. To prevent paging in garbage, the executable inode is not
modifiable.
> 2) What happens to the running program if/when it finds it needs to
> access its "text" and the file has been removed?
The kernel should maintain open connections to the inode until all processes
executing it have exitted.
> Note that if the file is nfs-mounted, then when you remove it but somebody
> still has it open, the file gets renamed something like .nfs123456789 and
> the system automagically takes care of getting rid of this temporary file
> when it can. But the case I am talking about here (in this thread) is when
> nfs isn't involved.
NFS doesn't keep the file open on the host and has to do a pretend unlink to
mimic the behaviour of local files.
--
:-<> Siri Seal of Disavowal #000-001. Disavowed. Denied. Deleted.
'I desire mercy, not sacrifice.'
When is a Kenyan not a Kenyan? When he's a Canadian.
That's People's Commissioner Siri Cruz now. Punch!
gaz...@shell.xmission.com (Kenny McCormack) wrote:
> 1) Why does it let me remove it, but not copy over it? Note that
> other, more contentious OSes, wouldn't never let you remove it if
> it is open or "in-use".
rm unlinks the directory entry from the inode, but the inode continues to exist
> 1) Why does it let me remove it, but not copy over it? Note that
> other, more contentious OSes, wouldn't never let you remove it if
> it is open or "in-use".
until all opens close. rm than copy creates a new directory entry and inode.
The kernel is allowed to page in read only parts from the executable file to the
real memory. To prevent paging in garbage, the executable inode is not
modifiable.
> 2) What happens to the running program if/when it finds it needs to
> access its "text" and the file has been removed?
executing it have exitted.
> Note that if the file is nfs-mounted, then when you remove it but somebody
> still has it open, the file gets renamed something like .nfs123456789 and
> the system automagically takes care of getting rid of this temporary file
> when it can. But the case I am talking about here (in this thread) is when
> nfs isn't involved.
mimic the behaviour of local files.
--
:-<> Siri Seal of Disavowal #000-001. Disavowed. Denied. Deleted.
'I desire mercy, not sacrifice.'
When is a Kenyan not a Kenyan? When he's a Canadian.
That's People's Commissioner Siri Cruz now. Punch!
Kaz Kylheku
May 29, 2015, 9:49:52 AM5/29/15
to
On 2015-05-29, Kenny McCormack <gaz...@shell.xmission.com> wrote:
> I've discovered that, on Linux, if I (try to) "cp" a new version of an
> executable over an existing, running one (say, /usr/bin/xyz), I
> sometimes/usually get error "text file busy" and the file copy fails. This
> is normal and expected.
>
> But, I've found that I can just remove ("rm") the file and that works
> (IMLE, of course...). And, of course, once it is removed, you can copy
> over a new version.
>
> So my questions are:
>
> 1) Why does it let me remove it, but not copy over it? Note that
> other, more contentious OSes, wouldn't never let you remove it if
> it is open or "in-use".
In Unix, objects in the file system have a persistent reference count. An
> I've discovered that, on Linux, if I (try to) "cp" a new version of an
> executable over an existing, running one (say, /usr/bin/xyz), I
> sometimes/usually get error "text file busy" and the file copy fails. This
> is normal and expected.
>
> But, I've found that I can just remove ("rm") the file and that works
> (IMLE, of course...). And, of course, once it is removed, you can copy
> over a new version.
>
> So my questions are:
>
> 1) Why does it let me remove it, but not copy over it? Note that
> other, more contentious OSes, wouldn't never let you remove it if
> it is open or "in-use".
object that appears under two different names in the same directory, or in
multiple dirctories, has a reference count of two. An object which is removed
from the directory structure has a reference count of zero.
There is an additional reference count: an in-memory reference count
which tracks how many descriptors are open on an object.
An object is not recycled (i.e. keeps existing) until both reference counts go
to zero: it has to be removed from the directory structure and all file
descriptors have to be closed. On the "last close", the object becomes
garbage.
If a file is deleted, but open, and the OS crashes, then you have a situation
of an object that doesn't exist in the directory structure, but has not been
reclaimed properly (continues to occupy storage). One of the jobs of a "fsck"
program is to find these and release them properly.
> 2) What happens to the running program if/when it finds it needs to
> access its "text" and the file has been removed?
own executable, it is screwed.
For instance, if we look at Linux, /proc/self/exe is just a symlink,
so that won't help.
Programs don't usually need access to their executable. Sometimes they do
though; for instance executables created by interpreted languages might have
an "image" appended to a generic executable part. Usually, that is accessed
at start up. Another use case is programs accessing their own debug symbols.
> Note that if the file is nfs-mounted, then when you remove it but somebody
> still has it open, the file gets renamed something like .nfs123456789 and
> the system automagically takes care of getting rid of this temporary file
> when it can. But the case I am talking about here (in this thread) is when
> nfs isn't involved.
Because the in-memory reference counts are not sufficient for NFS.
NFS is supposed to be "stateless", in that you can reboot the NFS server,
and things continue working from the clients' perspectives (when the server
comes back up). These explicit .nfs123456789 files can survive a reboot, which
then helps the client achieve the usual Unix semantics that the file remains
accessible while it is open.
Barry Margolin
May 29, 2015, 10:20:59 AM5/29/15
to
> I've discovered that, on Linux, if I (try to) "cp" a new version of an
> executable over an existing, running one (say, /usr/bin/xyz), I
> sometimes/usually get error "text file busy" and the file copy fails. This
> is normal and expected.
>
> But, I've found that I can just remove ("rm") the file and that works
> (IMLE, of course...). And, of course, once it is removed, you can copy
> over a new version.
>
> So my questions are:
>
> 1) Why does it let me remove it, but not copy over it? Note that
> other, more contentious OSes, wouldn't never let you remove it if
> it is open or "in-use".
It's the same as when you remove a file that some process has an open FD
> executable over an existing, running one (say, /usr/bin/xyz), I
> sometimes/usually get error "text file busy" and the file copy fails. This
> is normal and expected.
>
> But, I've found that I can just remove ("rm") the file and that works
> (IMLE, of course...). And, of course, once it is removed, you can copy
> over a new version.
>
> So my questions are:
>
> 1) Why does it let me remove it, but not copy over it? Note that
> other, more contentious OSes, wouldn't never let you remove it if
> it is open or "in-use".
for. The directory entry is removed, but the inode and file contents
stick around until all processes close it.
> 2) What happens to the running program if/when it finds it needs to
> access its "text" and the file has been removed?
has been removed.
> Note that if the file is nfs-mounted, then when you remove it but somebody
> still has it open, the file gets renamed something like .nfs123456789 and
> the system automagically takes care of getting rid of this temporary file
> when it can. But the case I am talking about here (in this thread) is when
> nfs isn't involved.
With a local file, it's handled internally within the kernel, but NFS
needs this kludge because the server's kernel doesn't know about file
openings on clients.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
Rainer Weikusat
May 29, 2015, 11:13:49 AM5/29/15
to
gaz...@shell.xmission.com (Kenny McCormack) writes:
> I've discovered that, on Linux, if I (try to) "cp" a new version of an
> executable over an existing, running one (say, /usr/bin/xyz), I
> sometimes/usually get error "text file busy" and the file copy fails. This
> is normal and expected.
>
> But, I've found that I can just remove ("rm") the file and that works
> (IMLE, of course...). And, of course, once it is removed, you can copy
> over a new version.
Something which wasn't mentioned so far: This is actually desirable
> I've discovered that, on Linux, if I (try to) "cp" a new version of an
> executable over an existing, running one (say, /usr/bin/xyz), I
> sometimes/usually get error "text file busy" and the file copy fails. This
> is normal and expected.
>
> But, I've found that I can just remove ("rm") the file and that works
> (IMLE, of course...). And, of course, once it is removed, you can copy
> over a new version.
because it means the file can be updated with affecting any processes
currently using it. These will continue to see the old file until they
stop referring to it.
Siri Cruz
May 29, 2015, 11:35:58 AM5/29/15
to
In article <87oal38...@doppelsaurus.mobileactivedefense.com>,
Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
> gaz...@shell.xmission.com (Kenny McCormack) writes:
> > I've discovered that, on Linux, if I (try to) "cp" a new version of an
> > executable over an existing, running one (say, /usr/bin/xyz), I
> > sometimes/usually get error "text file busy" and the file copy fails. This
> > is normal and expected.
> >
> > But, I've found that I can just remove ("rm") the file and that works
> > (IMLE, of course...). And, of course, once it is removed, you can copy
> > over a new version.
>
> Something which wasn't mentioned so far: This is actually desirable
> because it means the file can be updated with affecting any processes
without?
> gaz...@shell.xmission.com (Kenny McCormack) writes:
> > I've discovered that, on Linux, if I (try to) "cp" a new version of an
> > executable over an existing, running one (say, /usr/bin/xyz), I
> > sometimes/usually get error "text file busy" and the file copy fails. This
> > is normal and expected.
> >
> > But, I've found that I can just remove ("rm") the file and that works
> > (IMLE, of course...). And, of course, once it is removed, you can copy
> > over a new version.
>
> Something which wasn't mentioned so far: This is actually desirable
> because it means the file can be updated with affecting any processes
> currently using it. These will continue to see the old file until they
> stop referring to it.
> stop referring to it.
Rainer Weikusat
May 29, 2015, 11:52:20 AM5/29/15
to
Siri Cruz <chine...@yahoo.com> writes:
> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>
>> gaz...@shell.xmission.com (Kenny McCormack) writes:
>> > I've discovered that, on Linux, if I (try to) "cp" a new version of an
>> > executable over an existing, running one (say, /usr/bin/xyz), I
>> > sometimes/usually get error "text file busy" and the file copy fails. This
>> > is normal and expected.
>> >
>> > But, I've found that I can just remove ("rm") the file and that works
>> > (IMLE, of course...). And, of course, once it is removed, you can copy
>> > over a new version.
>>
>> Something which wasn't mentioned so far: This is actually desirable
>> because it means the file can be updated with affecting any processes
> without?
>> currently using it. These will continue to see the old file until they
>> stop referring to it.
Well, yes. When not being tightly controlled, my mind has a tendency to
> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>
>> gaz...@shell.xmission.com (Kenny McCormack) writes:
>> > I've discovered that, on Linux, if I (try to) "cp" a new version of an
>> > executable over an existing, running one (say, /usr/bin/xyz), I
>> > sometimes/usually get error "text file busy" and the file copy fails. This
>> > is normal and expected.
>> >
>> > But, I've found that I can just remove ("rm") the file and that works
>> > (IMLE, of course...). And, of course, once it is removed, you can copy
>> > over a new version.
>>
>> Something which wasn't mentioned so far: This is actually desirable
>> because it means the file can be updated with affecting any processes
> without?
>> currently using it. These will continue to see the old file until they
>> stop referring to it.
invert things, eg, in former times, it used to happen to me that I would
write code doing the exact opposite of what I wanted it to do. Probably
due to increased practice, this hasn't happened for some years. But it's
still a problem for 'casually' written texts.
Kenny McCormack
May 29, 2015, 12:12:08 PM5/29/15
to
In article <87oal38...@doppelsaurus.mobileactivedefense.com>,
Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
is:
Yes, that's the right way to do it. Remove it first, then copy in the
new one. Everything will work out correctly.
--
Religion is regarded by the common people as true,
by the wise as foolish,
and by the rulers as useful.
(Seneca the Younger, 65 AD)
Richard Kettlewell
May 29, 2015, 12:20:24 PM5/29/15
to
gaz...@shell.xmission.com (Kenny McCormack) writes:
> So, the tl;dr version of all the verbiage that got posted in this thread
> is:
>
> Yes, that's the right way to do it. Remove it first, then copy in the
> new one. Everything will work out correctly.
Rename-into-place is the usual idiom, to avoid an interval in the name
> So, the tl;dr version of all the verbiage that got posted in this thread
> is:
>
> Yes, that's the right way to do it. Remove it first, then copy in the
> new one. Everything will work out correctly.
doesn’t exist.
--
http://www.greenend.org.uk/rjk/
Barry Margolin
May 29, 2015, 3:39:52 PM5/29/15
to
In article
<wwvk2vr...@l1AntVDjLrnP7Td3DQJ8ynzIq3lJMueXf87AxnpFoA.invalid>,
This is why the rename(2) system call is allowed to succeed when the
target filename already exists. It ensures that the removal and rename
are atomic.
If you want to copy it rather than rename (which would be necessary if
the source is on a different filesystem), you need to do it in two
steps. First copy to a temporary name on the destination filesystem,
then rename that to the target.
<wwvk2vr...@l1AntVDjLrnP7Td3DQJ8ynzIq3lJMueXf87AxnpFoA.invalid>,
target filename already exists. It ensures that the removal and rename
are atomic.
If you want to copy it rather than rename (which would be necessary if
the source is on a different filesystem), you need to do it in two
steps. First copy to a temporary name on the destination filesystem,
then rename that to the target.
Huibert Bol
May 31, 2015, 5:22:32 PM5/31/15
to
Kaz Kylheku wrote:
>> 2) What happens to the running program if/when it finds it needs to
>> access its "text" and the file has been removed?
>
> Unless there is some special mechanism for a process to gain access to its
> own executable, it is screwed.
>
> For instance, if we look at Linux, /proc/self/exe is just a symlink,
> so that won't help.
It only *looks* like a symlink, you can actually access it even if the
>> 2) What happens to the running program if/when it finds it needs to
>> access its "text" and the file has been removed?
>
> Unless there is some special mechanism for a process to gain access to its
> own executable, it is screwed.
>
> For instance, if we look at Linux, /proc/self/exe is just a symlink,
> so that won't help.
original executable is removed.
--
Huibert
"Okay... really not something I needed to see." --Raven
Barry Margolin
May 31, 2015, 9:18:09 PM5/31/15
to
In article <201505290...@kylheku.com>,
virtual memory.
Kaz Kylheku <k...@kylheku.com> wrote:
> > 2) What happens to the running program if/when it finds it needs to
> > access its "text" and the file has been removed?
>
> Unless there is some special mechanism for a process to gain access to its
> own executable, it is screwed.
Why does it need a "special mechanism"? The text is in the process's
> > 2) What happens to the running program if/when it finds it needs to
> > access its "text" and the file has been removed?
>
> Unless there is some special mechanism for a process to gain access to its
> own executable, it is screwed.
virtual memory.
Kaz Kylheku
Jun 1, 2015, 12:11:43 AM6/1/15
to
On 2015-06-01, Barry Margolin <bar...@alum.mit.edu> wrote:
> In article <201505290...@kylheku.com>,
> Kaz Kylheku <k...@kylheku.com> wrote:
>
>> > 2) What happens to the running program if/when it finds it needs to
>> > access its "text" and the file has been removed?
>>
>> Unless there is some special mechanism for a process to gain access to its
>> own executable, it is screwed.
>
> Why does it need a "special mechanism"? The text is in the process's
> virtual memory.
Not all areas of the executable are mapped, and the process may want to get to
> In article <201505290...@kylheku.com>,
> Kaz Kylheku <k...@kylheku.com> wrote:
>
>> > 2) What happens to the running program if/when it finds it needs to
>> > access its "text" and the file has been removed?
>>
>> Unless there is some special mechanism for a process to gain access to its
>> own executable, it is screwed.
>
> Why does it need a "special mechanism"? The text is in the process's
> virtual memory.
some of the other ones.
James K. Lowden
Jun 1, 2015, 10:03:05 AM6/1/15
to
some of the file? Is there one that works that way? I would expect
the whole file is mapped (with mmap basically), and reference to any
unmapped page provokes the paging system to bring it into memory.
In any case the process surely holds a descriptor and can mmap any
other portion of the file, whether or not it has a name. It doesn't
*need* a "special mechanism" unless it throws that away, too, does it?
--jkl
James K. Lowden
Jun 1, 2015, 10:06:34 AM6/1/15
to
On Fri, 29 May 2015 16:12:05 +0000 (UTC)
gaz...@shell.xmission.com (Kenny McCormack) wrote:
> >Something which wasn't mentioned so far: This is actually desirable
> >because it means the file can be updated with affecting any processes
> >currently using it. These will continue to see the old file until
> >they stop referring to it.
>
> So, the tl;dr version of all the verbiage that got posted in this
> thread is:
>
> Yes, that's the right way to do it. Remove it first, then copy
> in the new one. Everything will work out correctly.
Something else, too, IIUC: cp(1) does not create-and-rename. It opens
gaz...@shell.xmission.com (Kenny McCormack) wrote:
> >Something which wasn't mentioned so far: This is actually desirable
> >because it means the file can be updated with affecting any processes
> >currently using it. These will continue to see the old file until
> >they stop referring to it.
>
> So, the tl;dr version of all the verbiage that got posted in this
> thread is:
>
> Yes, that's the right way to do it. Remove it first, then copy
> in the new one. Everything will work out correctly.
the file and replaces the contents. The semantics are different,
because all hard links to the inode continue to see consistent
information.
--jkl
Casper H.S. Dik
Jun 1, 2015, 10:08:18 AM6/1/15
to
"James K. Lowden" <jklo...@speakeasy.net> writes:
>Are you saying that when a executable is launched, the OS maps only
>some of the file? Is there one that works that way? I would expect
>the whole file is mapped (with mmap basically), and reference to any
>unmapped page provokes the paging system to bring it into memory.
No; an executable contains a number of sections; these are all
>Are you saying that when a executable is launched, the OS maps only
>some of the file? Is there one that works that way? I would expect
>the whole file is mapped (with mmap basically), and reference to any
>unmapped page provokes the paging system to bring it into memory.
mapped in different places. And some of sections aren't mapped,
e.g., there is really no reason to map the debugging symbols.
>In any case the process surely holds a descriptor and can mmap any
>other portion of the file, whether or not it has a name. It doesn't
>*need* a "special mechanism" unless it throws that away, too, does it?
file descriptor.
Casper
Scott Lurndal
Jun 1, 2015, 12:13:10 PM6/1/15
to
"James K. Lowden" <jklo...@speakeasy.net> writes:
>On Mon, 1 Jun 2015 04:11:37 +0000 (UTC)
>Kaz Kylheku <k...@kylheku.com> wrote:
>
>> On 2015-06-01, Barry Margolin <bar...@alum.mit.edu> wrote:
>> > In article <201505290...@kylheku.com>,
>> > Kaz Kylheku <k...@kylheku.com> wrote:
>> >
>> >> > 2) What happens to the running program if/when it finds it
>> >> > needs to access its "text" and the file has been removed?
>> >>
>> >> Unless there is some special mechanism for a process to gain
>> >> access to its own executable, it is screwed.
>> >
>> > Why does it need a "special mechanism"? The text is in the
>> > process's virtual memory.
>>
>> Not all areas of the executable are mapped, and the process may want
>> to get to some of the other ones.
>
>Are you saying that when a executable is launched, the OS maps only
>some of the file?
Yes. Specifically, text, rodata/data and bss are mapped. The
>Kaz Kylheku <k...@kylheku.com> wrote:
>
>> On 2015-06-01, Barry Margolin <bar...@alum.mit.edu> wrote:
>> > In article <201505290...@kylheku.com>,
>> > Kaz Kylheku <k...@kylheku.com> wrote:
>> >
>> >> > 2) What happens to the running program if/when it finds it
>> >> > needs to access its "text" and the file has been removed?
>> >>
>> >> Unless there is some special mechanism for a process to gain
>> >> access to its own executable, it is screwed.
>> >
>> > Why does it need a "special mechanism"? The text is in the
>> > process's virtual memory.
>>
>> Not all areas of the executable are mapped, and the process may want
>> to get to some of the other ones.
>
>Are you saying that when a executable is launched, the OS maps only
>some of the file?
debug symbol tables, the ELF headers and the comment section
are not mapped. for C++, the unwind structures and rtti information
is also mapped as part of the data.
Note that the sections of an ELF file are not ordered
by virtual address in the file itself, so mapping the
entire thing would be pointless.
Barry Margolin
Jun 1, 2015, 12:25:56 PM6/1/15
to
In article <kw%ax.14862$FK.1...@fx13.iad>,
The earlier message said "needs access its 'text'", which I thought
referred to the text segment.
referred to the text segment.
0 new messages