pivot_root. unmount old root
Katharina Haselhorst
I'm running debian lenny with xen kernel 2.6.26, amd64. Inside a domU I
need to make a pivot_root and unmount the old root afterwards.
I've done the following:
cd /newroot
(newroot contains a minimal system from initrd, dev/console, dev/null
and old-root are available unter newroot/)
exec <dev/console >dev/console 2>&1
pivot_root . old-root
exec chroot . bin/sh
umount -n old-root
but I always get: device is busy - so I can't umount the old root.
/proc/mounts shows that the proc fs is still mounten under old-root/proc
and an umount old-root/proc doesn't work either (device is busy, too).
I also tried to umount proc before doing the pivot_root, but no luck either.
Why doesn't pivot_root remove the proc mountpoint from old-root? As I
understand the documentation - it should clear all dependencies so that
the old-root can be unmounted afterwards...
Any ideas or suggestions to help would be great!
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4BA4A71F...@mathematik.uni-marburg.de
Rogerio Luz Coelho
If so stop X before atempting a umount
Rogerio
Katharina Haselhorst
new root before actually doing the pivot-root. proc/mounts doesn't show
any mountpoints under old-root afterwards, but an umount still gives
device busy...
On 03/20/2010 02:33 PM, Rogerio Luz Coelho wrote:
> are you doing this with X enabled?
>
> If so stop X before atempting a umount
>
> Rogerio
>
> 2010/3/20 Katharina Haselhorst <bra...@mathematik.uni-marburg.de
> <mailto:bra...@mathematik.uni-marburg.de>>
>
> Hello,
>
> I'm running debian lenny with xen kernel 2.6.26, amd64. Inside a
> domU I need to make a pivot_root and unmount the old root afterwards.
> I've done the following:
>
> cd /newroot
> (newroot contains a minimal system from initrd, dev/console,
> dev/null and old-root are available unter newroot/)
> exec <dev/console >dev/console 2>&1
> pivot_root . old-root
> exec chroot . bin/sh
> umount -n old-root
>
> but I always get: device is busy - so I can't umount the old root.
> /proc/mounts shows that the proc fs is still mounten under
> old-root/proc and an umount old-root/proc doesn't work either
> (device is busy, too).
>
> I also tried to umount proc before doing the pivot_root, but no luck
> either.
> Why doesn't pivot_root remove the proc mountpoint from old-root? As
> I understand the documentation - it should clear all dependencies so
> that the old-root can be unmounted afterwards...
>
> Any ideas or suggestions to help would be great!
>
> K. Haselhorst
>
>
> --
> To UNSUBSCRIBE, email to debian-us...@lists.debian.org
> <mailto:debian-us...@lists.debian.org> with a subject of
> "unsubscribe". Trouble? Contact listm...@lists.debian.org
> <mailto:listm...@lists.debian.org>
> Archive:
> http://lists.debian.org/4BA4A71F...@mathematik.uni-marburg.de
>
>
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4BA4ED31...@mathematik.uni-marburg.de
Stephen Powell
>
> no, I don't have X. I just tried to mount --move /dev and /proc to the
> new root before actually doing the pivot-root. proc/mounts doesn't show
> any mountpoints under old-root afterwards, but an umount still gives
> device busy...
Perhaps the "fuser" command would come in handy here. "fuser" is part
of the psmisc package and is useful for determining what process or
processes are preventing a device from being unmounted. For example,
let's say you want to umount "/home", which is a separate partition.
The command
fuser -m /home
will list the process ids which are accessing any file under /home.
Compare that to the output of "ps aux" to see which processes you
need to terminate in order to be able to umount /home.
--
.''`. Stephen Powell <zlin...@wowway.com>
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1660971449.20495501269...@md01.wow.synacor.com
Katharina Haselhorst
> fuser -m /home
>
> will list the process ids which are accessing any file under /home.
> Compare that to the output of "ps aux" to see which processes you
> need to terminate in order to be able to umount /home.
Since I'm doing a pivot_root before trying to umount the old root there
are still several processes keeping some files open inside the old root
subdirs. Init is still running, als well as rc and one shutdown script
(atm I'm doing the pivot_root just before shutdown, because most
processes are terminted by the time). If I kill one of these processes,
the systems gets shut down.
I don't understand why there is still that shutdown script running,
because I'm doing a exec chroot. The pivot_root's manual says: "Note
that exec chroot changes the running executable, which is necessary if
the old root directory should be unmounted afterwards."
And the init process is always running, so how does pivot_root handle
the open files of init?
I did move the /proc and /dev mountpoints from old-root to the new root
- might that cause some problems? If I don't do that - lsof or fuser of
course don't show any open files for old-root, but I'm not able to
unmount to old-root either.
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4BA62144...@mathematik.uni-marburg.de
Stephen Powell
> Since I'm doing a pivot_root before trying to umount the old root there
> are still several processes keeping some files open inside the old root
> subdirs. Init is still running, als well as rc and one shutdown script
> (atm I'm doing the pivot_root just before shutdown, because most
> processes are terminted by the time). If I kill one of these processes,
> the systems gets shut down.
> I don't understand why there is still that shutdown script running,
> because I'm doing a exec chroot. The pivot_root's manual says: "Note
> that exec chroot changes the running executable, which is necessary if
> the old root directory should be unmounted afterwards."
> And the init process is always running, so how does pivot_root handle
> the open files of init?
> I did move the /proc and /dev mountpoints from old-root to the new root
> - might that cause some problems? If I don't do that - lsof or fuser of
> course don't show any open files for old-root, but I'm not able to
> unmount to old-root either.
Maybe it's time to step back and ask a more basic question.
What is it that you are trying to accomplish? I know that you are trying
to do a pivot_root. But *why* are you trying to do a pivot_root?
Why do you think you need to do it? What is the real-world problem
that you are trying to solve?
--
.''`. Stephen Powell <zlin...@wowway.com>
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1519384701.20787971269...@md01.wow.synacor.com
Katharina Haselhorst
> Maybe it's time to step back and ask a more basic question.
> What is it that you are trying to accomplish? I know that you are trying
> to do a pivot_root. But *why* are you trying to do a pivot_root?
> Why do you think you need to do it? What is the real-world problem
> that you are trying to solve?
I want to go back into an initramfs setting so that only the running
kernel and one specific process remains. Goal is to be able to boot a
new system from there with a new hard drive (for example mounted via nfs
or some other way) withouth the need to reboot the entire kernel. The
whole thing is running within a xen domU atm. So the new hard disk could
be passed through via xend or mounted via nfs. And for cleanly
terminating the first system I need to unmount the root filesystem and
somehow replace the old init process with my specific process running in
that initramfs setting.
According to the pivot_root manpage, it should be possible to clear all
dependencies to the old root fs so that it can be unmounted after the
pivot_root command. And that's what not working for me.
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4BA7BF1E...@mathematik.uni-marburg.de
Stephen Powell
> Stephen Powell wrote:
>> Maybe it's time to step back and ask a more basic question.
>> What is it that you are trying to accomplish? I know that you are trying
>> to do a pivot_root. But *why* are you trying to do a pivot_root?
>> Why do you think you need to do it? What is the real-world problem
>> that you are trying to solve?
>
> I want to go back into an initramfs setting so that only the running
> kernel and one specific process remains. Goal is to be able to boot a
> new system from there with a new hard drive (for example mounted via nfs
> or some other way) withouth the need to reboot the entire kernel. The
> whole thing is running within a xen domU atm. So the new hard disk could
> be passed through via xend or mounted via nfs. And for cleanly
> terminating the first system I need to unmount the root filesystem and
> somehow replace the old init process with my specific process running in
> that initramfs setting.
> According to the pivot_root manpage, it should be possible to clear all
> dependencies to the old root fs so that it can be unmounted after the
> pivot_root command. And that's what not working for me.
You would probably want to run all the executable files in /etc/rc6.d
in alphabetical order, supplying the "stop" parameter, with the
exception of the last one, which on my system is S90reboot.
Then run something like this:
----------
#!/bin/sh
# Example: mount the new root file system over NFS from 10.0.0.1:/my_root
# and run init.
ifconfig lo 127.0.0.1 up # for portmap
# configure Ethernet or such
portmap # for lockd (implicitly started by mount)
mount -o ro 10.0.0.1:/my_root /mnt
killall portmap # portmap keeps old root busy
cd /mnt
pivot_root . old_root
exec chroot . sh -c 'umount /old_root; exec /sbin/init \
<dev/console >dev/console 2>&1
----------
The "# configure Ethernet or such" line is what is most likely
to give you trouble. Exactly what goes there is dependent on
your environment.
Another trick: "telinit -u" can sometimes be used to effectively
restart init, which may cause it to close the old files.
Debian uses this when applying maintenance
to the init program itself without rebooting.
HTH
--
.''`. Stephen Powell <zlin...@wowway.com>
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1055911490.20899381269...@md01.wow.synacor.com
Katharina Haselhorst
> in alphabetical order, supplying the "stop" parameter, with the
> exception of the last one, which on my system is S90reboot.
that's what I was doing - only with runlevel 0 without doing the actual
halt at the end.
> Then run something like this:
>
> ----------
>
> #!/bin/sh
> # Example: mount the new root file system over NFS from 10.0.0.1:/my_root
> # and run init.
> ifconfig lo 127.0.0.1 up # for portmap
> # configure Ethernet or such
> portmap # for lockd (implicitly started by mount)
> mount -o ro 10.0.0.1:/my_root /mnt
> killall portmap # portmap keeps old root busy
> cd /mnt
> pivot_root . old_root
> exec chroot . sh -c 'umount /old_root; exec /sbin/init \
> <dev/console>dev/console 2>&1
>
> ----------
I only see 2 places (without patching init itself) to put these commands:
1. into a new shutdown script in place of S90{halt,reboot}
2. into the rc script
But no matter which of these I choose there remains at least init and
perhaps the process executing rc running and keeping the old root busy.
So I cannot unmount it. At least on my system init has opened some
shared libraries in old-root/lib/... and rc as well (if it is still running)
The same would happen if I restarted init with telinit -u runlevel...
Your script above is from the manpage of pivot_root, isn't it? I really
wonder in which context this example (and also the other one given in
the manpage) could work? Have you sucessfully tried it on your system?
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4BA7ECFC...@mathematik.uni-marburg.de
Stephen Powell
> Stephen Powell wrote:
>> in alphabetical order, supplying the "stop" parameter, with the
>> exception of the last one, which on my system is S90reboot.
>
> that's what I was doing - only with runlevel 0 without doing the actual
> halt at the end.
>
Good.
>> Then run something like this:
>>
>> ----------
>>
>> #!/bin/sh
>> # Example: mount the new root file system over NFS from 10.0.0.1:/my_root
>> # and run init.
>> ifconfig lo 127.0.0.1 up # for portmap
>> # configure Ethernet or such
>> portmap # for lockd (implicitly started by mount)
>> mount -o ro 10.0.0.1:/my_root /mnt
>> killall portmap # portmap keeps old root busy
>> cd /mnt
>> pivot_root . old_root
>> exec chroot . sh -c 'umount /old_root; exec /sbin/init \
>> <dev/console>dev/console 2>&1
>>
>> ----------
>
> I only see 2 places (without patching init itself) to put these commands:
> 1. into a new shutdown script in place of S90{halt,reboot}
> 2. into the rc script
>
> But no matter which of these I choose there remains at least init and
> perhaps the process executing rc running and keeping the old root busy.
> So I cannot unmount it. At least on my system init has opened some
> shared libraries in old-root/lib/... and rc as well (if it is still running)
>
> The same would happen if I restarted init with telinit -u runlevel...
>
> Your script above is from the manpage of pivot_root, isn't it?
Yes.
> I really wonder in which context this example (and also the other one given
> in the manpage) could work? Have you sucessfully tried it on your system?
No, I've never had occasion to.
But where exactly is the failure occuring?
Does the mount command fail?
Does the pivot_root command fail? Does exec chroot fail?
Kill should be able to kill any process *except* init itself.
And "telinit -u" should be able to refresh init.
Something like this occurs during boot with the transition from the initial
RAM filesystem to the permanent root filesystem. Perhaps you should
study the scripts in /etc/rcS.d to find how the root file system is
changed there. Maybe that will give you some clues.
--
.''`. Stephen Powell <zlin...@wowway.com>
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1037565534.20929511269...@md01.wow.synacor.com
bra...@mathematik.uni-marburg.de
>> given
>> in the manpage) could work? Have you sucessfully tried it on your
>> system?
>
> No, I've never had occasion to.
> But where exactly is the failure occuring?
> Does the mount command fail?
> Does the pivot_root command fail? Does exec chroot fail?
The umount command fails with "device is busy". privot root and chroot
work fine.
> Kill should be able to kill any process *except* init itself.
> And "telinit -u" should be able to refresh init.
I did a telinit u after the chroot and killed the remaining processes.
After that lsof doesn't show any open files under old-root but umount
still fails with "device is busy".
> Something like this occurs during boot with the transition from the
> initial
> RAM filesystem to the permanent root filesystem. Perhaps you should
> study the scripts in /etc/rcS.d to find how the root file system is
> changed there. Maybe that will give you some clues.
No, the transition from ramfs to the root filesystem isn't done by the
init scripts but by init itself. I issues a switch-root, that means all
contents from the ramfs is deleted and the root is mounted over with the
new root filesystem. I can't do that because the old rootfs is a normal
partition on a hard drive and the contents can't be simply deleted. So I
need to do some kind of pivot-root. Perhaps there is an alternative? The
problem atm is that it's not possible to unmount to old root fs.
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/54563.137.248.74.196...@www.mathematik.uni-marburg.de
Stephen Powell
> Stephen Powell wrote:
>> Does the mount command fail?
>> Does the pivot_root command fail? Does exec chroot fail?
>
> The umount command fails with "device is busy". privot root and chroot
> work fine.
So everything works except the umount. Interesting. From the example
in the man page they obviously expect it to be possible to umount the
old root file system.
>> Kill should be able to kill any process *except* init itself.
>> And "telinit -u" should be able to refresh init.
>
> I did a telinit u after the chroot and killed the remaining processes.
> After that lsof doesn't show any open files under old-root but umount
> still fails with "device is busy".
Maybe lsof doesn't necessarily list everything. What does "fuser -m /old_root"
show? What about doing "fuser -k -m /old_root"?
--
.''`. Stephen Powell <zlin...@wowway.com>
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/814445304.210912912693...@md01.wow.synacor.com
bra...@mathematik.uni-marburg.de
>> work fine.
>
> So everything works except the umount. Interesting. From the example
> in the man page they obviously expect it to be possible to umount the
> old root file system.
>
>>> Kill should be able to kill any process *except* init itself.
>>> And "telinit -u" should be able to refresh init.
>>
>> I did a telinit u after the chroot and killed the remaining processes.
>> After that lsof doesn't show any open files under old-root but umount
>> still fails with "device is busy".
>
> Maybe lsof doesn't necessarily list everything. What does "fuser -m
> /old_root"
> show? What about doing "fuser -k -m /old_root"?
Ok, I finally got the old root unmounted. There were some tmpfs mounted
unter old-root/dev/shm and old-root/lib/init/... - after unmounting these
I could also unmount the old-root. It's strange that neither lsof nor
fuser did show any open files for these mountpoints... I really wonder how
the pivot_root guys expect their examples to work ;)
What I still need to figure out is how I can replace init with a custom
process... but I think for that I will have to patch init itself.
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/35890.137.248.74.37...@www.mathematik.uni-marburg.de
Stephen Powell
> Stephen Powell wrote:
>>
>> /old_root"
>> show? What about doing "fuser -k -m /old_root"?
>
> Ok, I finally got the old root unmounted. There were some tmpfs mounted
> unter old-root/dev/shm and old-root/lib/init/... - after unmounting these
> I could also unmount the old-root. It's strange that neither lsof nor
> fuser did show any open files for these mountpoints... I really wonder how
> the pivot_root guys expect their examples to work ;)
Of course! Why didn't I think of that! There is nothing keeping the file
system structure from from being umounted, which is what "fuser -m" was
designed to show. But it still has to be umounted "from the bottom up."
I guess I just assumed that the shutdown scripts would umount that stuff,
but I guess since it's not a permanent file system they don't bother.
If you're shutting down or rebooting, you don't need to do that. They
remount the permanent root file system read-only, which causes the write
cache to be flushed to disk, but that's as far as they go. If you
want to do a pivot_root and umount the old root, you do actually have to
umount that stuff. I'm sorry I didn't think of it. Nice sleuthing!
>
> What I still need to figure out is how I can replace init with a custom
> process... but I think for that I will have to patch init itself.
Why do you need to patch init?
--
.''`. Stephen Powell <zlin...@wowway.com>
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1814062284.21318171269...@md01.wow.synacor.com
Katharina Haselhorst
>> process... but I think for that I will have to patch init itself.
>
> Why do you need to patch init?
I'm not sure if I really have to but I don't want to run all init
scripts of the new system right away but do some other stuff. Would be
nice to put the whole system back into the state of the early initramfs
phase. But perhaps some workaround with init is sufficient. I'll do some
research on that topic.
Thx, for your help with the pivot_root issue!
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4BAC756...@mathematik.uni-marburg.de
Stephen Powell
> Stephen Powell wrote:
>>> What I still need to figure out is how I can replace init with a custom
>>> process... but I think for that I will have to patch init itself.
>>
>> Why do you need to patch init?
>
> I'm not sure if I really have to but I don't want to run all init
> scripts of the new system right away but do some other stuff. Would be
> nice to put the whole system back into the state of the early initramfs
> phase. But perhaps some workaround with init is sufficient. I'll do some
> research on that topic.
> Thx, for your help with the pivot_root issue!
You're welcome. But really, you were the one who solved the mystery,
not me. But I'm glad you have it working. And good luck to you for the
rest of your project.
By the way, what is the big picture, if you don't mind my asking? Are you
trying to eventually achieve a "diskless workstation"? What is the
overall goal here?
--
.''`. Stephen Powell <zlin...@wowway.com>
: :' :
`. `'`
`-
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1727285811.21829111269...@md01.wow.synacor.com
bra...@mathematik.uni-marburg.de
> trying to eventually achieve a "diskless workstation"? What is the
> overall goal here?
It's a cluster where machines get booted, shutdown or replaced regularly.
So it would be nice to keep the kernel running.
K. Haselhorst
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/56601.137.248.144.89...@www.mathematik.uni-marburg.de