Making a zpool bootable on an x86 system
Tim Bradshaw
which is otherwise properly set up to be a root filesystem for an x66
system, will arrange for it to be actually bootable?
The context is that we have a system which had a UFS root filesystem,
we've done a live upgrade to ZFS, and either mistakes have been made or
LU doesn't work right, or something, so it will not now boot. It is
(was) running 10u7. If you boot from the 10u7 DVD and choosing the
single-user shell option it will find the zpool itself and mount it on
/a as you'd expect.
Things we've done:
* checked the fdisk stuff looks OK (both disks have a singe solaris
fdisk partition, which is active);
* installgrub on each disk;
* checked the menu.lst file (we are not above completely typing this in
from scratch, but it looks OK);
* updated the boot archive via bootadm update-archive -R /a
It does one of several things (I've not been closely involved in this
saga so I'm not sure which it does when):
* displays "GRUB" and freezes (I guess this is stage1 failing to find
stage2 but not knowing it has?);
* displays "grub> " from where we could may be work out how to proceed
but actually can't due to stupidity or tiredness (I think this must be
stage2 loading but not finding a menu?);
* displays a boot menu which looks plausible but which then complains
about paths being wrong.
If we could reliably get it to do the last one we could fix it, but it
does not seem to reliably get that far.
My feeling is that there must be some concrete set of steps to do,
involving installing bootblocks and configuring the menu file which
will make the machine bootable, assuming the zpool is OK, which I think
it is, but I don't know what it is.
We do have a case open with Sun, but so far they don't know what the
set of steps is either.
Every time I get annoyed at SPARC systems, I just have to remind myself
about x86 booting and I realise how bad things could (and will shortly)
be.
Thanks
--tim
Richard B. Gilbert
Well, I hope you have a recent and good backup! Something is very wrong
here but figuring out what is not easy from a distance.
tim....@gmail.com
did you confirm that the disks have SMI as opposed to EFI labels?
(format -e -> disk -> label)
Ian Collins
> Does anyone have a set of steps which, given a zpool (2 mirrored disks)
> which is otherwise properly set up to be a root filesystem for an x66
> system, will arrange for it to be actually bootable?
What's the output of "zpool status"? There might be some clues there as
to why it won't boot.
> The context is that we have a system which had a UFS root filesystem,
> we've done a live upgrade to ZFS, and either mistakes have been made or
> LU doesn't work right, or something, so it will not now boot. It is
> (was) running 10u7. If you boot from the 10u7 DVD and choosing the
> single-user shell option it will find the zpool itself and mount it on
> /a as you'd expect.
>
<snip>
>
> My feeling is that there must be some concrete set of steps to do,
> involving installing bootblocks and configuring the menu file which will
> make the machine bootable, assuming the zpool is OK, which I think it
> is, but I don't know what it is.
If you followed the documented path, it should have just worked. I've
migrated several x86 systems from UFS to ZFS boot without any hair pulling.
--
Ian Collins
Tim Bradshaw
<tim....@gmail.com> said:
> did you confirm that the disks have SMI as opposed to EFI labels?
> (format -e -> disk -> label)
Yes, they do. the pool is in the s0s of them.
Tim Bradshaw
> What's the output of "zpool status"? There might be some clues there
> as to why it won't boot.
I can't cut & paste it, due to being on the wrong machine, but it's
fine. I did a scrub in fact just in case there was hidden nastiness in
there and it passes.
> If you followed the documented path, it should have just worked. I've
> migrated several x86 systems from UFS to ZFS boot without any hair
> pulling.
I didn't actually do the upgrade (I'm helping out later) but I think it
was close to the documented path. One thing is that there was a
*previous* LU (UFS->UFS, and I think from 10u6 to 10u6 though the
source may have been older than u6) where LU told us to to various
GRUB-related things as there have clearly been changes in the way GRUB
works. I'm wondering if that left some latent toxins in there.
I've also done various LUs on x86 systems and it has always worked
fine. But that's kind of the way with x86: it all works fine for
months then some appalling tentacled horror from about 1980 crawls out
of the cracks and slowly digests you in its ichor.
Dave
> I didn't actually do the upgrade (I'm helping out later) but I think it
> was close to the documented path. One thing is that there was a
> *previous* LU (UFS->UFS, and I think from 10u6 to 10u6 though the source
> may have been older than u6) where LU told us to to various GRUB-related
> things as there have clearly been changes in the way GRUB works. I'm
> wondering if that left some latent toxins in there.
It might be worth doing an update to u7, and hoping it clears out any toxins.
I recently set my Open Solaris machine so the 500 GB ZFS boot disk is mirrored,
and I've confirmed it will boot from either disk (i.e. I physically removed one,
and found it will boot with either). But I can't recall too much about the exact
procedure, other than following the docs I found, so can't be too helpful
myself. But an upgrade might just clear the problem. At least it will give you
some clues what the problem is not.
--
I respectfully request that this message is not archived by companies as
unscrupulous as 'Experts Exchange' . In case you are unaware,
'Experts Exchange' take questions posted on the web and try to find
idiots stupid enough to pay for the answers, which were posted freely
by others. They are leeches.
Tim Bradshaw
> It might be worth doing an update to u7, and hoping it clears out any toxins.
Sorry, my post was wrong. We went from U6 (I think) to U7 (I'm sure).
We can't go to anything newer than U7 unfortunately.
Casper H.S. Dik
>Does anyone have a set of steps which, given a zpool (2 mirrored disks)
>which is otherwise properly set up to be a root filesystem for an x66
>system, will arrange for it to be actually bootable?
>The context is that we have a system which had a UFS root filesystem,
>we've done a live upgrade to ZFS, and either mistakes have been made or
>LU doesn't work right, or something, so it will not now boot. It is
>(was) running 10u7. If you boot from the 10u7 DVD and choosing the
>single-user shell option it will find the zpool itself and mount it on
>/a as you'd expect.
Can you still boot from 10u7? Can you luactivate then?
It worked for me going from 2 ufs lu environment, luupgraded to ZFS
and then add the other ufs lu environment as the mirror for the ZFS
rpool.
But this was under Soalris Nevada (I think I was running build 103
at the time, or so says zpool history)
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
Tim Bradshaw
> It worked for me going from 2 ufs lu environment, luupgraded to ZFS
> and then add the other ufs lu environment as the mirror for the ZFS
> rpool.
We worked it out (actually a Sun support person did), and it's a bit
silly. We were following a process which worked for SPARC but does not
work for x86. Essentially the process was:
* split UFS mirror
* repartition split off disk to have a big s0 starting at block 0 (MISTAKE)
* luupgrade onto a zpool on that s0.
The problem is that x86 doesn't work like that: there's this magic s8,
which is one cylinder (or however many blocks it thinks is a cylinder),
and s0 must start on cylinder 1. Clearly bits of GRUB live in s8, and
if s0 overlaps it bad things happen. I am not sure why ZFS did not
complain, because it must have been finding bits of itself being
clobbered when we did the installgrub, but may be it does not care
about those bits.
This explains a lot of the mysteriosity - we had it in a state where we
could boot from DVD, do installgrub, boot from the disks but then it
would not reboot (because GRUB had been eaten).
So, I think I know what to do now, which is to split mirrors again,
reslice the now-unused boot disk to be right, make a zpool on it, and
then do a zfs send -R | zfs receive to clone the old pool onto it.
(Alternatively we may fall back to reinstalling from a flar, which we
have).
Thanks to everyone.