booting from 200Gig disks
Wolfgang S. Rupprecht
I had installed openbsd 3.5 (-current ???) on an amd64 system (asus
k8v-deluxe-se, athlon64 3200) with a single 200Gig disk on it.
(OpenBSD bonnet.wsrcc.com 3.5 GENERIC#2 amd64) Things mostly work and
I'm writing this note from that system.
The one thing that doesn't work at all is booting directly with the
bootblocks located on the disk. Booting hangs in instant after the
second stage loader prints out its banner. The ascii "/|\-" spinner
is shown frozen on the screen with the cursor just behind it. Every
time I reboot I have to insert the install CD, interrupt its
count-down and tell it to load the kernel from disk. That works just
fine. Whenever I can manage to interrupt the countdown in time, that
is, which is only half the time.
After much head-scratching and pouring over the code in biosboot.S and
related files in /usr/src/sys/arch/amd64/stand/ it looks to me like
every stage of booting is limited to using the short LBA mode
(eg. non-LBA48). That LBA code is only good for reaching sectors
within the first ~130 Gig of the disk. I've got 70 more gigs, and I
guess I'm unlucky or something because some of /bsd must be located
there. When I copy my whole disk onto a smaller disk using pax and
then running installboot again, I get a disk that boots just fine.
(Just to stave off the question, no, running installboot on the big
disk doesn't change anything.) It certainly looks like a problem that
only happens on big disks.
The part that is sad is that the loaders (especially the second stage
where there isn't a space crunch) don't appear to gripe about an out
of range sector and silently go substituting the aliased sector from
under 130G. That gripe could have saved me much time.
I have tried to Google for the bios calls that access the disk via the
LBA-48, but come up empty. Does anyone here by any chance know this
kind of bios arcana? I'd love to try to whack that into my second
stage loader, which is the one that is effecting me. Catching the
boot and typing something before the timout happens is getting old.
Really, really, old.
-wolfgang
--
Wolfgang S. Rupprecht http://www.wsrcc.com/wolfgang/
The above "From:" address is valid. Don't mess with it.
Theo de Raadt
> k8v-deluxe-se, athlon64 3200) with a single 200Gig disk on it.
> (OpenBSD bonnet.wsrcc.com 3.5 GENERIC#2 amd64) Things mostly work and
> I'm writing this note from that system.
OK hang on here. You made a 200GB root partition?
Meaning that your /var and /usr and /tmp and everything is in the
same partition?
Meaning you do not get the following benefits:
/dev/sd0a on / type ffs (local)
/dev/sd0d on /tmp type ffs (local, nodev, nosuid)
/dev/sd0g on /usr type ffs (local, nodev)
/dev/sd0h on /usr/X11R6 type ffs (local, nodev)
/dev/sd0i on /usr/local type ffs (local, nodev)
/dev/sd0j on /usr/obj type ffs (local, nodev, nosuid)
/dev/sd0e on /var type ffs (local, nodev, nosuid)
/dev/sd0m on /home type ffs (local, nodev, nosuid)
Look at those nosuid and nodev flags.
Wolfgang S. Rupprecht
> OK hang on here. You made a 200GB root partition?
Yes. It is an experiment to see if one partition per spindle is
faster due to the different strategy routines not tripping over each
other. I've had < 50Meg /'s with a few hundred megs for /var and /usr
up to now. If I don't like it I can easily go back to a traditional
partition layout.
Even if you don't agree with my disk layout, the problem would still
exist if someone had a 140Gig windows partition in fdisk partition 0,
occupying the low-end of the disk and tried to install OpenBSD above
it.
> Look at those nosuid and nodev flags.
Well, it would be nice, but I'm not going to lose any sleep over it at
this point. I imagine if I really cared about such things I would
probably want to load pathnames into the kernel of the few directories
I wanted to allow suid programs and device node in.
-wolfgang
--
Wolfgang S. Rupprecht <wolf...@wsrcc.com> http://www.wsrcc.com/wolfgang/
Ted Unangst
> Yes. It is an experiment to see if one partition per spindle is
> faster due to the different strategy routines not tripping over each
> other. I've had < 50Meg /'s with a few hundred megs for /var and /usr
> up to now. If I don't like it I can easily go back to a traditional
> partition layout.
there is only strategy routine per disk.
--
you are more than the sum of what you consume
Tom Cosgrove
>
> I'll try to make this short, unlike the debugging that got me here.
>
> k8v-deluxe-se, athlon64 3200) with a single 200Gig disk on it.
> (OpenBSD bonnet.wsrcc.com 3.5 GENERIC#2 amd64) Things mostly work and
> I'm writing this note from that system.
>
> bootblocks located on the disk. Booting hangs in instant after
> the second stage loader prints out its banner. The ascii "/|\-"
> spinner is shown frozen on the screen with the cursor just behind it.
> Every time I reboot I have to insert the install CD, interrupt its
> count-down and tell it to load the kernel from disk. That works just
> fine. Whenever I can manage to interrupt the countdown in time, that
> is, which is only half the time.
I find this strange.
Are you saying that is freezes at this point?
probing: pc0 com0 com1 mem[634K 127M a20=on]
disk: fd0 hd0+
>> OpenBSD/amd64 BOOT 2.06
boot>
booting hd0a:bsd: \
But it then works if you use the /boot from the CD? And tell it to use
the kernel on disk, the same kernel that the disk boot couldn't load?
First of all, you could always create a CD with "boot hd0a:/bsd" in
an /etc/boot.conf file in the boot filesystem.
You need to do something like
$ sudo vnconfig svnd0 ./cdrom35.fs
$ sudo mount -t ffs /dev/svnd0c /mnt
$ ls -l /mnt
total 3584
-rw-r--r-- 1 root wheel 53248 Apr 4 03:00 boot
-rw-r--r-- 1 root wheel 1771791 Apr 4 03:00 bsd
$ sudo mkdir /mnt/etc
$ sudo sh -c "echo boot hd0a:/bsd > /mnt/etc/boot.conf"
$ ls -l /mnt/etc
total 1
-rw-r--r-- 1 root wheel 12 Apr 5 14:00 boot.conf
$ cat /mnt/etc/boot.conf
boot hd0a:/bsd
$ sudo umount /mnt
$ sudo vnconfig -u svnd0
> After much head-scratching and pouring over the code in biosboot.S
> and related files in /usr/src/sys/arch/amd64/stand/ it looks to me
> like every stage of booting is limited to using the short LBA mode
> (eg. non-LBA48).
Can you explain precisely why you think that?
> That LBA code is only good for reaching sectors
> within the first ~130 Gig of the disk. I've got 70 more gigs, and I
> guess I'm unlucky or something because some of /bsd must be located
> there. When I copy my whole disk onto a smaller disk using pax and
> then running installboot again, I get a disk that boots just fine.
> (Just to stave off the question, no, running installboot on the big
> disk doesn't change anything.) It certainly looks like a problem that
> only happens on big disks.
>
> The part that is sad is that the loaders (especially the second stage
> where there isn't a space crunch) don't appear to gripe about an out
> of range sector and silently go substituting the aliased sector from
> under 130G. That gripe could have saved me much time.
Obviously the problem is not in the second-stage loader, since the one
on the CD manages to load the kernel on the disk.
Further, if you look at sys/arch/amd64/stand/libsa/biosdev.c:EDD_rw()
you will see that the sector number passed in is u_int64_t daddr,
which is passed in its entirety to the BIOS call.
> I have tried to Google for the bios calls that access the disk via the
> LBA-48, but come up empty.
Not surprisingly, since the BIOS calls are "limited" to 64 bits of
sector number, which is more than either of the 28-bit or 48-bit LBA
sizes.
> Does anyone here by any chance know this
> kind of bios arcana?
Heh :-)
> I'd love to try to whack that into my second
> stage loader, which is the one that is effecting me. Catching the
> boot and typing something before the timout happens is getting old.
> Really, really, old.
The problem seems to be in biosboot, not in the second-stage boot
loader. If you send through the details I asked for in my earlier
email, we might have a chance of finding out where or why things are
stalling.
However, a quick fix would be to create a modified boot CD, as above.
Thanks
Tom
Wolfgang S. Rupprecht
> Are you saying that is freezes at this point?
>
> probing: pc0 com0 com1 mem[634K 127M a20=on]
> disk: fd0 hd0+
> >> OpenBSD/amd64 BOOT 2.06
> boot>
> booting hd0a:bsd: \
Typed in from a berry-juice-on-papyrus note I took during booting:
Using drive 0, partition 3.
Loading ...
Probing: pc0 com0 com1 mem[639k 1022M a20=on]
disk: fd0 hd0+
>> OpenBSD/amd64 BOOT 2.06
/
The cursor is directly under the last "/". Note there isn't a "boot>"
or a "booting hd0a:bsd: ".
> First of all, you could always create a CD with "boot hd0a:/bsd" in
> an /etc/boot.conf file in the boot filesystem.
Neat! I didn't realize it could read a conf file. Will do.
Checking I now see that my disk had a /etc/boot.conf and the "set"
line was empty. Moving the file to a new name had no effect.
> The problem seems to be in biosboot, not in the second-stage boot
> loader. If you send through the details I asked for in my earlier
> email, we might have a chance of finding out where or why things are
> stalling.
I sent them off to mi...@openbsd.org.
-wolfgang