BTX loader hangs after version info
James Seward
Two days ago I csup'd my desktop at home, which was running RELENG_7
from about 7.0-RELEASE time, to bring it up-to-date (still on
RELENG_7). I followed my usual buildkernel/world procedure (the usual
one) which has worked fine all the way since 5.x. After installing
kernel and restarting in single user, it was working fine. However,
following installworld it will not boot.
It stops immediately after "BTX loader 1.00 BTX version 1.02", but
with the cursor on the line *above* the first "B". Nothing futher
happens, but the system responds to Ctrl-Alt-Del.
I have managed to start it using the install CD and csup'd back to a
version just before the commit to BTX that moved it to 1.02 (March
18th, I think). However, that version too hangs after "BTX loader 1.00
BTX version 1.01".
My desktop is currently building RELENG_7_0 to see if that will work,
but I won't know that until later as I'm at work and it is at home :)
The install CD (BTX 1.00/1.01) boots fine. Nothing else changed on my
system between the last successful boot and the unsuccessful one.
Any suggestions/advice for what I can try next, or what I can do to
help the troubleshooting process?
My desktop is an Athlon64 but I am using i386, on an Asus A8V-E Deluxe board.
/JMS
Jeremy Chadwick
> Two days ago I csup'd my desktop at home, which was running RELENG_7
> from about 7.0-RELEASE time, to bring it up-to-date (still on
> RELENG_7). I followed my usual buildkernel/world procedure (the usual
> one) which has worked fine all the way since 5.x. After installing
> kernel and restarting in single user, it was working fine. However,
> following installworld it will not boot.
>
> It stops immediately after "BTX loader 1.00 BTX version 1.02", but
> with the cursor on the line *above* the first "B". Nothing futher
> happens, but the system responds to Ctrl-Alt-Del.
This sounds like a regression in the recent BTX fixes which were done by
John Baldwin on March 18th. Strange, since those fixes addressed
booting issues lots of users were having with BTX. See the very bottom
of the below page, "BTX crashes on start":
http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues
After installworld, did you happen to use bsdlabel -B?
> Any suggestions/advice for what I can try next, or what I can do to
> help the troubleshooting process?
>
> My desktop is an Athlon64 but I am using i386, on an Asus A8V-E Deluxe board.
I've CC'd John here, who might have some ideas.
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
James Seward
> http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues
My problem doesn't match the description of "screen continually
scrolls registers or dumps registers then reboots"; it just freezes.
While looking at the code for btx/btxldr I did notice a debug knob in
the Makefile; should I turn this on? (And do I have to rebuild all of
world to encourage it to update BTX? Presumably I can build/install a
subset of it to save time?)
> After installworld, did you happen to use bsdlabel -B?
No, my procedure was: make buildkernel buildworld, make installkernel,
reboot (to single user), mergemaster -p, make installworld,
mergemaster (installed everything but passwd/group), reboot. This is
the point where it broke :)
Thanks,
James
Jeremy Chadwick
> On Thu, May 22, 2008 at 1:24 PM, Jeremy Chadwick <koi...@freebsd.org> wrote:
> > http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues
>
> My problem doesn't match the description of "screen continually
> scrolls registers or dumps registers then reboots"; it just freezes.
>
> While looking at the code for btx/btxldr I did notice a debug knob in
> the Makefile; should I turn this on?
Probably not.
> (And do I have to rebuild all of
> world to encourage it to update BTX? Presumably I can build/install a
> subset of it to save time?)
cd /sys/boot && make clean && make && make install will build and
install new boot blocks in /boot, as well as /boot/loader. It will not
apply new boot blocks to your disk (that's what bsdlabel does).
> > After installworld, did you happen to use bsdlabel -B?
>
> No, my procedure was: make buildkernel buildworld, make installkernel,
> reboot (to single user), mergemaster -p, make installworld,
> mergemaster (installed everything but passwd/group), reboot. This is
> the point where it broke :)
My bad, sorry.
The problem you're experiencing is likely in /boot/loader, which is what
prints the "BTX version is x.xx" message. /boot/loader doesn't require
your boot blocks be updated; it's updated during installworld.
BTX version 1.01 is what comes with 7.0-RELEASE, while a RELENG_7
snapshot or a recently-rebuilt world (of RELENG_7) would use 1.02.
I'm not sure what's breaking there for you; I'll have to dig through the
CVS commit logs to check. Do you have anything in /boot/loader.conf?
Mark Kirkwood
28-Feb is fine, but Mar, Apr, May code all hangs after printing "loading
/boot/defaults/loader.conf" - presumably reading my /boot/loader.conf?
Interestingly I can usually get it to boot by escaping to the loader
prompt and then just pressing return.
Oddly some other machines (Supermicro P3TDER and Asus PRO31J Laptop)
behave normally with src from Mar->May.
In all cases the canonical procedure from UPDATING was used (buildworld,
kernel, reboot single, mergemaster -p, installworld, delete-old,
mergemaster, reboot).
I happy to help collect some debug info (how do you switch this on for
the loader?), tho the machine exhibiting the problem is my workstation
(of course)!
regards
Mark
John Baldwin
I would be wary of that as it might make things worse? These problems are all
from starting /boot/loader. boot2 is still working fine and thus there is
still the possiblity of using boot2 to load /boot/loader.old as a workaround.
If you update boot2 and it breaks you can't fix that w/o booting off of some
other media such as a CD.
Debugging these hangs is not easy to do remotely. If you know assembly then
there are some things you can play with. For example, in the case where it
hangs after printing out the BTX version (from btxldr.S) you could start
adding debugging to btx.S to print out '.' characters in various places and
see how many get printed out before it hangs. However, doing this requires
familiarity with assembly and is a lot easier with physical access to a box.
--
John Baldwin
James Seward
> If you update boot2 and it breaks you can't fix that w/o booting off of some
> other media such as a CD.
This is where I'm at anyway :)
Oh, I forgot to mention I'm booting with grub (which then hands over
to BTX if I choose my FreeBSD option). I guess I should try getting
rid of that temporarily.
Jeremy, I do have a couple of lines in loader.conf but off hand I
can't remember what precisely. I think it's a DMA hint, and disabling
AGP (a solution I think sos@ came up with for problem I had some time
during 6.2's release cycle). Unfortunately I was busy last night so
couldn't try anything further, but I should be able to tonight and
over the weekend.
Thanks,
James
John Baldwin
We could get a hang if a vm86 request was ever done with interrupts disabled
for some reason. I'll see if I can add some assertions to make BTX
explicitly panic if that happens.
--
John Baldwin
John Baldwin
Try this patch. I'm not 100% certain this will fix it as I can't reproduce
the issue, but I think it might help. Specifically, when the boot code makes
a v86 call, the loader/boot2/whatever swaps in/out a new set of registers via
the v86 structure including the eflags register. However, none of the boot
programs actually initialized the v86 structure. Thus, the BIOS routines
would start off running with whatever garbage was in v86.efl when each boot
program started. This meant that we could end up invoking BIOS routines with
interrupts disabled, and I think this might explain a hard hang (if a BIOS
routine was waiting for an interrupt the interrupt would never fire). The
patch fixes all the boot programs to initialize v86 to a better known state.
At the least it sets v86.efl to a sane value (0x202) rather than random. (The
random might have always been 0x0 BTW, not sure on that one.)
--- //depot/vendor/freebsd/src/sys/boot/i386/boot2/boot2.c 2008/02/28 17:10:57
+++ //depot/user/jhb/boot/sys/boot/i386/boot2/boot2.c 2008/05/23 21:59:59
@@ -24,6 +24,7 @@
#include <machine/bootinfo.h>
#include <machine/elf.h>
+#include <machine/psl.h>
#include <stdarg.h>
@@ -83,8 +84,8 @@
#define NDEV 3
#define MEM_BASE 0x12
#define MEM_EXT 0x15
-#define V86_CY(x) ((x) & 1)
-#define V86_ZR(x) ((x) & 0x40)
+#define V86_CY(x) ((x) & PSL_C)
+#define V86_ZR(x) ((x) & PSL_Z)
#define DRV_HARD 0x80
#define DRV_MASK 0x7f
@@ -237,6 +238,7 @@
dmadat = (void *)(roundup2(__base + (int32_t)&_end, 0x10000) - __base);
v86.ctl = V86_FLAGS;
+ v86.efl = PSL_RESERVED_DEFAULT | PSL_I;
dsk.drive = *(uint8_t *)PTOV(ARGS);
dsk.type = dsk.drive & DRV_HARD ? TYPE_AD : TYPE_FD;
dsk.unit = dsk.drive & DRV_MASK;
--- //depot/vendor/freebsd/src/sys/boot/i386/gptboot/gptboot.c 2008/02/28 17:10:57
+++ //depot/user/jhb/boot/sys/boot/i386/gptboot/gptboot.c 2008/05/23 21:59:59
@@ -23,6 +23,7 @@
#include <machine/bootinfo.h>
#include <machine/elf.h>
+#include <machine/psl.h>
#include <stdarg.h>
@@ -81,8 +82,8 @@
#define NDEV 3
#define MEM_BASE 0x12
#define MEM_EXT 0x15
-#define V86_CY(x) ((x) & 1)
-#define V86_ZR(x) ((x) & 0x40)
+#define V86_CY(x) ((x) & PSL_C)
+#define V86_ZR(x) ((x) & PSL_Z)
#define DRV_HARD 0x80
#define DRV_MASK 0x7f
@@ -235,6 +236,7 @@
dmadat = (void *)(roundup2(__base + (int32_t)&_end, 0x10000) - __base);
v86.ctl = V86_FLAGS;
+ v86.efl = PSL_RESERVED_DEFAULT | PSL_I;
dsk.drive = *(uint8_t *)PTOV(ARGS);
dsk.type = dsk.drive & DRV_HARD ? TYPE_AD : TYPE_FD;
dsk.unit = dsk.drive & DRV_MASK;
--- //depot/vendor/freebsd/src/sys/boot/i386/loader/main.c 2007/10/24 04:07:14
+++ //depot/user/jhb/boot/sys/boot/i386/loader/main.c 2008/05/23 21:59:59
@@ -35,6 +35,7 @@
#include <stand.h>
#include <string.h>
#include <machine/bootinfo.h>
+#include <machine/psl.h>
#include <sys/reboot.h>
#include "bootstrap.h"
@@ -86,6 +87,10 @@
initial_bootdev = kargs->bootdev;
initial_bootinfo = kargs->bootinfo ? (struct bootinfo *)PTOV(kargs->bootinfo) : NULL;
+ /* Initialize the v86 register set to a known-good state. */
+ bzero(&v86, sizeof(v86));
+ v86.efl = PSL_RESERVED_DEFAULT | PSL_I;
+
/*
* Initialise the heap as early as possible. Once this is done, malloc() is usable.
*/
--- //depot/vendor/freebsd/src/sys/boot/pc98/loader/main.c 2007/10/24 11:57:58
+++ //depot/user/jhb/boot/sys/boot/pc98/loader/main.c 2008/05/23 22:03:45
@@ -35,6 +35,7 @@
#include <stand.h>
#include <string.h>
#include <machine/bootinfo.h>
+#include <machine/psl.h>
#include <sys/reboot.h>
#include "bootstrap.h"
@@ -86,6 +87,10 @@
initial_bootdev = kargs->bootdev;
initial_bootinfo = kargs->bootinfo ? (struct bootinfo *)PTOV(kargs->bootinfo) : NULL;
+ /* Initialize the v86 register set to a known-good state. */
+ bzero(&v86, sizeof(v86));
+ v86.efl = PSL_RESERVED_DEFAULT | PSL_I;
+
/*
* Initialise the heap as early as possible. Once this is done, malloc() is usable.
*/
--
John Baldwin
Peter Holm
I can confirm that this patch fixes the loader problem seen with
my old Tyan S2720 MB.
- Peter
James Seward
> [patch]
I have not yet tried this patch, but here is my progress so far:
* Rolling back to 7.0-RELEASE made it worse - now after choosing
FreeBSD from grub the machine would immediately reboot.
* I booted with the CD again and came forward to RELENG_7. This still
caused an immediate reboot.
* I booted with the CD and replaced grub with "boot0cfg -B" (bsdlabel,
which I tried earlier, had no effect). This fixed it, mostly.
My desktop now boots fine and completes normal startup. It even starts
X. However, from the moment I press F1 to choose FreeBSD to the moment
it's done booting, all I see on the screen is "BTX loader 1.00 BTX
version is 1.02" and the cursor blinking below it. My only clues that
it's booting at the time are disk activity, and the cursor changing
from a flashing underline to a solid block (which it always has as the
kernel proper takes over).
The next thing I see is "FreeBSD/i386 (hostname.goes.here) (ttyv0)". I
have also noticed that I don't get any output when I restart - no rc.d
scripts telling me things are stopping, no vnodes syncing, nothing.
It's apparently happening though, as everything is
starting/stopping/umounting correctly.
I don't see that I have anything configured to boot to serial rather
than the console, but is there anything I can do/check to make sure
this definitely isn't happening? Unfortunately, putting a cable on the
serial port is not an option as I currently do not own one suitable :)
Thanks,
James
Eugene Grosbein
> X. However, from the moment I press F1 to choose FreeBSD to the moment
> it's done booting, all I see on the screen is "BTX loader 1.00 BTX
> version is 1.02" and the cursor blinking below it. My only clues that
> it's booting at the time are disk activity, and the cursor changing
> from a flashing underline to a solid block (which it always has as the
> kernel proper takes over).
It seems you have /boot/device.hints missing or broken.
Just do "cp /usr/src/sys/i386/conf/GENERIC.hints /boot/device.hints".
Eugene Grosbein
Mark Kirkwood
>
> Try this patch. I'm not 100% certain this will fix it as I can't reproduce
> the issue, but I think it might help. Specifically, when the boot code makes
> a v86 call, the loader/boot2/whatever swaps in/out a new set of registers via
> the v86 structure including the eflags register. However, none of the boot
> programs actually initialized the v86 structure. Thus, the BIOS routines
> would start off running with whatever garbage was in v86.efl when each boot
> program started. This meant that we could end up invoking BIOS routines with
> interrupts disabled, and I think this might explain a hard hang (if a BIOS
> routine was waiting for an interrupt the interrupt would never fire). The
> patch fixes all the boot programs to initialize v86 to a better known state.
> At the least it sets v86.efl to a sane value (0x202) rather than random. (The
> random might have always been 0x0 BTW, not sure on that one.)
>
>
Unfortunately this patch does *not* cure the issue for my old Supermicro
P3TDDE, it still hangs just before presenting the menu. I had to boot
off the livefs and copy /boot/loader.old -> /boot/loader to get back to
being bootable again - but at least the old fella is on a more
up-to-date 7-STABLE now :-)
Cheers
Mark
Mark Kirkwood
Given that the patch *did* cure Peters Tyan S2720, I'll double check I
didn't fat finger applying the patch (mind you the Tyan has AMI BOIS -
same as my Supermicro P3TDERs that *do* work ok with current 7-STABLE,
whereas the P3TDDE has Award BIOS).
Anyway, I'll double check and report back...
Cheers
Mark
Peter Holm
I now have booted some more with this patch and it would seem that the
problem is still there, from time to time! Most of the time I now boot
without any problems but once in a while the loader still hangs.
I'll try to gather some statistics...
--
Peter
Peter Holm
I did 18 boots with and with out John's patch. With the patch I got 6
actual boots and 12 hangs in the loaders progress bar.
Without the patch I got 10 boots and 8 hangs.
But, my Tyan M/B is old and with known ACPI issues so I'm not sure if
this is of much value.
Mark, it would be nice if you also observe if a sequence of reboots
eventually boots your system. My longest bad streek was 8 reboots.
- Peter
Mark Kirkwood
correctly applied. I'll report back in several reboots :-)
Mark
Mark Kirkwood
Yeah, I see the same thing - with John's patch applied, out of 9 reboots
I got 7 hangs and 2 actual boots. (didn't try without the patch).
Mark
Peter Holm
OK, that is at least nice with consistency across different HW.
--
Peter