Bug#857597: "isolinux.bin missing or corrupt" when booting USB flash drive in old PC

[David Christensen]

syslinux at zytor.com:

I have two older computers with Intel D865GBFLK motherboards (~2003)
and Pentium 4 HT CPU's. They both have the latest available BIOS
installed. I would like to put Debian on them.


I have downloaded:


http://cdimage.debian.org/debian-cd/current/i386/iso-cd/debian-8.7.1-i386-xfce-CD-1.iso


When I put debian-8.7.1-i386-xfce-CD-1.iso on a CD-R, it boots correctly.


When I put debian-8.7.1-i386-xfce-CD-1.iso on a USB flash drive and boot
it, I see the following on the console:

isolinux.bin missing or corrupt


The same USB flash drive boots correctly in newer computers.


Memtest86+ 5.01 on a USB flash drive boots correctly in the older
computers:

http://www.memtest.org/download/5.01/memtest86+-5.01.iso.gz


My conclusion is that debian-8.7.1-i386-xfce-CD-1.iso, when put on a USB
flash drive, is not compatible with the BIOS in the older computers.


I have filed a Debian bug report, and have been advised to post here:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857597


Please advise.


TIA,

David

[Geert Stappers]

On Fri, Mar 17, 2017 at 09:51:52PM -0700, David Christensen via Syslinux wrote:
>
> I have two older computers with Intel D865GBFLK motherboards (~2003)
> and Pentium 4 HT CPU's. They both have the latest available BIOS
> installed. I would like to put Debian on them.
>

> When I put debian-8.7.1-i386-xfce-CD-1.iso on a CD-R, it boots correctly.
>

... tests with USB memory stick and memtest86 ...

>
> My conclusion is that debian-8.7.1-i386-xfce-CD-1.iso, when put on a
> USB flash drive, is not compatible with the BIOS in the older
> computers.
>
>
> I have filed a Debian bug report, and have been advised to post here:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857597
>
>
> Please advise.

Choose for either running Debian on ~14 year old hardware installed by CD-R
or booting from USB memory stick on the very same hardware.

The path of finding out what makes USB memtest boot possible
and translating it into making USB booting Debian Installer
would be the more interresting one.

My advise is to take the memtest.gz build process
and replace the "memtest kernel" by the more regular kernel (plus initrd)
of d-i.


Groeten
Geert Stappers
--
Leven en laten leven

[Thomas Schmitt]

Hi,

David Christensen wrote:
> http://cdimage.debian.org/debian-cd/current/i386/iso-cd/debian-8.7.1-i386-xfce-CD-1.iso

Background info:

This is an isohybrid made by xorriso, equivalent to a run of
isohybrid --uefi

There is a probably useless Apple Partition Map pointing to the UEFI
System Partition. The whole UEFI aspect is to be ignored by a BIOS
from 2003, i suppose.

The block address pointer from MBR to isolinux.bin looks ok.
Confirmed by xorriso analysis and by newer BIOSes which boot.


> isolinux.bin missing or corrupt

Somehow the MBR code, when run on the old BIOS, seems not to like
isolinux.bin and thus refuses to hop on it.

If i get
http://git.zytor.com/syslinux/syslinux.git/tree/mbr/isohdpfx.S#n201
right, then the error message is emitted if reading succeeds but does
not yield the expected values in bytes 64 to 67, which serve as magic
number to identify an isohybrid-ready El Torito boot image.

(I could need a reference document for the assembler language in isohdpfx.S.
Does anybody know a good one ?)


Have a nice day :)

Thomas

[Geert Stappers]

On Sun, Mar 19, 2017 at 05:18:21PM -0700, David Christensen via Syslinux wrote:
> On 03/19/2017 09:05 AM, Ady Ady via Syslinux wrote:
> > Now, besides this whole discussion and with no reference to anyone in
> > particular nor offense intended, I have to admit that I am a little bit
> > tired of answering questions related to how an ISO image either fails
> > or succeeds when booting from USB (we both have been part of these
> > discussions during the last few years) . I am very much aware that
> > maintainers of Linux distributions want to keep distributing _only_ ISO
> > images and they want to support _only_ the "simple" dd method, in spite
> > of the fact that for the most part optical media is not being used for
> > these purposes anymore. At the same time that distro maintainers want
> > to "simplify" their workload, The Syslinux Project keeps ignoring the
> > matter. Although these words could be interpreted in a negative manner
> > (not my intention), the point is about actual positive effects: let's
> > try to provide some kind of simple path to move forward, considering
> > that distro maintainers and Syslinux' developers won't.

As I see it, it is the story of four wellknown people
Everybody, Somebody, Anybody and Nobody.

There was an important job to be done and Everybody
was sure that Somebody would do it.

Anybody could have done it, but Nobody did it.

Somebody got angry about that because it was Everybody's job.

Everybody thought that Anybody could do it,
but Nobody realized that Everybody wouldn't do it.

It ended up that Everybody blamed Somebody
when Nobody did what Anybody could have done.


Being Nobody boils down into helping Somebody. It is no by-passing of
Anybody, nor telling that Everbody has the wrong expections.

It is Somebody accepting the help from Nobody.


> I used debian-8.7.1-i386-xfce-CD-1.iso because that is what Debian
> makes available.
>
>
> I have experienced problems with Debian's "isohybrid" images, USB
> flash drives, and older computers for many years. So long as the
> computer had an optical drive, I had a work-around. Now I have a
> computer without an optical drive, and I decided to try to get to
> the bottom of the problem.
>
>
> I have had better luck using "memstick.bin", etc., images produced
> by other projects (Memtest86+, FreeBSD).
>
>
> If someone knows of a Debian 8.7.1 i386 Xfce installer
> "memstick.bin" image, please provide a URL.

I do vaguely remember seeing memtest86+ in an advanced level
of a Debian boot menu. It could be on an allready installed system,
it could be on a to be installed system. I haven't checked.
Thing I'm telling: I think it is allready available.


> If someone knows of a document that explains how to build a
> "memstick.bin" image with the Debian 8.7.1 i386 Xfce installer on
> it, please provide a URL.

First build an unmodified version of the XFCE installer.
Then add the memtest option.

How to build d-i is documented at https://wiki.debian.org/DebianInstaller/Build



Groeten
Geert Stappers
--
It is meaningless what you do,
but is important that you do it.
--Mahatma Ghandi

[Thomas Schmitt]

Hi,

intermediate report:

We found a bug in isohdpfx.bin which spoils the C/H/S address conversion
for BIOSes which do not support LBA addressing.
http://www.syslinux.org/archives/2017-March/025692.html

It turned out that David Christensen's BIOS does not support INT 13 AH 41
which causes isohdpfx.bin to use C/H/S addressing.
If one forces this C/H/S addressing on qemu-system-i386, then the MBR
reads the wrong blocks because it confuses heads-per-cylinder and
sectors-per-head.

Martin Str|mberg posted a patch:
-----------------------------------------------------------------------
--- a/mbr/isohdpfx.S
+++ b/mbr/isohdpfx.S
@@ -48,8 +48,8 @@ isolinux_start_hybrid = 0x7c00+64+4
stack = 0x7c00
partoffset = (stack-8)
driveno = (stack-14)
-heads = (stack-16)
-sectors = (stack-18)
+sectors = (stack-16)
+heads = (stack-18)
ebios_flag = (stack-20)
secpercyl = (stack-24)
-----------------------------------------------------------------------
which with his qemu BIOS repairs C/H/S addressing.

Regrettably this patch did not reach David Christensen, because he
unsubscribed at sysl...@zytor.com before the bug was found.

It is not sure whether this really fixes the problem with David's BIOS
because other than qemu's BIOS it did not read a wrong block content
out of the test image but rather a block with all zeros.
(The test image has its blocks filled with their address number. All
zero is not among them, because at block 0 sits the MBR.)

I am Cc-ing David with this mail in the hope that he will test the
patch and report its outcome.

-------------------------------------------------------------------
David:

If it is the decisive trick, then patching the fixed MBR into the
first 432 bytes of the ISO should make it bootable.

If it is patched onto the start of the block_seq image on stick, then
it should report block content that is not all zero, but rather
00 00 20 FC ... 00 00 20 FC

If it does not solve the problem, then i would be interested in learning
which block content is read after the patch. I.e. in this case please
try again with Martin's diagnostic MBR with the patch applied and let
me know the diagnostic output.

[Thomas Schmitt]

Hi,

more update for Bug#857597:

Correcting the wrong stack address of sectors/head yielded progress
on David's old BIOS.
The MBR code now correctly addresses and loads the first 4 blocks of
boot image file /isolinux/isolinux.bin .

Nevertheless, the stack interface between MBR and loaded isolinux.bin
has a similar bug as the LBA-to-CHS conversion had in the MBR code.
So again, addressing fails when the code in block 0 to 3 of isolinux.bin
tries to load its blocks 4 to 79 via C/H/S addressing.

Failure symptom is this message:

ISOLINUX 6.03 20150819 CHDDisolinux: Image checksum error, sorry...

Now a new MBR change by Martin Str|mberg is ready for testing, which adapts
the PUSH stack operations of the MBR code to the POP stack operations of
the isolinux.bin code.
(This makes the previous correction unnecessary, as the old wrong
memory addresses become valid by the changed stack order.)

--------------------------------------------
diff --git a/mbr/isohdpfx.S b/mbr/isohdpfx.S
index 17e1efe..14eca14 100644
--- a/mbr/isohdpfx.S
+++ b/mbr/isohdpfx.S
@@ -151,7 +151,7 @@ next:

/* Check to see if we have EBIOS */
pushw %dx /* drive number */
- movb $0x41, %ah /* %al == 0 already */
+ movb $0x41, %ah
movw $0x55aa, %bx
xorw %cx, %cx
xorb %dh, %dh
@@ -167,20 +167,22 @@ next:
read_sector_cbios: movb $0x42, %ah ; jmp read_common */
movl $0xeb42b4+((read_common-read_sector_cbios-4) << 24), \
(read_sector_cbios)
- jmp 1f
+ jmp 2f
1:
+ xor %cx, %cx /* Clear EBIOS flag. */
+2:
popw %dx
pushw %cx /* EBIOS flag */

/* Get (C)HS geometry */
movb $0x08, %ah
int $0x13
- andw $0x3f, %cx /* Sector count */
popw %bx /* EBIOS flag */
- pushw %cx /* -16: Save sectors on the stack */
movzbw %dh, %ax /* dh = max head */
incw %ax /* From 0-based max to count */
- pushw %ax /* -18: Save heads on the stack */
+ pushw %ax /* -16: Save heads on the stack */
+ andw $0x3f, %cx /* Sector count */
+ pushw %cx /* -18: Save sectors on the stack */
mulw %cx /* Heads*sectors -> sectors per cylinder */

pushw %bx /* -20: EBIOS flag */
-----------------------------------------------------

The binary MBR code
http://www.ludd.ltu.se/~ams/tmp/isohdpfx.bin.170324
is supposed to fix C/H/S addressing in Debian ISOs if copied to their
first block:

iso=debian-8.7.1-i386-xfce-CD-1.iso

dd conv=notrunc bs=1 count=432 if=isohdpfx.bin.170324 of="$iso"

The ISO then still boots for me by -hda of qemu-system-i386. But that's
not a C/H/S needy BIOS.

So please cross fingers everybody ...

-------------------------------------------------------------------------

The all-zero block content read by the buggy address conversion on David's
BIOS is explainable by the two possible disk geometries of the now known
conversion 8444 -> 1/6/3:
H/C= 128 S/H= 63
H/C= 195 S/H= 42
In both cases the wrongly computed sector numbers of 1/2/125 or 1/1/60
exceed the respective sectors/head parameters.
(Conversion hand computed. Bear with me.)

-------------------------------------------------------------------------

[Thomas Schmitt]

Hi,

David Christensen confirmed that above fixed MBR enabled isohybrid
booting on his legacy system.

Time for a summary of findings.

------------------------------------------------------------
How to reproduce the bug:

Any BIOS which does not announce "Device Access using the packet structure"
by the reply to INT 13 AH 41 will fail to boot by the old MBR, if the numbers
for heads/cylinder and for sectors/head do not happen to be identical.

The drive geometry numbers are handed out by the BIOS on INT 13 AH 8
depending on its perception of the drive. I never saw both being the same.
See also:
https://en.wikipedia.org/wiki/INT_13H#INT_13h_AH.3D41h:_Check_Extensions_Present
https://en.wikipedia.org/wiki/INT_13H#INT_13h_AH.3D41h:_Check_Extensions_Present

------------------------------------------------------------
How to work around:

Overwrite the old MBR by the fixed one.

wget http://www.ludd.ltu.se/~ams/tmp/isohdpfx.bin.170324

iso=debian-8.7.1-i386-xfce-CD-1.iso
or
iso=/dev/sdc

dd conv=notrunc bs=1 count=432 if=isohdpfx.bin.170324 of="$iso"

According to
https://git.kernel.org/pub/scm/boot/syslinux/syslinux.git/log/mbr/isohdpfx.S
the fix should apply to ISOLINUX versions since 3.82.
The most recent touching of the code part in question was on 2009-05-31.

------------------------------------------------------------
Proposals for debian-cd and syslinux package:

Martin Str|mberg has announced that
http://www.ludd.ltu.se/~ams/tmp/isohdpfx.bin.170324
will exist only a limited time. (I have a copy.)

I propose that Debian gets it too and offers it for download together
with a description what problem it might fix and how to apply it.
If it gets a Debian URL and if i get pointed to an empty wiki page,
i would volunteer to write the description.

Next, one will have to decide whether to use the fixed MBR with future
Debian ISOs and whether to put the patch into Debian's syslinux source
package.

We got no comment by hpa yet. Any review by a skilled assembler
programmer is welcome. Especially one will have to check for any
inadverted side effects caused by changing the stack push sequence
to the expectations of isolinux.bin.

[Thomas Schmitt]

Hi,

the following patch has been committed to SYSLINUX to fix the C/H/S
addressing problems in isohybrid MBR and isolinux.bin :

http://git.zytor.com/syslinux/syslinux.git/commit/?id=32c09027423f61c305e2423e52f5f69ecad8e2c0
diff --git a/mbr/isohdpfx.S b/mbr/isohdpfx.S
index 17e1efe..f9e9691 100644
--- a/mbr/isohdpfx.S
+++ b/mbr/isohdpfx.S
@@ -175,12 +175,12 @@ next:

/* Get (C)HS geometry */
movb $0x08, %ah
int $0x13
- andw $0x3f, %cx /* Sector count */
popw %bx /* EBIOS flag */
- pushw %cx /* -16: Save sectors on the stack */
movzbw %dh, %ax /* dh = max head */
incw %ax /* From 0-based max to count */
- pushw %ax /* -18: Save heads on the stack */
+ pushw %ax /* -16: Save heads on the stack */
+ andw $0x3f, %cx /* Sector count */
+ pushw %cx /* -18: Save sectors on the stack */
mulw %cx /* Heads*sectors -> sectors per cylinder */
pushw %bx /* -20: EBIOS flag */

The other part of the changeset proposed on 25 Mar 2017 is now committed as
http://git.zytor.com/syslinux/syslinux.git/commit/?id=8739e2ff9ba3f92652c8df846924fd00e1ce2753
It fixes the problem that an early decision against LBA addressing after
INT 13 AH 41 could push a non-zero CX value to the stack, which
isolinux.bin would pop and interpret as instruction to use LBA addressing.
(isolinux.bin is wrong here. It should only use LBA if bit 0 of the
popped CX is set. But even that could be wrongly sent from MBR if not
the "Clear EBIOS flag" fix is applied.)