Info on the BIOS requirements for USB-HDD, USB-FDD, and USB-ZIP booting.

[Rod Pemberton]

I've found some info on the BIOS requirements for USB-HDD, USB-FDD,
USB-ZIP booting.

I'd love to find a formal standards document which describes what the
pdf explains (link below). Apparently, there is a specification for
USB-ZIP somewhere that specifies what is needed for USB-ZIP booting.

http://soft.goodmc.ru/CD&DVD/WinSetupFromUSB-1-7/files/tools/RMPrepUSB/LANG/RMPrepUSB.pdf

There are also some descriptions with some additional information of
what each option for what their USB formatting program does on their
website:

https://www.rmprepusb.com/


A quick summary:

USB-FDD is DL=0, no MBR, no partition, apparently needs a correct
device name, e.g., manufacturer's name of a physical floppy device

USB-ZIP is DL=0, one partition, boots VBR directly, skips boot code,
63 head/32 sectors preferred, translates disk calls from start of the
partition which hides data prior to the partition start, may need to
modify values in the VBR to boot correctly

USB-HDD is DL=80h, two partitions, or it may auto-select USB-ZIP with
only one partition for some BIOSes

...


So, their program doesn't support FAT12. It supports only FAT16,
FAT32, NTFS.

According to their website, their program doesn't create an MBR or
partition table for booting USB-FDD, and sets the drive number in the
PBR. (What???) The pdf document says to boot as a USB-FDD the device
must have a device name similar to "TEAC floppy drive", i.e., a real
floppy device. That could be why nothing is booting from the USB stick
for USB-FDD for me.

The pdf also says a USB-ZIP should have only one partition, whereas
USB-HDD should have two partitions, even if one is fake. This is
because the USB-HDD option can auto-select booting as either USB-HDD or
USB-ZIP on some BIOSes.

Apparently, USB-ZIP boots the VBR code directly, skipping the MBR or
boot code, and translates disk calls to start at the beginning of the
partition. According to their website, their program option modifies
the VBR to indicate the device is a floppy, sets 64 heads/32 sectors
below 1GB, instead of LBA 255 heads/63 sectors, as 64/32 boots more
widely.

The pdf says that booting as USB-HDD can select USB-HDD or USB-ZIP if
the image has only one partition, which mine had. I was seeing exactly
what USB-ZIP booting describes. So, I'm thinking the "errant" hard disk
image was actually booting as USB-ZIP from the USB-HDD option. That
would also explain why it came up as A: (DL=0) when booting DOS, and why
tboot.bin was 80h. It would also explain my surprise when it booted DOS
instead of tboot.bin boot code.

Earlier in the document, it says that their program can set the heads
and sectors for USB booting, either 64 heads/32 sectors by default, or
optionally to 255 heads/63 sectors.

Also, their program can force LBA mode by setting 1023 cylinders, 255
heads, 63 sectors in the partition, even if the actual partition
capacity is below 8GB. It says that doing this should "help to
successfully boot an operating system from a USB drive." I'm assuming
that means Windows or DOS, if not Linux too.

So, for a bootable USB stick to correctly boot DOS, it would seem that
I might need to use a hard disk image with only one partition (for
USB-ZIP DL=0 A:), perhaps re-sized to 1.44MB, and perhaps force LBA
mode, and perhaps not FAT12 but FAT16. I'm assuming that the heads and
sectors are the corruption issue, but maybe, the BIOS not recognizing
FAT12 was the source of the corruption? ...

HTH,


Rod Pemberton
--
Does the increase in pedestrian deaths correspond with the increase in
Millennials?

[Benjamin David Lunt]

Hi Rod, Steve, everyone,

"Rod Pemberton" <EmailN...@voenflacbe.cpm> wrote in message
news:ottb5j$15so$3...@gioia.aioe.org...

>
> I've found some info on the BIOS requirements for USB-HDD, USB-FDD,
> USB-ZIP booting.
>
> I'd love to find a formal standards document which describes what the
> pdf explains (link below). Apparently, there is a specification for
> USB-ZIP somewhere that specifies what is needed for USB-ZIP booting.
> http://soft.goodmc.ru/CD&DVD/WinSetupFromUSB-1-7/files/tools/RMPrepUSB/LANG/RMPrepUSB.pdf

Good find, Rod. I will have to read some more of it later.

First, I have been following most of this (the other) thread
and have meant to comment. Sorry I haven't until now.

I did some research of my own, including some tests and came up
with a very similar conclusion as you have.

BTW, there is/was a discussion on this at
http://forum.osdev.org/viewtopic.php?f=1&t=32495

I tested with three machines:

HP Pavillion g6 Notebook, 64-bit, BIOS: "Insyde", version F.17
Dell Dimension 8110, 32-bit, BIOS "Dell", version A01
Dell Inspiron 1440, 32-bit, BIOS: "Dell", version A07

[This is all from memory, so I hope I am correct. I would have to
go over my notes for exactness if needed.]

The HP Pavillion's BIOS would not boot a floppy, image or otherwise,
even when I plugged in a physical TEAC USB floppy drive. It required
a valid Partition Table at LBA 0 of the Thumb Drive, with all
four partition entries valid, even if three of them were all zeros.
It required at least one entry to have 80h as the boot indicator,
and a non-zero value as the OS indicator.

It would not matter the CHS or LBA values, nor would it matter if
there was a valid BPB in LBA zero. (I did not check for 0xAA55
mattered). A member of osdev.org posted that the "Insyde" BIOS
has had problems with USB booting... (Mike, was that you?)

The Dell Dimension would not boot an 8gig Thumb Drive as a floppy
no matter what I did. FAT12, FAT16, FAT12 BPB, DOS BPB, etc. No
matter what I did, it would only boot as a HDD (DL = 80h). This
machine, however, would boot the TEAC USB floppy drive just fine.
Didn't even matter what the disk contents were. It booted because
it was a TEAC floppy drive.

The Dell Inspiron, please note the newer version of the BIOS, would
boot anything I gave it. If I formatted the 8gig Thumb Drive with
a FAT 12 BPB having 2880 sectors (no MBR or Partition Table), it would
boot as a floppy (DL = 0). If I gave it a different BPB, say a
FAT 16 and 65500 clusters, it would boot as a HDD (DL = 80h).
If I gave it a MBR and at least one partition entry, it would boot
as a HDD and boot my MBR code at LBA 0. It would boot the TEAC
as well.

I never once encountered any of them skipping the first 63 sectors,
nor booting directly to the VBR instead of the MBR. Each, when it
would boot, booted the MBR code.

(Please note that if I updated the Dimension to the BIOS version
of the Inspiron, I bet the Dimension would behave the same way
as the Inspiron.)

Anyway, from my testing, I concluded that if your bootable device
contained a valid BPB *and* a valid Partition Table with at least
one valid entry, any machine should boot it, as long as the machine's
BIOS was USB bootable compatible.

However, my research and testing is but a grain of sand in the
deserts of Egypt.

Thanks to you guys for this thread and research. I will add the
PDF and the text to my notes.

(If needed, I can pull out those machines, and maybe a few more
and test some code you have and post the results.)

Thanks,
Ben
http://www.fysnet.net/the_universal_serial_bus.htm

[James Harris]

On 08/11/2017 02:29, Benjamin David Lunt wrote:
> "Rod Pemberton" <EmailN...@voenflacbe.cpm> wrote in message
> news:ottb5j$15so$3...@gioia.aioe.org...
>>
>> I've found some info on the BIOS requirements for USB-HDD, USB-FDD,
>> USB-ZIP booting.

... (Useful info on USB booting snipped)

> Anyway, from my testing, I concluded that if your bootable device
> contained a valid BPB *and* a valid Partition Table with at least
> one valid entry, any machine should boot it, as long as the machine's
> BIOS was USB bootable compatible.

Thanks, Ben. That's good to hear.

I wonder how the structures on a non-volatile RAM device (NVRD) could be
set up in practice. I guess that one would need more than just LBA 0 to
be correct; it would genuinely need to appear to be a floppy image if
viewed in one way and a hard-disk image if viewed another way. And the
space taken up by the floppy image would have to be reserved - probably
in the partition table.

Is it going to far to say that if we assume the floppy/FAT type which
would be most compatible has 2880 sectors then at least the first 2880
sectors of the NVRD would have to be reserved, and the only pragmatic
way to reserve it would be to make it part of a dummy partition? To keep
the power-of-2 alignment let's round it up and say the dummy partition
would have to end at sector 4095 and the first real partition would
start at sector 4096.

But where would such a dummy partition start? Modern partitioning
schemes put the first sector at sector 2048 (assuming 512-byte sectors).
Older schemes begin at the start of the second virtual cylinder -
usually sector 63. It /might/ be possible with a custom partitioner to
start it at sector 1.

Frankly, if it did not start at sector 1 then the space from 1 could be
trampled on by one of the rogue programs which uses unallocatable space.
E.g. if it started at sector 63 then a rogue program could trample on
sectors 1 to 62. I don't think any floppy image - however carefully laid
out - could survive that. Since the floppy image would have FATs and the
root directory in those sectors I can't think of any way to lay out the
floppy image to avoid that kind of corruption. And even if there were to
be a clever way to get those sectors to be ignored, a program seeing it
as a floppy disk could still write files there. So ISTM those sectors
would need to be protected by starting a dummy partition from sector 1.

So where does that get to? I think it gets to the idea of having a dummy
partition which runs from sector 1 to sector 4095, inclusive. The first
2880 sectors of the NVRD would include the first 2879 of the dummy
partition and would be a floppy image and could be used as such. But if
the NVRD were viewed as a partitioned medium there would be a partition
which covered sectors 1 to 4095.

Of course, there's always the possibility of a BIOS seeing a partition
starting in sector 1 and assuming the partition table to be invalid.
That seems unlikely. But it is possible.

>
> However, my research and testing is but a grain of sand in the
> deserts of Egypt.

http://wiki.osdev.org/Problems_Booting_From_USB_Flash includes the
following on the history of BIOS USB boot attempts:

Eventually firmware vendors realised it'd be nice to be able to boot
from USB. Unfortunately there was still no standard describing how "boot
from USB" should work, so different firmware vendors made different
choices. Some decided that because USB is a removable device (unlike
hard drives at the time) it should act like a floppy disk. Some decided
that because USB is (potentially) much larger than a floppy disk it
should act like a hard disk. Some decided to support both and provide a
setting in their BIOS configuration so that half the time it doesn't
work until the end user figures out why and finds/changes the BIOS setting.

It didn't take too long before firmware vendors started inventing
schemes to try to auto-detect whether a USB device should behave like
floppy or like hard drive. Their intentions were good (trying to avoid
end user hassle), but there was no standard describing how "auto-detect
how to boot from USB" was supposed to work so their implementations were
hideous hacks that were incompatible between firmware vendors.

Some checked for a "valid looking" BIOS Parameter Block (BPB) in the MBR
and decided it should behave like a floppy if there was a BPB (and
behave like a hard disk if there was no BPB). Some checked for a "valid
looking" partition table in the MBR and decided it should behave like a
hard disk if there was a partition table (and behave like a floppy disk
if there was no partition table). Some checked for a "valid looking" BPB
and a "valid looking" partition table and made "unknown" decisions if
there was neither or both (possibly including deciding that the device
is not bootable when there's no BPB and not partition table).

--- end of quote ---

That seems to be a logical description of how the 'standards' may have
developed.


--
James Harris

[Scott Lurndal]

James Harris <james.h...@gmail.com> writes:
>On 08/11/2017 02:29, Benjamin David Lunt wrote:

>But where would such a dummy partition start? Modern partitioning
>schemes put the first sector at sector 2048 (assuming 512-byte sectors).
>Older schemes begin at the start of the second virtual cylinder -
>usually sector 63. It /might/ be possible with a custom partitioner to
>start it at sector 1.
>

The GPT specification requires 16kbytes minimum for the partition
table which typically starts at LBA 2 (legacy MBR at LBA0, GPT header
at LBA1). Consider also 4k sector sizes are becoming more common.

https://en.wikipedia.org/wiki/GUID_Partition_Table

See particularly the last paragraph of "Features" which explains
why the first partition starts at sector 63 in legacy MBR
installations.

[Benjamin David Lunt]

"James Harris" <james.h...@gmail.com> wrote in message
news:otvjot$4af$1...@dont-email.me...

> So where does that get to? I think it gets to the idea of having a dummy
> partition which runs from sector 1 to sector 4095, inclusive. The first
> 2880 sectors of the NVRD would include the first 2879 of the dummy
> partition and would be a floppy image and could be used as such. But if
> the NVRD were viewed as a partitioned medium there would be a partition
> which covered sectors 1 to 4095.

I think that would be a good practice as well.

Ben

[James Harris]

On 08/11/2017 20:15, Scott Lurndal wrote:
> James Harris <james.h...@gmail.com> writes:
>> On 08/11/2017 02:29, Benjamin David Lunt wrote:
>
>> But where would such a dummy partition start? Modern partitioning
>> schemes put the first sector at sector 2048 (assuming 512-byte sectors).
>> Older schemes begin at the start of the second virtual cylinder -
>> usually sector 63. It /might/ be possible with a custom partitioner to
>> start it at sector 1.
>>
>
> The GPT specification requires 16kbytes minimum for the partition
> table which typically starts at LBA 2 (legacy MBR at LBA0, GPT header
> at LBA1).

I don't think there's any way a floppy image could coexist with GPT-boot
structures.

> Consider also 4k sector sizes are becoming more common.

True. Aside from those which emulate 512-byte sectors I don't know how
4k-sector drives would change the structures we have been talking about.
At a guess, I would assume that the first partition would still start
1Mbyte into the medium rather than at sector 2048. The following page
currently has 1Mbyte alignment as the most coarse.

https://en.wikipedia.org/wiki/Partition_alignment

>
> https://en.wikipedia.org/wiki/GUID_Partition_Table
>
> See particularly the last paragraph of "Features" which explains
> why the first partition starts at sector 63 in legacy MBR
> installations.

You mean that partitions start on track boundaries (rather than cylinder
ones)? Good point.


--
James Harris

[James Harris]

Since you agree, I thought it would be worth trying. The most useful way
to set it up is probably to use an assembler. That way, the image can
include data structures and code as needed.

I had a go at it and the resulting program is below. The generated file
will mount as an image as FAT12 /and/ can be manipulated with fdisk,
which is cool, if a bit weird!

To recap, the basic idea is to put it and some boot code at the start of
something like a USB flash memory so as to maximise the number of BIOSes
that will be able to boot it. As a guess, if the BIOS's USB boot option
were to be set to USB-FDD it should boot as a floppy (DL=0 and with the
specified FAT12 geometry) or if the BIOS were set to USB-HDD it should
boot as a hard disk (DL=0x80 but I am not sure what geometry settings it
would use; in layout terms it assumes the "defaults" of 63 sectors per
track and 255 tracks per cylinder). A BIOS set to USB boot (without
specifying -FDD or -HDD) should, hopefully, autodetect one of the above.
All untested as yet.

If more boot-code space is needed (quite likely) that can easily be made
available by making RESERVED_SECTORS higher than 1.

Feedback from any of you guys would be welcome. If you have a means of
testing it, does it work in practice? Any changes needed? Any queries on
it?

The code follows. I called it hyimg.nasm.


;**********************************************************************
;
; Produce a hybrid FDD/HDD image
;
;**********************************************************************

;Assemble to produce a disk image which can be seen as either
;a hard disk or a floppy disk.
;
;Basic layout of the image:
;
; 1 sector Combined hard-disk MBR and FAT-12 floppy VBR
; 2879 sectors The remainder of a 1.4Mbyte floppy image
; Padding sectors Padding to complete a dummy partition
;
;That can be followed by the first usable hard-disk partition.
;
;The MBR's partition table will have an entry which reserves
;the space occupied by the floppy and some padding. The padding
;will get to a boundary which is suitably aligned for the first
;real, usable partition.

;The image is to be bootable either as a floppy disk or as a
;hard disk.
;
;Configure the parameters by changing the master constants,
;below, and then build with
;
; nasm -f bin -O 2 hyimg.nasm -o hybrid.img
;
;There are various ways the generated image file could be
;used. If could be copied to the start of a device such as
;a USB flash memory unit. It could be extended by adding
;sectors at the end (e.g. with dd and >>). Or this Nasm file
;could be included from within another Nasm file which appends
;bytes for one or more partitions (which would need to be
;added to the partition table).

;Master constants
%define DUMMY_PARTITION_KB 4096 ;Size of the dummy partition
%define DUMMY_PARTITION_TYPE 0xda ;Dummy partition typecode
%define FAT_MEDIA_TYPE 0xf0 ;Filesystem media type
%define FATS 2 ;Number of FATs
%define FLOPPY_SECTORS 2880 ;Sectors in the floppy image
%define HD_SEC_PER_TRK 63 ;Sectors per HD track
%define HD_TRK_PER_CYL 255 ;Tracks per HD cylinder
%define HIDDEN_SECTORS 0 ;Sectors prior to this sector
%define INITIAL_LBA 1 ;Dummy partition's initial LBA
%define OEM_NAME "jasscl02" ;OEM name
%define PTAB_OFFSET 446 ;Partition-table offset in the MBR
%define RESERVED_SECTORS 1 ;Sectors before the 1st FAT
%define ROOTDIR_SECTORS 14 ;Sectors in the root directory
%define ROOTENTRY_SIZE 32 ;Size of each entry in the rootdir
%define SECSIZE_LOG2 9 ;Binary alignment of sector size
%define SECTORS_PER_CLUSTER 1 ;Sectors per cluster
%define SECTORS_PER_FAT 9 ;Sectors per FAT
%define SECTORS_PER_TRACK 18 ;Sectors per track
%define TRACKS_PER_CYLINDER 2 ;Tracks per cylinder, aka heads
%define VOLUME_ID 0x12345678 ;Volume ID
%define VOLUME_LABEL "NO NAME " ;Volume label

;Derived constants
%assign SECTORS_PER_CYLINDER SECTORS_PER_TRACK * TRACKS_PER_CYLINDER
%assign HD_SEC_PER_CYL HD_SEC_PER_TRK * HD_TRK_PER_CYL
%assign FINAL_LBA ((DUMMY_PARTITION_KB * 1024) >> SECSIZE_LOG2) - 1

%assign ICYL INITIAL_LBA / HD_SEC_PER_CYL
%assign rest INITIAL_LBA % HD_SEC_PER_CYL
%assign ITRK rest / HD_SEC_PER_TRK
%assign ISEC rest % HD_SEC_PER_TRK + 1

%assign FCYL FINAL_LBA / HD_SEC_PER_CYL
%assign rest FINAL_LBA % HD_SEC_PER_CYL
%assign FTRK rest / HD_SEC_PER_TRK
%assign FSEC rest % HD_SEC_PER_TRK + 1


;**********************************************************************
;
; Code start
;
;**********************************************************************

cpu 8086

boottop:
jmp short bootstart ;Standard jump (see FAT12 spec)
nop ;Standard filler (see FAT12 spec)

BS_OEMName db OEM_NAME
BPB_BytsPerSec dw 1 << SECSIZE_LOG2
BPB_SecPerClus db SECTORS_PER_CLUSTER
BPB_RsvdSecCnt dw RESERVED_SECTORS
BPB_NumFATs db FATS
BPB_RootEntCnt dw (ROOTDIR_SECTORS << SECSIZE_LOG2) / ROOTENTRY_SIZE
BPB_TotSec16 dw FLOPPY_SECTORS
BPB_Media db FAT_MEDIA_TYPE
BPB_FATSz16 dw SECTORS_PER_FAT
BPB_SecPerTrk dw SECTORS_PER_TRACK
BPB_NumHeads dw TRACKS_PER_CYLINDER
BPB_HiddSec dd HIDDEN_SECTORS
BPB_TotSec32 dd 0 ;Zero says to use BPB_TotSec16
BS_DrvNum db 0x00 ;Zero means floppy
BS_Reserved1 db 0
BS_BootSig db 0x29 ;Indicates the next 3 fields are present
BS_VolID dd VOLUME_ID
BS_VolLab db VOLUME_LABEL
BS_FilSysType db "FAT12 "

%if BPB_BytsPerSec - BS_OEMName <> 8
%error "OEM name is the wrong size"
%endif

%if $ - BS_OEMName <> 59
%error "Geometry table as defined in this code has the wrong size"
%endif

%if BS_OEMName - boottop <> 3
%error "Geometry table is in the wrong place"
%endif


;**********************************************************************
;
; Boot code
;
;**********************************************************************

bootstart:

;Boot code goes here
;%include hyboot.nasm


;**********************************************************************
;
; Partition table for when the image is seen as a hard disk
;
;**********************************************************************

;Pad to partition table
times PTAB_OFFSET - $ + boottop db 0

;The entry for the dummy partition
db 0x00 ;Bootable/not-bootable indicator byte
db ITRK
db ((ICYL >> 2) & 0xc0) | (ISEC & 0x3f)
db ICYL & 0xff
db DUMMY_PARTITION_TYPE ;Type of this partition
db FTRK
db ((FCYL >> 2) & 0xc0) | (FSEC & 0x3f)
db FCYL & 0xff
dd INITIAL_LBA ;Start sector
dd FINAL_LBA - INITIAL_LBA + 1 ;No of sectors

;The other partitions
times 3 * 16 db 0


;**********************************************************************
;
; Boot signature
;
;**********************************************************************

;The boot signature
db 0x55, 0xaa


;**********************************************************************
;
; Other reserved sectors, can contain loadable code
;
;**********************************************************************

times (RESERVED_SECTORS << SECSIZE_LOG2) - $ + boottop db 0


;**********************************************************************
;
; FATs
;
;**********************************************************************

%rep FATS

db 0xf8, 0xff, 0xff
times 0xa00 - 3 db 0

%endrep


;**********************************************************************
;
; Root directory
;
;**********************************************************************

times ROOTDIR_SECTORS << SECSIZE_LOG2 db 0


;**********************************************************************
;
; Rest of the floppy image
;
;**********************************************************************

times (FLOPPY_SECTORS << SECSIZE_LOG2) - $ + boottop db 0


;**********************************************************************
;
; Padding to the end of the dummy partition
;
;**********************************************************************

times DUMMY_PARTITION_KB * 1024 - $ + boottop db 0


;**********************************************************************
;
; First usable partition starts here
;
;**********************************************************************






--
James Harris

[James Harris]

On 10/11/2017 16:31, James Harris wrote:

...

> ;**********************************************************************
> ;
> ; FATs
> ;
> ;**********************************************************************
>
> %rep FATS
>
> db 0xf8, 0xff, 0xff
> times 0xa00 - 3 db 0

Oops - when converting from fixed values to parameterised ones I missed
the above. To fix the last line, it should be

times (SECTORS_PER_FAT << SECSIZE_LOG2) - 3 db 0

>
> %endrep


The thing that's puzzling me ATM is where I ever got 0xa00 from!


--
James Harris