File System ID - "The alt.os.development boot sector descriptors standards committee"
Leif
ByrdKernel:
> So using the above formula to calculate it, "partition_type" 0x7F01 could
be
> Moebius, 0x7F02 could be RDOS, and 0x7F69 could be mine. :)
You mean RDFS I suppose? :)
BTW, I like names in second sector, alternatively, names in the OEM
identifier
best. However, this probably means somebody should be in charge of
registering FS names, to avoid collisions.
Leif
www.rdos.net
Stéphane Martineau
> identifier
> best. However, this probably means somebody should be in charge of
> registering FS names, to avoid collisions.
You should take position by voting in the thread "New Partition Type ID
(Please Vote!)"
ByrdKernel
> You mean RDFS I suppose? :)
Oops...using OS names instead of FS names. :) But yes, I did mean RDFS.
> BTW, I like names in second sector, alternatively, names in the OEM
> identifier
> best. However, this probably means somebody should be in charge of
> registering FS names, to avoid collisions.
You know, it occurs to me that if we're going to use fields (such as OEM ID, file
system name and version, etc) that have been present in the FAT spec for years,
why even bother with a specific partition type, or in setting a standard? The
FAT boot sector is in common use, and already has ASCII identifiers for all kinds
of things.
I thought the purpose was to break away from the standard a bit without
compromising it?
ByrdKernel (Mike)
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
Stéphane Martineau
> compromising it?
IMO, that's the point! The OEM field is essentially for FAT partition not
for us, we should ignore that!
xby
>
> Using the next sector for the partition signature is a good idea, that way
> you don't "loose" any precious byte in the boot sector of the partition.
A word of caution: in the case of FAT32, sectors between the boot
sector and the FAT sectors are reserved by Microsoft for fixed sector
assignments for the purpose of data recovery. Be careful before
assigning any of those for your own purpose.
Stéphane Martineau
> sector and the FAT sectors are reserved by Microsoft for fixed sector
> assignments for the purpose of data recovery. Be careful before
> assigning any of those for your own purpose.
The definition is for new partition type not for existing ones. But anyway,
we've agreed on using the first sector so it should work with existing FS
too.
Matthias Paul
>> Why so far into the middle? Why not, say at location 4? And do we
>> need to give 14 bytes to it, given that space is at a premium in
>> boot sectors? Also, 32 bits (or 64 bits, or a 128-bit GUID) at
>> location 4 is conviently in the middle of the MS-DOS OEM name. :)
>
> The OEM name is often printed on screen by bootloaders and
> partition tools. There is no semantics to it though as far as I
> know, and it is probably not a good idea to put a binary field
> there.
In contrast to popular belief there *are* some undocumented semantics
for the OEM label at least in MS-DOS/PC DOS, Windows 9x/SE/ME, and
probably also in OS/2 and Windows NT/2K/XP.
We found it out the hard way at Caldera UK back in 1997 when we
changed the OEM label of DR-DOS from "IBM 3.3" to seemingly
valid "DRDOS702" and "DRDOS 7" labels in FORMAT/SYS/FDISK:
According to my research and the kind help of a few people in the
OpenDOS/DR-DOS mailing list (subscription at http://www.delorie.com),
if the format of the OEM label is not in the format "nnnnx.y" (where
"nnnnn" is a random string and "x.y" corresponds to a valid main and
subversion before 5.0), MS-DOS/PC DOS will ignore some of the entries
in the BPB and recalculate the "untrusted" ones from the other entries
in the BPB. This is special cased in the IO.SYS/IBMBIO.COM for various
OEM version numbers. Presumably, this is to work around problems
in some of their old formatting tools, which wrote wrong data
into the BPB (old versions of the OS/2 FDISK are known to have
had such serious bugs).
So, if the semantics of the BPB are not used in exactly the same
format as expected by retail versions of MS-DOS/PC DOS, MS-DOS/PC DOS
may - under some further trigger conditions - calculate wrong values
and thereby fail to log in the partition(s) correctly.
Since DR-DOS 7.0x still used the BPB semantics of Compaq MS-DOS 3.31
(with several extensions not related to the problem), all worked
fine as long as the OEM label was "IBM 3.3". When we changed this
to reflect the true formatter (unknowningly in a format not matching
the "nnnnnx.y" template), MS-DOS/PC DOS would assume a wrong cluster
size and calculate wrong geometry values for partitions smaller than
128 Mb (although all it had to do is using the correct values stored
in the BPB by DR-DOS to avoid these problems).
This causes a DIR to show garbage and any write access to the
corresponding partition under MS-DOS/PC DOS to corrupt data!
On the other hand, there is absolutely no problem for operating
systems, which read out and use the values in the BPB according
to the offical specification, so DR-DOS has had no problem to
read/write to such partitions (it also has no problems to read/
write to partitions created by MS-DOS/PC DOS, of course).
The fix is to just change the OEM label back to "IBM 3.3"
and MS-DOS/PC DOS will smoothly use these partitions without
going havoc.
A similar observation is that older versions of PC Tools DISKFIX
do not like the version number in the OEM label to be higher
than 5.0. It seems, this is the reason why, after being "MSDOS6.0"
in MS-DOS 6.0 beta versions already, the OEM label stayed at
"MSDOS5.0" in released version of MS-DOS 6.0 - 6.22 and even was
changed back to "MSWIN4.0" and "MSWIN4.1" with Windows 95/98/SE/ME.
Yet another observation is that OS/2 Warp 3 (probably other
versions as well) has problems to access FAT16 partitions created
by Novell DOS 7 ("NWDOS7.0"), OpenDOS 7.01 ("OPENDOS7") or
DR-DOS 7.0x ("DRDOS702" and "DRDOS 7"), while it has had no
problems to access partitions created by DR DOS 6.0 (which
still used "IBM 3.3").
I am quite sure that this is just a matter of the OEM label,
at least reformatting the partitions with the MS-DOS/PC DOS
FORMAT removes the problem - I still have to check if just
"fixing" the OEM label would do as well, but I strongly
assume it would.
Finally, another "interesting" use of the OEM label (although
"only" on removable media) is that the Windows 9x/SE/ME GUI
(possibly also Windows 2000/XP, but I don't know for sure,
since I don't use them) will overwrite the OEM label of a
floppy disk on the *first* access to the medium, even if
this is only a simple (seemingly read-only) command like
DIR A:
The OEM label will look like "IHCnnnnn" afterwards, where
"nnnnn" appears to be a pseudo-random serial number or
checksum of some kind.
According to some Microsoft Windows Knowledge Base Articles
(Q148637: "Windows 95/98 Overwrites Boot-Sector Field on
Floppy Disks" and Chengi Jimmy Kuo's paper "What is not a virus"
delivered at the NCSA conference in Washington, DC on
1996-04-01/02, they document this behaviour as being related
to the Volume Tracker in Windows 9x/SE/ME, which writes a
special number to the floppy disk in order to keep track
of it when flipping media. I only wonder why we already
have a serial number entry in the BPB, then... ;-)
When Windows does not find a recognized OEM label (like the
already mentioned "IBM 3.3") it will ignore some of the
BPB entries and recalculate them from other entries. Since
some of the BPB values written by 3rd party software do not
exactly match the values stored there by retail MS-DOS/PC DOS
versions, this will cause Windows to effectively "destroy"
many floppies formatted by 3rd party formatters and operating
systems even on the "harmless" DIR, since without the proper
OEM label it will no longer be able to log in such floppies
afterwards, although the values found in the BPB are perfectly
valid according to the standards. Typically, you will receive
error messages like "Sector not found" or "Read error" then.
As a non-exclusive example, this also affects floppies formatted
under DR-DOS. They can be read without problems as long as the
OEM label is "IBM 3.3" and the medium is switched to read-only.
However, most people leave the lever in the read-write position,
so Windows will overwrite the OEM label with its "IHCnnnnn" string
even on a simple DIR A:, and this will make the floppy unreadable
under Windows 9x/SE/ME afterwards. Changing the OEM label back
to "IBM 3.3" will temporarily fix the problem. Of course,
they can still be read by DR-DOS, even with the OEM label
changed to "IHCnnnnn".
I guess, millions of seemingly faulty floppy disks have been
thrown away as being unusable, although all that has happened
was that Windows silently changed the OEM label...
Since there is no known method to disable Windows Volume Tracker's
bad manners, the easiest way to fix the problem is to add known
OEM labels to an exclusion list under the registry key:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\ ...
... FileSystem\NoVolTrack
Over the past months I have collected many OEM labels from various
sources for Ralf Brown's Interrupt List, and thereby I also created
a Windows registry key import file named IHC.REG.
Once it is finished I will make the final version of the file
available on http://www.drdos.org. If there is a strong demand,
I could also post the current issue of the file, which I already
use for several months now, here (since it contains many OEM labels,
already it's 35 Kb plain ASCII, 6 Kb compressed as .ZIP archive).
Hope it helps,
Matthias
--
<mailto:Matthi...@post.rwth-aachen.de>; <mailto:mp...@drdos.org>
http://www.uni-bonn.de/~uzs180/mpdokeng.html; http://mpaul.drdos.org
"Programs are poems for computers."
Stéphane Martineau
OEM field is in fact a very important microsoft field that should
not be tampered with.
Thanks for all that info Matthias!
Maxim S. Shatskih
> "only" on removable media) is that the Windows 9x/SE/ME GUI
> (possibly also Windows 2000/XP, but I don't know for sure,
Just looked at FASTFAT source from w2k.
It NEVER uses the OEM name in the boot sector, under no circumstances.
I'm just surprised what is the need to use this useless label at all.
Are FAT layouts dependent on it? No. So then why use it at all? To
label the volume? FAT has another mechanism for volume labels.
Max
ByrdKernel
> Just looked at FASTFAT source from w2k.
> It NEVER uses the OEM name in the boot sector, under no circumstances.
Okay...how does Win2K determine whether a FAT disk or partition is a VALID FAT
disk or partition. Guessing?
Maxim S. Shatskih
VALID FAT
> disk or partition. Guessing?
By checking the BPB values to be sane. Only the BPB is used, not this
string.
Do you want the source for this function?
Max
Maxim S. Shatskih
> OEM field is in fact a very important microsoft field that should
> not be tampered with.
...and your guess is wrong, since it contradicts the NT's FASTFAT
source.
Max
Stéphane Martineau
Max, are you a troll ? :) Seriously, did you read the post before making
that reply ? Matthias Paul clearly stated that various ms operating
system, mostly dos versions, did use the field. He was talking about it's
practical experience with it. So it doesn't matter if the NT FASTFAT
source that you have do not use it, this doesn;t change the fact that
microsoft has and may still use that field for their own undocumented
purpose :)
So no, based on Matthias experiences, which are much more valid imo
then you're own assumptions, my guess is not wrong.
But it doesn't matter my friend, just like anybody else here, feel free to
think whatever you want :)
Maxim S. Shatskih
systems".
> So no, based on Matthias experiences, which are much more valid imo
> then you're own assumptions
The fact NT does not use the field is a fact and not an assumption.
Max
Tim Robinson
| Sorry for being too NT-centric and forgetting other MS's "operating
| systems".
Which "other operating systems"?
--
Tim Robinson
http://www.themoebius.org.uk/
ByrdKernel
> By checking the BPB values to be sane. Only the BPB is used, not this
> string.
> Do you want the source for this function?
No thanks...I trust you. :) But let's just say, for example, that someone
produced a filesystem that used the same bytes for the same purpose, but wasn't
FAT. However, the maker of this OS clearly marked the partition type as being a
different value (which one doesn't matter, but one that is definitively NOT any
of the FAT family). I have seen several OSDev'ers doing this...none have gotten
very far, but it's a valid approach to filesystems, IMHO.
On this theoretical filesystem, FASTFAT would read this as a FAT partition.
Correct? Of course, that would be totally incorrect, and it would be FASTFAT's
fault, since Microsoft decided to abandon the standard they created years ago
(and that others still use).
BTW, while you're looking at source, look at the source for the FDISK and FORMAT
utilities (or the library functions they use)...they mark the partition as a FAT
partition even in Win2K and XP, no?
ByrdKernel
> ...and your guess is wrong, since it contradicts the NT's FASTFAT
> source.
No, because he wasn't talking about NT. His guess is right, since he was talking
about other OS's in the DOS/Windows family.
ByrdKernel
> The fact NT does not use the field is a fact and not an assumption.
No, based on an earlier post of yours, the fact that NT's filesystem mounting
code does not use it is a fact. NT uses it...when it creates/formats a
partition, and when you run WINDISK on it. :) Possibly at other times.
ByrdKernel
> Which "other operating systems"?
Presumably, the ones Matthias was talking about. I don't remember the list, but
they're in his post about the OEM ID.
Maxim S. Shatskih
"operating
> | systems".
>
> Which "other operating systems"?
DOS and Win9x.
Max
Maxim S. Shatskih
someone
> produced a filesystem that used the same bytes for the same purpose,
but wasn't
> FAT.
This kind of FS will break both NT and Linux.
But the chances for such filesystem are very low. The set of possible
first sector contents is much, much larger then a set of 256 values.
So, clashes on type byte are real, and do occur, and clashes on first
sector format are nearly fantastic.
BTW - NTFS does check the OEM ID in the boot sector to be "NTFS", and
also the BPB values except SectorsPerCluster to be all zeroes.
:-)
> fault, since Microsoft decided to abandon the standard they created
years ago
IBM created the standard, and failed to maintain it - no authority to
assign these values.
And yes, MS decided to abandon it. A guy from MS's filesystem team
explicitly told - do not use these values.
> utilities (or the library functions they use)...they mark the
partition as a FAT
> partition even in Win2K and XP, no?
They do. The disk management UI seem to use this value, but not the
main mount path.
Max
Matthias Paul
> We found it out the hard way at Caldera UK back in 1997 when we
> changed the OEM label of DR-DOS from "IBM 3.3" to seemingly
> valid "DRDOS702" and "DRDOS 7" labels in FORMAT/SYS/FDISK:
As I wrote elsewhere in the post, some of the OEM labels
had already been changed for Novell DOS 7 (in 1994) -
and had caused problems with OS/2 (at least with Warp 3).
> [...] if the format of the OEM label is not in the format
> "nnnnx.y" (where "nnnnn" is a random string and "x.y"
> corresponds to a valid main and subversion before 5.0),
> MS-DOS/PC DOS will ignore some of the entries in the BPB
> and recalculate the "untrusted" ones from the other entries
> in the BPB.
Of course, I meant "nnnnnx.y" in both cases.
And since version 5.0 is fine as well, it should have
read "subversion 5.0 or before."
> So, if the semantics of the BPB are not used in exactly the
> same format as expected by retail versions of MS-DOS/PC DOS,
> MS-DOS/PC DOS may - under some further trigger conditions -
> calculate wrong values and thereby fail to log in the
> partition(s) correctly.
What I erroneously named "retail version" here, were not
necessarily retail versions, but just unmodified generic
versions of MS-DOS/PC DOS (as sold by Microsoft, IBM,
or many of the smaller OEMs) in contrast to some of the
heavily OEM-modified "OEM versions" of MS-DOS/PC DOS.
Sorry, for not making this more clear right from the start.
Matthias Paul
>> Which "other operating systems"?
>
> Presumably, the ones Matthias was talking about. I don't
> remember the list, but they're in his post about the OEM ID.
Here is the list of all operating systems I am currently aware
of which actively interpret and use OEM label data in one way
or the other (beyond just writing OEM labels):
- MS-DOS 3.1 - 6.22
- IBM PC DOS 3.1 - 7 & 2000
- many, but not all OEM issues of MS-DOS/PC DOS 3.1+
- Windows 95 (all known OSRs and OPKs) & 98 & 98 SE & ME
- IBM OS/2 (at least Warp 3, probably other versions as well)
This applies to all known North American and Western European
issues, as well as the significantly different Russian,
Arabic, Hebrew, Japanese, Chinese, and Hangul editions.
Not being mentioned in the list does not mean that the OEM
label is not used, it just means, that I don't know it.
One thing I /do/ know is, that Digital Research's single-user series
including DOS Plus 1.2 - 2.1, DR DOS 3.31 - 6.0, DR DOS "Panther" &
"StarTrek", NetWare PalmDOS 1.0, Novell DOS 7, Caldera OpenDOS 7.01,
and DR-DOS 7.02 - 7.05 do not use the OEM label.
This also applies to the multi-user DOS series including all
flavours of Concurrent CP/M-86 and Concurrent DOS, and I am
quite sure, it still applies to all more recent issues of
FlexOS, Multiuser DOS, System Manager, and REAL/32.
Both, PhysTechSoft's and Paragon Technology Systems' PTS-DOS families
(including S/DOS), Datalight's ROM-DOS and Heiko Goeman's WinDOS
are not known to use the OEM label, but I cannot make any definite
statement in regard to them, of course.
So far, it seems as if Windows NT & 2000 & XP does not use the
OEM label as well, but I am not sure about it.
Greetings,
Matthias
PS. We should distinguish between the OEM ID (DOS INT 21h/AX=3000h)
and the OEM label (at the start of a BPB in a boot sector).
Matthias Paul
>> Finally, another "interesting" use of the OEM label (although
>> "only" on removable media) is that the Windows 9x/SE/ME GUI
>> (possibly also Windows 2000/XP, but I don't know for sure,
>
> Just looked at FASTFAT source from w2k.
> It NEVER uses the OEM name in the boot sector, under no
> circumstances.
Thanks for the info. That would be really good to know.
However, counting from the name and the fact that this is
NT's filesystem driver, I'm not sure, if FASTFAT would be
the right driver level to look at for any dealings with
OEM labels. But then, I have no immediate insight into the
internals of Windows NT, in particularly not Microsoft's
file naming schemes.
Anyway, I would look at lower level drivers in order to find
possible traces of OEM label special cases.
Another way to find out what happens would be to try to read
a DR-DOS formatted non-write-protected floppy under Windows
NT/2K/XP, eject the floppy, and see if the OS is able to log
it in again, afterwards. Also, read out the boot sector and
see if the OEM label has changed.
Well, I cannot remember having observed such a behaviour
under NT 4.0, but without any form of NT here right now,
I cannot find out myself.
I would certainly be interested to learn how the NT family
handles volume tracking. Just use a CRC over the boot sector
and FAT or something similarly more reasonable than destroying
the OEM label?
Do you know of a better way to keep the Windows 95/98/SE/ME
Volume Tracker from overwriting the OEM label but to add
up to 256 faked OEM labels to the registry? For example, is
there a key to disable this behaviour with a single switch?
> I'm just surprised what is the need to use this useless label
> at all. Are FAT layouts dependent on it? No. So then why use
> it at all? To label the volume? FAT has another mechanism for
> volume labels.
Actually, there are two different ways to assign volume labels
on FAT partitions:
The first one is stored in the file system itself using
an 8.3 root directory entry with (only) the volume attribute
set (11 bytes). It can be accessed via INT 21h/AH=11h.
The other label is stored in the DOS 4.0+ extended BPB
at sector offset +2Bh..+35h (11 bytes) and can be accessed
via INT 21h/AH=69h.
As you already pointed out, the OEM label at sector offset
+03h..+0Ah (8 bytes) in the boot sector is something
completely different.
The FAT filesystem itself does not need the OEM label
in any way, these are really situated at two different
hierarchy levels in the DOS architecture:
All the physical UDSC geometry data necessary for the low-
level disk driver in the DOS BIOS should be readily available
in the boot sector's BPB. In fact, while the built-in disk
driver uses this (and other) information, not even this is
actually required for DOS as long as the corresponding block
device driver is capable of returning a valid BPB structure
during init and after media change - where that information
actually comes from is don't care for DOS itself. (Some DOS
RAM disk drivers go as far as storing their resident code
in place of a valid "boot sector". This may confuse some disk
tools, but not DOS itself, as long as the driver returns valid
data through its device driver interface.)
Filesystem stuff is handled at a much higher level in the FDOS,
which is part of the BDOS kernel, not the DOS BIOS. It works on
logical drives and is based on data in the DPB/DDSC records.
Of course, the data in the DPB/DDSC records derived from
BPB/UDSC data, and, as I will try to describe in better
details below, some Microsoft and IBM operating systems
use the OEM label to aid the correct interpretation of
a boot sector's BPB. But, still, there is no use for an
OEM label at filesystem level, the DOS kernel does not
deal with it.
(That's also, why NT's FASTFAT driver might be the wrong
place to look at when searching for code dealing with
OEM labels, in case it exists.)
Unfortunately, there is not one single BPB format. While most
of the entries did not change over all the years since DOS 2.0+,
some entries differ between various OEMs and DOS versions,
so the code interpreting the BPB must deal with a number of
special cases in order to work with all formats.
This shouldn't be that much of a problem, as the DOS 4.0+
extended BPB has a magic signature (29h) at sector offset +26h,
DOS 7.10+ FAT32 BPBs can be easily detected as well (not on
floppies, anyway), and those entries, which exist in alternative
forms, have well defined magic values like 0 or -1, which
mark the old entries as invalid, when the new ones should be
used instead.
Some of the BPB entries are only used by the bootstrap loader
in the boot sector itself - so, these entries are "don't care"
for the DOS BIOS and the DOS kernel itself.
The same applies to proprietary extensions (like the kernel
file name and target load position in DR-DOS boot sectors,
or the boot unit in Compaq MS-DOS boot sectors) which are
outside the usual range of the BPB and only deal with boot
time configuration options.
They are interpreted only by the bootstrap loader in the boot
sector (and the corresponding FORMAT/SYS/FDISK utilities),
and impose no extra complexity on DOS.
Versions of MS-DOS/PC DOS prior to 3.1 seem to have only used
the FAT ID byte in order to determine the media format, not
the DOS 2.0+ BPB - some older OEM versions of MS-DOS only
wrote boot sectors (and therefore BPBs) for bootable media,
while non-bootable floppies just had an empty boot sector.
On the other hand, I remember, that some Atari ST formatting
tools under GEMDOS did not write any FAT IDs at all, hence
the media had invalid FAT IDs such as F6h or F4h (maybe
even 00h or E5h), and were not readable by issues of
MS-DOS/PC DOS prior to 3.1 (maybe 3.2?).
As a plausibility test, the DOS 3.3+ BIOS will look at the
first byte(s) of the boot sector to be either E9h (JMP NEAR),
or EBh,xxh,90h (JMP SHORT+NOP), or 69h (for unknown reasons
only on floppies). (In addition to this, DR-DOS 7.xx also
accepts the reverse: 90h,EBh (NOP+JMP SHORT).)
DOS 2.0 - 3.2 also checked the boot sector signature at
offset +1FEh to be 55h,AAh, but many 3rd party floppies
did not (and still do not) provide this signature, so it
is no longer being tested on since DOS 3.3+. (Old issues
of DR DOS performed this test as well, but at least
DR DOS 6.0+ does no longer.)
Most issues of MS-DOS/PC DOS 3.1 (but not, for example,
Falcon Technology MS-DOS 3.1 or some German OEM version
of PC DOS 3.1) required the OEM label to start with "IBM "
(4 bytes, not 5!). The same held true for the generic, as
well as most OEM issues of MS-DOS/PC DOS 3.2 - 3.3, while
MS-DOS 4.01 added "MSDOS" and "OS2 " (again, 4 bytes only!)
to the list of recognized strings (but not, for example,
PC DOS 4.00).
Just to give an example of an OEM version, Zenith MS-DOS
3.20 and 3.21 are known to sense for different OEM labels,
and probably dealt with them in a different way than generic
issues of MS-DOS/PC DOS, but I haven't examined any details.
The OEM version numbers "2.0" as well as "3.0" and "3.1"
were treated special in MS-DOS/PC DOS 3.1 - 3.3. (Actually,
the latter is not completely true, as the DOS BIOS triggered
for anything "3." with the following byte in the range
00h..31h, something I would call a bug.)
MS-DOS 4.01 also special cased OEM labels starting with
"IBM 10" (with a single space in between!) and "OS2 10"
(again!), unless the label was continued with a "." character
as, for example, in "IBM 10.0" and "OS2 10.0". (Geoff Chappell
writes about DOS 5.0+: "Major versions that are multiples of
10 other than 10 itself or 20 are regarded as unknown and
access to the drive is disabled.", but I could not find proof
for this interpretation.)
If the OEM name and version did not match these criteria,
the DOS BIOS treated the disk as unknown and assigned
default parameters, else the BPB was interpreted according
to the OEM version number. The exact details differ between
DOS versions, and are therefore difficult to find out
and describe.
For DOS 5.0+, if no valid BPB could be found, the DOS BIOS
will read the FAT ID byte and try to build a default BPB
accordingly. It may also query the system BIOS (INT 13h/AH=08h)
or, for harddisks, use geometry data from the partition table
in order to determine default characteristics.
With MS-DOS 5.0 - 6.22 and PC DOS 5.0 - 2000, the first
five bytes "?????" in the OEM label are don't care now,
but the version number is still required to be in the
proper format "x.y" (in most cases, that is):
For example, if the media descriptor (in the BPB) is an
even value in the range F2h..FEh, for any OEM label version
number which does not start with "3." (2E33h) or has a
sub-version smaller than "2" (32h), the sectors/cluster
value is forced to 1 regardless of the actual value in
the BPB.
I can vaguely remember some old single-sided DOS floppies
to have had faulty sectors/cluster values, so the only
reasonable explanation for this odd behaviour I was able
to think of so far was, that their engineering actually
tried to cope either with that very problem or at least
a similarly buggy BPB.
However, even when putting aside any "deliberate coding"
nightmares (which I don't think apply in this specific
case - it's just careless coding) and just trying to make
a "good sense" interpretation of what that behaviour might
be good for, I cannot help myself but to come to the
conclusion, that it is extremely buggy in itself:
Let's assume for now, they wanted to put any OEM version
before 3.2 under special treatment for single-sided media.
Well, most (but not all) of the even media descriptor values
were originally used for "non-double-sided" media only -
see IBM PC DOS 3.3 Technical Reference (1987), which
gives an algorithmical definition of the media descriptor
rather than the usual lookup table as in similar Microsoft
and 3rd party documentation.
So far so good, but AFAI understand it, their solution could
also affect harddisks (usually with a media descriptor F8h)
and various double-sided media (for example, some Japanese AX
or Amstrad PC media use FEh for various double-sided formats).
Further, it does not only affect the OEM versions 2.0..3.1,
which seem to have been the intended targets, but also any
OEM version 4.0 and up, any OS/2 disks having a "0" in place
of the "3", and most (but not all) disks with OEM version
numbers not in the proper "x.y" format. For example, "211",
"3-3", "3./", "702", " 7", or "IHC" would be treated specially,
while "3.5" or "3.:" won't). Rather inconsistent, isn't it?
Now, OEM labels destroyed under Windows 95/98/SE/ME (not
under plain MS-DOS 7.0/7.10/8.0) will always look like
"?????IHC".
(Some sources speculate that this might be a remainder of
some "CHICAGO" string in reverse spelling - mind, that the
Windows 95 Beta was named "Chicago". This would line up
with the reported behaviour of Windows NT/2K/XP, which does
not seem to show the same bad habits).
Anyway, destroying the OEM label will effectively render
all disks meeting the above mentioned criteria unreadable
at least under the listed DOS versions - unless they have
had a 1:1 representation of sectors per clusters already.
This one example explains much, but not everything observed.
There are many more special cases. AFAIK, for example,
MS-DOS/PC DOS 5.0+ (maybe earlier) enforces the following
characteristics regardless of the values provided in the
BPB: 2 FATs, 512 bytes per sector and 1 reserved sector.
Floppy disks are assumed to have zero hidden sectors and only
the low byte of the number of root directory entries from the
16-bit BPB value is used, the high byte is ignored.
For harddisks, the 512 bytes per sector and 1 reserved sector
assumption above is only maintained if the harddisk's format
is unknown. The number of hidden sectors, however, always comes
from the corresponding partition table, while the total count
of sectors is retrieved from the BPB - unless that value is
zero in both, the old 16-bit and the new 32-bit entries.
For the count of sectors per track and heads, it queries
INT 13h/AH=08h and ignores the BPB entries. If the disk
contains an extended BPB with a zero count of FATs under
DOS 4.0+, the disk's BPB will be adopted without overrides.
The exception under DOS 5.0+ is that the count of sectors
is still read from the partition table if the corresponding
BPB value is zero.
I assume that there are many more special cases and variations
under older or newer DOS versions still to track down.
For example the sector size limitation (at least since MS-DOS/
PC DOS 5.0+) seems to have been relaxed in MS-DOS 7.10 - 8.0,
which support sector sizes 32..2048 bytes now, just like
DR DOS 6.0 - 7.05 do. Even DR DOS 3.31 - 5.0 already supported
128..2048 bytes, the least. The Windows 95/98/SE/ME GUI,
however, still only supports sector sizes 512..2048 bytes.
Well, I mentioned a few more examples in my original post.
Anyway, please understand that this description is still
very incomplete.
On the other hand, while DR-DOS uses very similarly staggered
mechanisms in order to determine a disks characteristics,
it never uses the OEM label and also does not impose most
of the other hardwired restrictions on the disk geometry.
Greetings,
Maxim S. Shatskih
> OEM label as well, but I am not sure about it.
You can be sure, since the source is such.
NTFS, on the other hand, requires the OEM label to be "NTFS ",
together with the requirement that most FAT BPB fields be zeros -
except SectorsPerCluster and BytesPerSector (both seem to be used by
NTFS too).
Look at NtfsIsBootSectorNtfs routine in disassembly.
In fact, this field influences nothing, the FAT layout is not
dependent on it. So, usage of this field can even be some
anti-competitive trick :-)
Max
Maxim S. Shatskih
> NT's filesystem driver, I'm not sure, if FASTFAT would be
> the right driver level to look at for any dealings with
> OEM labels.
Surely the right level. Partition content is the deal of FSD, not of
any lower driver. Lower drivers make no assumption on this. You can
write, for instance, ext2 FSD for NT. Any OEM labels in ext2's
bootsector?
On the other hand, MBR is managed in Disk.sys+ClassPnP.sys, and - more
exactly - the parsing is done in IoReadPartitionTable call in hal.dll
(so, you need another HAL for disks with another MBR structure). BTW,
I consider Linux to be much smarter in terms of partition table
handling.
This is so for hard disks, floppies are other song, since there are
several kinds of floppies (from 360KB to 2.88MB), each requiring its
own controller setup. For floppy, the FAT BPB is interesting not only
to FAT, but for floppy driver itself, to set up the controller
properly for this kind of floppies.
> Anyway, I would look at lower level drivers in order to find
> possible traces of OEM label special cases.
The hard disk stack does not use it. I can check the floppy driver
source to be sure.
> I would certainly be interested to learn how the NT family
> handles volume tracking. Just use a CRC over the boot sector
> and FAT or something similarly more reasonable than destroying
> the OEM label?
Ohhh... I know this in the details - had to study this in one
real-world project.
Volume tracking is handled by both FSD and the underlying disk
driver.The only job of the disk driver is:
- if there is a chance that media was changed (it is the particular
SCSI unit attention condition for main storage stack, where ATA
emulates SCSI, and, for floppies - IIRC the timeout or the change line
hardware if the chip supports it) - then the driver must set some bit
in its device object. It will later be reset by the FSD, and only by
the FSD.
- when the bit is set, all IO requests to the driver must be failed,
the failure code depends on whether there is any media in the drive.
If there is - then STATUS_VERIFY_REQUIRED is a failure code.
- some IO requests are marked as "verify override". These requests
must not be failed with STATUS_VERIFY_REQUIRED and must honestly read
the media.
- also the block device driver must maintain the counter of media
changes.
- there is IOCTL_DISK_CHECK_VERIFY, which either fails (if no media),
or returns the aforementioned media change count.
This is all for block device driver. For FSD, the situation is much
harder.
- on mount path, the FSD always resets the media change bit in the
block device object, and all mount reads are marked as "verify
override". Also the FSD extracts the media change count and stores it
to the volume structure.
- the FSD checks for media change by IOCTL_DISK_CHECK_VERIFY on any
file open.
- if that check fails, or if any IO request issued by the FSD is
failed with STATUS_VERIFY_REQUIRED - then the verify procedure is
initiated by IoVerifyVolume call to the IO subsystem, which in turn
calls the verify path of the FSD.
- the verify path calls IOCTL_DISK_CHECK_VERIFY again, and, if the
media change count have changed - starts verification reads from the
disk with "verify override".
- the data from the possibly new media is compared to the current
volume structure. This comparison is FS-dependent for sure, FASTFAT
compares boot sector's serial number + the whole BPB + the volume
label in the root directory. Surely, the first check is whether the
new media has a sane FAT boot sector - if not, verify fails
immediately.
- if verify fails, then the FSD marks the volume is "not mounted".
This means - any attempts to initiate IO on this volume's files will
fail. Nevertheless, the files are still open, and the caches are
preserved (except the FAT cache - FASTFAT tears down the FAT table
caches and block allocation engine in this state). The "not mounted"
volume dies when all its files are closed.
- when failed verify is returned to the IO subsystem, it marks this
block device as not mounted at all. Any subsequent file opens on this
disk will result in new mount triggered. So, it is possible to have
several FS volumes per block device - one mounted, another "not
mounted".
- the last thing. Mount path has a hack. If there are "not mounted"
volume records, it tries to find the one among them which matches the
metadata (same comparison as on verify). If it is found - it is
"remounted" back, and becomes a mounted FS volume for the block
device. FASTFAT also restarts the allocation engine on remount by
re-reading the FAT table to the cache and rebuilding the allocation
bitmap. Don't remember off-head whether the MCBs (in-memory runlists)
for open files are rebuilt.
- so, if you eject the floppy, then insert another floppy, and then
re-insert the first floppy - the first volume is remounted, and the
work with all open files on it continues.
OEM label is not used for this. Only the boot sector's serial number +
BPB + the volume label.
CDFS does the same, and even NTFS too, though they have another
comparison algorithms.
> All the physical UDSC geometry data necessary for the low-
> level disk driver in the DOS BIOS should be readily available
> in the boot sector's BPB.
Even for hard disks? For hard disk, it is either in CMOS setup or is
asked from the drive itself by IDENTIFY or READ CAPACITY (OK, the
latter returns the LBA block count, but CHS can be emulated by the
BIOS in some silly way, or even asked by MODE SENSE from the rigid
geometry page - to be not dependent on BIOS translation mode).
> actually required for DOS as long as the corresponding block
> device driver is capable of returning a valid BPB structure
> during init and after media change
So, for DOS, even the block-level hard disk driver knows the BPBs?
> (That's also, why NT's FASTFAT driver might be the wrong
> place to look at when searching for code dealing with
> OEM labels, in case it exists.)
Maybe. Anyway FASTFAT reads any FAT floppies, and thus the OEM label
is not necessary for work. Thus the suspect that it serves
anti-competitive means only.
> Unfortunately, there is not one single BPB format. While most
> of the entries did not change over all the years since DOS 2.0+,
> some entries differ between various OEMs and DOS versions,
> so the code interpreting the BPB must deal with a number of
> special cases in order to work with all formats.
...as does FASTFAT :-) it has some comments on "older DOS volumes".
Some older DOSes used buggy FAT layout, and have workarounds for that.
FASTFAT re-implements them.
Also note - FASTFAT never uses the information from FAT32's FsInfo
sector. It only writes to it to mark the volume as dirty, and rewrites
the whole sector. It never uses any FsInfo data, and writes to it only
for compatibility with Win9x.
Also - the "volume dirty" bit is implemented differently for FASTFAT
and for Win9x.
> DOS 7.10+ FAT32 BPBs can be easily detected as well (not on
> floppies, anyway),
SectorsPerFat == 0 is the condition, at least the one used by FASTFAT.
After checking for this, the BPB is treated as FAT32 BPB.
> mark the old entries as invalid, when the new ones should be
> used instead.
Sectors and LargeSectors are such, SectorsPerFat and
LargeSectorsPerFat are also - the second also means that the volume is
FAT32.
> They are interpreted only by the bootstrap loader in the boot
> sector (and the corresponding FORMAT/SYS/FDISK utilities),
> and impose no extra complexity on DOS.
BTW - how the boot sector knows the _partition_ it is on? The drive
number is passed via DL register, and what about the partition? I
expect that the FORMAT and SYS tools must write the partition start
CHS values to the boot sector when writing it. Is it wrong?
It's much easier for NTFS which reserves around 16 first sectors for
bootstrap, then MFT bitmap follows, then MFT itself.
> As a plausibility test, the DOS 3.3+ BIOS will look at the
> first byte(s) of the boot sector to be either E9h (JMP NEAR),
> or EBh,xxh,90h
FASTFAT does the same. The first 3 bytes must be the x86 JMP opcode,
or FASTFAT will not mount.
> The exception under DOS 5.0+ is that the count of sectors
> is still read from the partition table if the corresponding
> BPB value is zero.
Some DOS versions used buggy FAT layout, where FAT is too short to
describe the whole number of clusters. To work with these volumes,
FASTFAT computes "ClustersDescribableByFat" value and uses it instead.
> 128..2048 bytes, the least. The Windows 95/98/SE/ME GUI,
> however, still only supports sector sizes 512..2048 bytes.
FASTFAT supports any power of 2 from 128 to 4096 inclusively.
NT in general is not so able of handling sector sizes > 4096 due to
tight link between VM and file cache.
NTFS seem to check the BPB's sector size to match the IOCTL's one
(reported by the block device driver). FASTFAT IIRC does not require
this.
Max
Jonathan de Boyne Pollard
B> that used the same bytes for the same purpose, but wasn't FAT.
B> However, the maker of this OS clearly marked the partition type as
B> being a different value (which one doesn't matter, but one that is
B> definitively NOT any of the FAT family).
B> On this theoretical filesystem, FASTFAT would read this as a
B> FAT partition. Correct? Of course, that would be totally incorrect,
B> and it would be FASTFAT's fault, since Microsoft decided to abandon
B> the standard they created years ago (and that others still use).
This is not a theoretical situation. It actually happened. Yes, it was
incorrect. Yes, it was FASTFAT's fault. IBM's Boot Manager lives in a
type 0x0A volume. It contains something resembling an EBPB in its
first sector. It most definitely is not a FAT-format volume. Windows
NT 2000's filesystem recognition code, which was broken when that
operating system was released, erroneously ignored the partition type,
saw the BPB, thought that the partition was FAT, and proceeded to run
the FAT CHKDSK on it at boot time. Of course, since sectors 1
/et seq./ of a Boot Manager partition do not _contain_ a File
Allocation Table, Windows NT 2000's CHKDSK completely scrambled
what it _did_ contain in the name of "repair".
Why did this happen ? One might suppose this to be an example of
Microsoft's predatory and malicious design tactics. Every time that
Windows NT 2000 booted, it would actively corrupt and destroy an IBM
Boot Manager partition. However, the adage about never attributing to
malice what can be explained by stupidity applies here. Maxim is fond
of uncritically parroting "a member of the Windows NT filesystem team"
who foolishly said that partition types are irrelevant and should be
ignored. The fact that this downright _wrong_ advice is bandied about
by "a member of the Windows NT filesystem team" tells us that the level
of knowledge about how MBR-style disc partitioning actually works is
surprisingly poor amongst those whose job it is to actually write and
maintain the code in Microsoft's operating systems that deals with it.
Therefore we should not be surprised when, as in the aforementioned
case, Windows NT gets it wrong.
It is ironic that the very Knowledgebase article (Q265003) wherein
Microsoft describes fixing this Windows NT 2000 problem provides further
fuel for the theory that the Windows NT filesystems people don't know
very much about filesystems. Microsoft's Knowledgebase article states
that an IBM Boot Manager volume contains one FAT rather than two. This
is incorrect. An IBM Boot Manager volume is _not a FAT-format volume at
all_ and contains no FATs whatever.