Re: Memory behaves strangely between 0x500 and 0x7C00, but not in Bochs (x86)
Rod Pemberton
"Stéphane thibaud" <snth...@gmail.com> wrote in message
news:eab1583a-4d46-4469...@b1g2000hsg.googlegroups.com...
> I have written a lot of code that is loaded from a disc at address
> 0x500, this code worked like it should most of the time but now I
> encountered an error that seems to have to do with corrupted memory:
>
> mov ecx,ModuleData ;ECX contains the offset in ModuleData
>
> add ecx,0x00000008
> mov eax,dword[ecx]
> cmp eax,0 ;If there are no module dependencies
> jz NEAR .checkGroupDeps ;skip module checking
>
> Where ModuleData resides at 0x1274
>
> The problem is: in Bochs it jumps (like it should do in this case) but
> on real hardware it doesn't (with Microsoft's Virtual PC - it's the
> same story... it doesn't jump either). The value's aren't simply
> zeroed out in Bochs... I checked that the data on my CD was really
> loaded by looking at the values around this dword. Does anyone know
> what could reside at these addresses... after the IDT and BIOS-data
> that should go up to 0x500? Remember that most of my code loaded at
> 0x500 does work... there's only this problem at 0x1274 (or maybe more
> to come...).
This or a similar problem (loading to 0x500 and failure) has come up a
couple times lately (on alt.os.development). IMO, the issue(s) were never
identified.
This is what I'm aware of:
0-0x3FF IVT - Interrupt Vector Table
0x400-0x4FF BDA - Bios Data Area
0x500-0x5FF MS-DOS parameter area
AFAICT, nothing (unless you're using DOS) should be there...
Rod Pemberton
Stéphane thibaud
> "Stéphane thibaud" <snthib...@gmail.com> wrote in message
>
> - Tekst uit oorspronkelijk bericht weergeven -
That sounds strange... is it some kind of widely accepted bug? I think
I'll just move my OS to 0x7C00 then and see if the problem still
occurs. It seems like a waste of bytes to me... from a CD (El Torito)
there's no problem in using other sectors than 0x7C00, so I thought
I'd be as conservative as possible. Too bad... ;) I'll post again when
I tried 0x7C00 to report if the issue was resolved.
Regards,
Stéphane Thibaud
H. Peter Anvin
>
> This or a similar problem (loading to 0x500 and failure) has come up a
> couple times lately (on alt.os.development). IMO, the issue(s) were never
> identified.
>
> This is what I'm aware of:
> 0-0x3FF IVT - Interrupt Vector Table
> 0x400-0x4FF BDA - Bios Data Area
> 0x500-0x5FF MS-DOS parameter area
>
> AFAICT, nothing (unless you're using DOS) should be there...
>
Actually the byte at 0x500 is used by the BIOS print screen routine
(which to a first order is never used, but still...)
0x600 is the standard location for having a copy of the MBR, so for
something loaded from a partition, you would normally expect DS:SI to
point somewhere between 0x600 and 0x7e0 (0x7be is probably the most
common location.)
Keep in mind that you don't have any guarantee where the stack is set
up, so you want to set the stack pointer up explicitly. This is
normally done by setting SS:SP to 0:0x7c00 and letting it grow down from
the boot sector.
Oh, and watch out for the occational buggy BIOS or chainloader which
jumps to 07C0:0000 instead of 0000:7C00.
-hpa
James Harris
...
> > This or a similar problem (loading to 0x500 and failure) has come up a
> > couple times lately (on alt.os.development). IMO, the issue(s) were never
> > identified.
>
> > This is what I'm aware of:
> > 0-0x3FF IVT - Interrupt Vector Table
> > 0x400-0x4FF BDA - Bios Data Area
> > 0x500-0x5FF MS-DOS parameter area
>
> > AFAICT, nothing (unless you're using DOS) should be there...
>
> > Rod Pemberton- Tekst uit oorspronkelijk bericht niet weergeven -
>
> > - Tekst uit oorspronkelijk bericht weergeven -
>
> That sounds strange... is it some kind of widely accepted bug? I think
> I'll just move my OS to 0x7C00 then and see if the problem still
> occurs. It seems like a waste of bytes to me... from a CD (El Torito)
> there's no problem in using other sectors than 0x7C00, so I thought
> I'd be as conservative as possible. Too bad... ;) I'll post again when
> I tried 0x7C00 to report if the issue was resolved.
It's not a bug per se. It could be a lack of knowledge. How about
moving your code up a little and see how it works? From Ralf Brown's
interrupt list (it has a section on memory as well as interrupts)
0x500 - print screen status
0x501 - NEC PC-9800 series - Screen mode
with MS-DOS use starting at 0x504. Note how the memory just above
0x4ff is reported in these:
http://www.kryslix.com/nsfaq/Q.5.html
http://heim.ifi.uio.no/~stanisls/helppc/bios_data_area.html
http://en.wikipedia.org/wiki/Master_boot_record
I'm not saying any of them are right, by the way, and the address you
mention (0x127c isn't it?) seems a long way off. If anything it sounds
more likely that normal PC's arbitrary memory contents comes in to
play (as opposed to Bochs' zeroed memory) perhaps an effect from
elsewhere in the code than where the fault appeared.
Why not try moving your code up only a few bytes. If that doesn't work
move up by a larger amount, or just past the questionable addresses?
If there is an undocumented BIOS-used address it would be good to
track it down rather than just avoiding it.
Stéphane thibaud
>
> I'm not saying any of them are right, by the way, and the address you
> mention (0x127c isn't it?) seems a long way off. If anything it sounds
> more likely that normal PC's arbitrary memory contents comes in to
> play (as opposed to Bochs' zeroed memory) perhaps an effect from
> elsewhere in the code than where the fault appeared.
>
> Why not try moving your code up only a few bytes. If that doesn't work
> move up by a larger amount, or just past the questionable addresses?
> If there is an undocumented BIOS-used address it would be good to
>
> - Tekst uit oorspronkelijk bericht weergeven -
I moved my whole code to 0x7C00 to check if there were still
problems... and yes, there were. This means that the error I am
experiencing is one of my own. No bugs or strange behaviour of memory.
I'm sorry for that! Now I have to find out what in my code behaves
differently on real hardware. I didn't set up all the Vectors in the
IVT... Could it be that that causes problems (perhaps an interrupt, by
a timer for example, jumps to a location where it shouldn't and
corrupts memory with wrong instructions?) ? I'll look further into it
tomorrow. This will be quite a challenge...
Regards,
Stéphane Thibaud
Wolfgang Kern
Rod Pemberton answered Stéphane thibaud:
...
> This or a similar problem (loading to 0x500 and failure) has come up a
> couple times lately (on alt.os.development). IMO, the issue(s) were never
> identified.
> This is what I'm aware of:
> 0-0x3FF IVT - Interrupt Vector Table
> 0x400-0x4FF BDA - Bios Data Area
> 0x500-0x5FF MS-DOS parameter area
> AFAICT, nothing (unless you're using DOS) should be there...
I have 0x500...0x5FF reserved for BIOS Data-Interface
and DPB 'Disk Parameter Block' at 0x522 in my notes.
I saw the BIOS using 0x522.. for FD-parameteres.
Even DOS6.00 temporary uses 0x500..0x6FF as sector-buffer
keeping the MBR's partition info at the end, it restores
at least the FD parameters after it and installs itself to
0x0070:0000 (0:0x0700).
Can't remember the exact reason for why my OS's 2nd stage
bootloader (32 Kb) goes to 0x7000:0000, perhaps an relict from
early version where I kept DOS6.00 in addition to my sytem.
__
wolfgang
James Harris
>
> > I'm not saying any of them are right, by the way, and the address you
> > mention (0x127c isn't it?) seems a long way off. If anything it sounds
> > more likely that normal PC's arbitrary memory contents comes in to
> > play (as opposed to Bochs' zeroed memory) perhaps an effect from
> > elsewhere in the code than where the fault appeared.
>
> > Why not try moving your code up only a few bytes. If that doesn't work
> > move up by a larger amount, or just past the questionable addresses?
> > If there is an undocumented BIOS-used address it would be good to
> > track it down rather than just avoiding it.- Tekst uit oorspronkelijk bericht niet weergeven -
>
> > - Tekst uit oorspronkelijk bericht weergeven -
>
> I moved my whole code to 0x7C00 to check if there were still
> problems... and yes, there were. This means that the error I am
> experiencing is one of my own. No bugs or strange behaviour of memory.
> I'm sorry for that! Now I have to find out what in my code behaves
> differently on real hardware. I didn't set up all the Vectors in the
> IVT... Could it be that that causes problems (perhaps an interrupt, by
> a timer for example, jumps to a location where it shouldn't and
> corrupts memory with wrong instructions?) ? I'll look further into it
> tomorrow. This will be quite a challenge...
Of course, 0x7c00 is where the boot sector will have been loaded by
the BIOS. If you are going to put something else there as you know you
will have to free the space up first.
The IVT is a good call. Someone else here found similar recently. I
can't find the post but IIRC he was getting an interrupt from the PIC
on a real PC which did not happen on Bochs.
Stéphane thibaud
>
> - Tekst uit oorspronkelijk bericht weergeven -
I'm creating my own OS so there's no problem in using 0x7C00... I can
set the Bootable CD to load it there.
But my problem just got weirder this morning: I wrote a function that
prints the memory location's value and put it at the very start of my
code... in Bochs this location contains zero as it should... but on
real Hardware it contains all ones (binary). It has nothing to do with
the area because wether I load my code at 0x0500 or 0x7C00... the
results are the same! I'm really out of ammo now...
The IVT wasn't the problem either, because I also put a 'cli'
instruction at the start of my code to see if it made any difference
(it didn't).
Any ideas after all this?
Alexei A. Frounze
Uninitialized pointer, index, FLAGS.DF, stack pointer, segment
register?
Alex
Wolfgang Kern
Stéphane thibaud wrote:
...
<quote>
But my problem just got weirder this morning: I wrote a function that
prints the memory location's value and put it at the very start of my
code... in Bochs this location contains zero as it should... but on
real Hardware it contains all ones (binary). It has nothing to do with
the area because wether I load my code at 0x0500 or 0x7C00... the
results are the same! I'm really out of ammo now...
The IVT wasn't the problem either, because I also put a 'cli'
instruction at the start of my code to see if it made any difference
(it didn't).
Any ideas after all this?
</q>
Yeah, Bochs may not always be aware of later used hardware :)
What is the exact memory location you wanted to display ?
if it's A000:0000 and you are still in textmode it will read all 1's,
same if it's above the VBIOS-ROM until BIOS-ROM-code acually starts
(the usually unused range from C8000 or D0000 to DFFFF or EFFFF).
Perhaps you stumbled over a value<->pointer misinterpretation
or as Alexei already said a segment register issue ...
__
wolfgang
Stéphane thibaud
>
> - Tekst uit oorspronkelijk bericht weergeven -
I looked further into it and noticed that the bytes around it were all
ones too (on real hardware). This leads me to thinking that a sector
of my CD wasn't loaded. Maybe the El Torito standard is implemented
differenltly on hardware than in Bochs... It won''t be easy to find
out what exactly causes this trouble. I already checked the most
obvious reasons, but I must say that my implementation of the 9660
standard isn't exactly perfect.
Regards,
Stéphane
Alexei A. Frounze
If that's the case, do a simple thing. Make a bootable floppy image
with your code. Try that one wherever you can (VMWare, VPC, VS, Bochs,
DOSEMU, DosBox, etc) and see if it works there. Then use wellknown
mkisofs program to make an ISO file for a CD out of your floppy image.
Something like this:
mkisofs -b floppy.img -v -r -l -L -o cd.iso DirectoryWithFilesForTheCD
Worked for me in the virtualized world and on the bare hardware.
Alex
Stéphane thibaud
Oh my god! I've found it! You were right, it was a segment register! I
left 'ds' uninititialized because I thought the segment registers
would be set to zero on reboot. It appears this is only the case for
the code segment register. I am so glad that this is over... Thank you
guys! I'd buy you a beer if I'd encounter you someday!
Best Regards,
Stéphane
Stéphane thibaud
>
> - Tekst uit oorspronkelijk bericht weergeven -
Hmm... I thought it was over... but not quite yet... It now jump there
as it should, but then another difference between Bochs and the real
world comes to the surface:
call ecx (where I verfied that ecx has the same value on the machine
as in Bochs) does only work on Bochs.
Both the machine and Bochs have the right data loaded at ecx... Looks
like another segment register-issue... 'CS' can't be changed right?
Or can it?
Stéphane thibaud
>
> - Tekst uit oorspronkelijk bericht weergeven -
Ehm... I found that last little issue already, cs was modifiable by
doing a jump with the segment specified.
Thanks again! This thread can be closed...
Sincerely,
Stéphane
H. Peter Anvin
>
> But my problem just got weirder this morning: I wrote a function that
> prints the memory location's value and put it at the very start of my
> code... in Bochs this location contains zero as it should... but on
> real Hardware it contains all ones (binary). It has nothing to do with
> the area because wether I load my code at 0x0500 or 0x7C00... the
> results are the same! I'm really out of ammo now...
>
There are no guarantees whatsoever what BIOS will do with this area.
The PMM specification recommends zeroing, but far from all BIOSes follow
that recommendation. You should consider any memory you haven't
initialized yourself as random garbage.
-hpa
H. Peter Anvin
>
> Oh my god! I've found it! You were right, it was a segment register! I
> left 'ds' uninititialized because I thought the segment registers
> would be set to zero on reboot. It appears this is only the case for
> the code segment register. I am so glad that this is over... Thank you
> guys! I'd buy you a beer if I'd encounter you someday!
>
The *only* thing you can trust is the combined value of CS and IP and to
have the disk number in DL(*). You can't even assume it will have the
correct 0000:7C00; I have seen BIOSes which will jump to 07C0:0000 instead.
When booted from a hard disk partition by an MBR, DS:SI is supposed to
point to the partition information.
The BIOS is *supposed* to point ES:DI to the $PnP information, if
present; I wouldn't trust that much especially in the presence of
chainloaders and so forth; it also conflicts with the initial conditions
for a PXE stack.
In general, BIOSes should be treated more or less as toxic waste: don't
rely on anything you can avoid relying on.
-hpa
(*) Actually, quite a few El Torito BIOSes get that wrong, too.
Alexei A. Frounze
> On 14 jun, 17:35, Stéphane thibaud <snthib...@gmail.com> wrote:
> > Both the machine and Bochs have the right data loaded at ecx... Looks
> > like another segment register-issue... 'CS' can't be changed right?
> > Or can it?- Tekst uit oorspronkelijk bericht niet weergeven -
>
> > - Tekst uit oorspronkelijk bericht weergeven -
>
> Ehm... I found that last little issue already, cs was modifiable by
> doing a jump with the segment specified.
No, doing a far jump.
Alex
H. Peter Anvin
>> Ehm... I found that last little issue already, cs was modifiable by
>> doing a jump with the segment specified.
>
> No, doing a far jump.
What other kind of jump do you have that specify a segment?
-hpa
Alexei A. Frounze
A near jump to a location stored in memory and referenced through a
non-default data segment, that is:
CS: JMP Ev
The segment is specified.
Alex
Alexei A. Frounze
It's even funnier, the conditional jumps may be prefixed with CS: or
DS: with the intent of letting the CPU know how (un)likely it to be
taken.
Alex
Rod Pemberton
news:48540B9A...@zytor.com...
> The *only* thing you can trust is the combined value of CS and IP and to
> have the disk number in DL(*). You can't even assume it will have the
> correct 0000:7C00; I have seen BIOSes which will jump to 07C0:0000
instead.
We've heard of those, but IIRC we weren't able to track down info on any
specific machines... only rumor that an old Compaq was involved...
Rod Pemberton
Rod Pemberton
> point to the partition information.
>
Where is this doc'd? I think Ben was using this in his loader... APM? EDD
1.0/3.0? ACPI? BBS? PMM?
Rod Pemberton
Rod Pemberton
> rely on anything you can avoid relying on.
>
BIOSes are known toxins, eh? So, that makes SMM unknown toxins...?
But, do you have a bit of a conflicting ideology? You also stated this
previously:
> Note, also, that if your BIOS supports it, INT 15h, AX=2401h is the best
> way to enable A20.
Assuming INT 15h is not "toxic waste" and reliable - since you did say
so.... - so, what makes the BIOS more reliable for A20 than the keyboard
controller or PS/2 gate? How do you think a typical BIOS enables A20?
(I.e., are you implying that many computers add a third circuit just for the
BIOS?)
Rod Pemberton
Mike Gonta
> You can't even assume it will have the correct 0000:7C00; I have seen
> BIOSes which will jump to 07C0:0000 instead.
Actually the "El Torito" Bootable CD-ROM Format Specification only provides
for a load segment and not a segment:offset.
http://http://www.phoenix.com/NR/rdonlyres/98D3219C-9CC9-4DF5-B496-A286D893E36A/0/specscdrom.pdf
page 10:
"This is the load segment for the initial boot image. If this
value is 0 the system will use the traditional segment of 7C0.
If this value is non-zero the system will use the specified segment."
Mike Gonta
look and see - many look but few see
H. Peter Anvin
> "H. Peter Anvin" wrote:
>
>> You can't even assume it will have the correct 0000:7C00; I have seen
>> BIOSes which will jump to 07C0:0000 instead.
>
> Actually the "El Torito" Bootable CD-ROM Format Specification only provides
> for a load segment and not a segment:offset.
>
> http://http://www.phoenix.com/NR/rdonlyres/98D3219C-9CC9-4DF5-B496-A286D893E36A/0/specscdrom.pdf
>
> page 10:
> "This is the load segment for the initial boot image. If this
> value is 0 the system will use the traditional segment of 7C0.
> If this value is non-zero the system will use the specified segment."
>
This is a bug in the El Torito specification, because the "traditional"
address is 0000:7C00. Whomever wrote the spec was unclear, and as a
result people implemented it incorrectly.
There are a number of boot sectors in the wild which will not work
correctly loaded at 07C0:0000.
-hpa
H. Peter Anvin
There were a series of Intel-branded AMI BIOSes which exhibited this bug
in the late 90's. That's the only one I know of for sure.
-hpa
H. Peter Anvin
I do have a sometimes conflicting ideology, at least "on the surface of
it." However, there is method to this particular madness -- the various
options for enabling A20 are implemented poorly by a number of the
device vendors, and yes, there are chips out there that have *neither*
the KBC nor port 92h but a totally different method -- AMD Elan is an
example. In some other cases, the traditional GPIOs were abused for
something totally unrelated -- like Olivetti laptops which would switch
off the screen in response to port 92h.
Then there are funny BIOS interactions with one device or another,
because various code in the BIOS made assumptions about which mode you
would use -- different for different BIOSes, of course.
Thus, *in practice*:
- if A20 is already enabled, do nothing.
- if INT 15h, AX=2401h is supported, use it first.
... seems to work best.
-hpa
Mike Gonta
> Mike Gonta wrote:
> > "H. Peter Anvin" wrote:
>
> >> You can't even assume it will have the correct 0000:7C00; I have seen
> >> BIOSes which will jump to 07C0:0000 instead.
>
> > Actually the "El Torito" Bootable CD-ROM Format Specification only provides
> > for a load segment and not a segment:offset.
>
>
> > page 10:
> > "This is the load segment for the initial boot image. If this
> > value is 0 the system will use the traditional segment of 7C0.
> > If this value is non-zero the system will use the specified segment."
>
> This is a bug in the El Torito specification, because the "traditional"
> address is 0000:7C00. Whomever wrote the spec was unclear, and as a
> result people implemented it incorrectly.
>
> There are a number of boot sectors in the wild which will not work
> correctly loaded at 07C0:0000.
Actually, the "traditional" address is 7C00h; the segment:offset
notation
is a function of the microprocessor in real mode.
I would say that the bug is in those boot sectors whose authors
assumed
certain values for the segment registers cs and/or ds.
The simple method is to access data through the default segment
register ds
after initializing it to zero and org (originating) the code at
absolute address
7C00h.
H. Peter Anvin
>>
>> There are a number of boot sectors in the wild which will not work
>> correctly loaded at 07C0:0000.
>
> Actually, the "traditional" address is 7C00h; the segment:offset
> notation is a function of the microprocessor in real mode.
> I would say that the bug is in those boot sectors whose authors
> assumed certain values for the segment registers cs and/or ds.
You would, but you would be wrong.
CS:IP = 0000:7C00h is the specification, dating back to the original IBM
PC. Violating is is a bug.
However, defensive programming is to be liberal in what you receive,
conservative in what you send. In other words:
- BIOSes, MBR and chainloaders should invoke boot sectors at 0000:7C00h.
- Boot sectors shouldn't assume that BIOSes aren't broken.
> The simple method is to access data through the default segment
> register ds after initializing it to zero and org (originating) the code at
> absolute address 7C00h.
... then doing a far jump to normalize CS:IP at your first opportunity.
Don't rely on specific values in CS or IP until then.
-hpa
Mike Gonta
> Mike Gonta wrote:
>>>
>>> There are a number of boot sectors in the wild which will not work
>>> correctly loaded at 07C0:0000.
>>
>> Actually, the "traditional" address is 7C00h; the segment:offset
>> notation is a function of the microprocessor in real mode.
>> I would say that the bug is in those boot sectors whose authors
>> assumed certain values for the segment registers cs and/or ds.
>
> You would, but you would be wrong.
Proof please.
> CS:IP = 0000:7C00h is the specification, dating back to the original IBM
> PC.
The CS:IP value results from a programming methodology.
IBM chose to do a far jmp to absolute address 7C00h which resulted
in 0000:7C00h. This is not a specification, especially when much boot
sector code had already written prior to the release of the IBM PC.
> Violating is is a bug.
Not a bug, maybe a small IBM PC incompatibility issue.