Unreal mode
Safronov's family
register. Hence, we can use interrupts of BIOS (like in real mode) and use 4
gigabytes of memory (like in protected mode) at the same time! Is it a
bug???
Vinzent Hoefler
[Unreal mode]
>Is it a bug???
No. This is quite clearly a feature.
Vinzent.
Herman Dullink
Although it is meant for 16-bit 8086 code, it doesn't mean you can't do
32-bit
in real mode. It's not done very much as the native protected mode is a
better environment for your (multitasking) applications.
Herman
Dennis Bliefernicht
I'd call it an anormaly. The CPU caches the loaded contents of the GDT
(base, limit, access) due to performance. You can't clear the caches
when jumping back to real mode because the code has to run anywhere.
So the user has to and if he doesn't, unreal :)
Anyway this is online useful in a limited way because the only
registers that can be loaded with this are FS and GS, because all the
other ones will be used by the BIOS (and I wouldn't be too sure on
FS/GS anymore). So you could address 4GiB of data but you won't be
able to run code in that limit anyway :-/
Vinzent Hoefler
[Unreal mode]
>Anyway this is online useful in a limited way because the only
>registers that can be loaded with this are FS and GS, because all the
>other ones will be used by the BIOS
So what? As long as the BIOS doesn't switch to protected mode and
resets the limits it doesn't hurt anybody. And because unreal mode is
quite incompatible with anything else anyway, the BIOS is of minor
concern. At least I wouldn't see a sense in calling the memory copy
functions of int 15h (a good candidate for switching to PM inbetween)
when I am already in Unreal mode. Well, some T&SR's *could* make use
of that, but even for that there is a perfectly safe solution:
Install your own exception handler and in the rare cases, it could
become active just do the Unreal mode initialization again. So did
Microsoft's HIMEM.SYS (yes, it uses Unreal mode, amazing, isn't it?).
One day I had the source code for some version (2.06, IIRC), perhaps
it is still flying around on the net, let's see...
Yes(!) it is, you can get it at:
<http://www.himinbi.org/~will/fileformat.virtualave.net/programming/himem/>.
>So you could address 4GiB of data but you won't be
>able to run code in that limit anyway :-/
Well, of course. But this isn't an BIOS issue, this is mainly because
EIP is trimmed down to 16 bits on jumps and such, so you would have a
hard time trying to jump outside of the 64K limit.
Vinzent.
--
Parents strongly cautioned -- this posting is intended for mature
audiences over 18. It may contain some material that many parents
would not find suitable for children and may include intense violence,
sexual situations, coarse language and suggestive dialogue.
Ben Peddell
> Dennis Bliefernicht wrote:
>
> [Unreal mode]
>
>
>>So you could address 4GiB of data but you won't be
>>able to run code in that limit anyway :-/
>
>
> Well, of course. But this isn't an BIOS issue, this is mainly because
> EIP is trimmed down to 16 bits on jumps and such, so you would have a
> hard time trying to jump outside of the 64K limit.
Actually, this is only true if the CPU was in 16-bit protected mode
before it dropped to real mode.
Anyway, even if you are in 16-bit mode, I think you'd be able to make
32-bit JMPs by prefixing them with the address size prefix.
Vinzent Hoefler
>Vinzent Hoefler wrote:
>> Dennis Bliefernicht wrote:
>>
>> [Unreal mode]
>>
>[snip]
>>
>>>So you could address 4GiB of data but you won't be
>>>able to run code in that limit anyway :-/
>>
>> Well, of course. But this isn't an BIOS issue, this is mainly because
>> EIP is trimmed down to 16 bits on jumps and such, so you would have a
>> hard time trying to jump outside of the 64K limit.
>
>Actually, this is only true if the CPU was in 16-bit protected mode
>before it dropped to real mode.
Well, yes, if we are talking about the CS-Limit.
>Anyway, even if you are in 16-bit mode, I think you'd be able to make
>32-bit JMPs by prefixing them with the address size prefix.
Yes, this is correct, you could do that explicitely. But from that
point on you probably break anything else. AFAICS, any Interrupt,
(16-Bit) call/return or (D)OS-Call would destroy your EIP. Quite hard
to write code that can handle that. ;)
Ross Ridge
[Unreal mode]
>Anyway this is online useful in a limited way because the only
>registers that can be loaded with this are FS and GS, because all the
>other ones will be used by the BIOS
Vinzent Hoefler <JeLlyFish...@gmx.net> wrote:
>So what? As long as the BIOS doesn't switch to protected mode and
>resets the limits it doesn't hurt anybody.
Any load of a segment register in real mode will reset the segments
limit to the normal 64K.
>Install your own exception handler and in the rare cases, it could
>become active just do the Unreal mode initialization again. So did
>Microsoft's HIMEM.SYS (yes, it uses Unreal mode, amazing, isn't it?).
Yes, Microsoft's HIMEM.SYS does evil things so you don't have to.
Be thankful there's no reason to use "unreal" mode yourself.
Ross Ridge
--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rri...@csclub.uwaterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/u/rridge/
db //
Korotkov Ivan
but it's base is initialized with FFFF0000, so it can access data beyond 1
MB limit while staying still in RM (and it does even - first instruction is
fetched from BIOS at FFFFFFF0). Though when BIOS BSP makes the first far
jump/call, this feature will be resetted.
Ivan
Korotkov Ivan
> become active just do the Unreal mode initialization again. So did
> Microsoft's HIMEM.SYS (yes, it uses Unreal mode, amazing, isn't it?).
It is. But how??? Data is accessed generally using DS, not FS/GS; how can
HIMEME enable EMS using Unreal mode?..
Ivan
Tom Thornhill
> Ben Peddell wrote:
>
> >Vinzent Hoefler wrote:
> >> Dennis Bliefernicht wrote:
> >>
> >> [Unreal mode]
> >>
> >[snip]
> >>
> >>>So you could address 4GiB of data but you won't be
> >>>able to run code in that limit anyway :-/
> >>
> >> Well, of course. But this isn't an BIOS issue, this is mainly because
> >> EIP is trimmed down to 16 bits on jumps and such, so you would have a
> >> hard time trying to jump outside of the 64K limit.
> >
> >Actually, this is only true if the CPU was in 16-bit protected mode
> >before it dropped to real mode.
>
> Well, yes, if we are talking about the CS-Limit.
>
> >Anyway, even if you are in 16-bit mode, I think you'd be able to make
> >32-bit JMPs by prefixing them with the address size prefix.
>
> Yes, this is correct, you could do that explicitely. But from that
> point on you probably break anything else. AFAICS, any Interrupt,
> (16-Bit) call/return or (D)OS-Call would destroy your EIP.
Yeah, anything that loads a segment register will set the limits back
to 64K. Pushing and poping them from the stack would have
the same effect.
Then, the next time you tried to access an offset greater than 64K
you'd get clobbered by a General Protection Fault.
> Quite hard to write code that can handle that. ;)
One thing you could try is to write a custom GP fault handler.
The idea being that you'd run your Unreal Mode code, which
relied on the segment limits being 4GB. Every so often the Bios or
an ISR would touch a segment register and set the limits back to
64K. Once control returned to your code, a GP fault would
happen soon enough, and then your GP fault handler would
fix up the limits and retry the instruction.
Incidentally, for data references, you'd be pretty safe - you could
use FS and GS, which most Bios calls / ISRs won't touch. This
scheme is really only necessary for code.
Mind you, it's odd that no one seems to use this technique - there's
a good chance that there is something fundamentally wrong with it.
Even if it worked, all the prefix bytes would slow things down - to
access 32 bit data in a segment above 64K, you'd need both
operand size and data size. If you want to use FS or GS, that's
another prefix. This will slow things down.
Having to handle a GP fault after every timer interrupt or Bios call
wouldn't be quick either.
--
Tom Thornhill
http://www.ridgecrop.demon.co.uk/
Herman Dullink
> >32-bit JMPs by prefixing them with the address size prefix.
> Yes, this is correct, you could do that explicitely. But from that
> point on you probably break anything else. AFAICS, any Interrupt,
> (16-Bit) call/return or (D)OS-Call would destroy your EIP. Quite hard
> to write code that can handle that. ;)
However, it _is_ possible to use 32-bit EIP in real mode. Similar to
'UnReal'
mode, you can load CS and SS with 32-bit descriptors before switching back
to Real mode.
Because virtually no software is written for 32-bit Real mode, you have to
write and handle everything yourself. Invoking BIOS means switching back
to 16-bit. 32-bit Protected mode is prefered because using V86 task is
easier than switching between 16-bit and 32-bit real mode :-)
H
Herman Dullink
> >resets the limits it doesn't hurt anybody.
> Any load of a segment register in real mode will reset the segments
> limit to the normal 64K.
a load of a segment register. Only when running in a Virtual 8086
the limit is reset to 64K during a load of a segment register.
H
Herman Dullink
> to 64K. Pushing and poping them from the stack would have
> the same effect.
affected by segment loads.
> Mind you, it's odd that no one seems to use this technique
Plenty applications. Himem uses this for XMS memory block
copies, to every DOS application using XMS use it (indirectly).
I use it for experiments where I need direct access to certain
addresses (e.g. memory mapped hardware devices).
Recently, I found that some hungarian scientists are using my
unreal code to monitor the effect of radiation on computer
memories ;-)
> Even if it worked, all the prefix bytes would slow things down
Only on some CPU's. Don't forget that the other way arround
is also true. 32-bit code handling 16-bit data also need prefix bytes.
H
Dennis Bliefernicht
even if it isn't the case. According to Intel Manual 3 16.1.1 (Address
Translation in Real-Address Mode) you'll get an exception everytime
you use a 32-bit address out of the range 0x0000-FFFF. So no use there.
Tom Thornhill
> > Yeah, anything that loads a segment register will set the limits back
> > to 64K. Pushing and poping them from the stack would have
> > the same effect.
> In a V86 task, yes. But in Real mode, only the base address is
> affected by segment loads.
Really? I thought even in Real Mode segment loads would set the
limit.
>
> > Mind you, it's odd that no one seems to use this technique
> Plenty applications. Himem uses this for XMS memory block
By "no one seems to use this technique", I meant "no one seems to
use a GP fault handler to fix up segment limits in unreal mode", rather
than no one uses Unreal Mode itself.
Still I think the time to use this sort of hack has passed.
>
> H
Ross Ridge
>resets the limits it doesn't hurt anybody.
Ross Ridge writes:
>Any load of a segment register in real mode will reset the segments
>limit to the normal 64K.
Herman Dullink <hdul...@hetnet.nl> wrote:
>Nope. In real mode, only the base address is affected during
>a load of a segment register. Only when running in a Virtual 8086
>the limit is reset to 64K during a load of a segment register.
Hmm... well, in practice it doesn't make much difference, since HIMEM.SYS
can't assume the processor is not in Virtual 8086 mode.
Herman Dullink
> Translation in Real-Address Mode) you'll get an exception everytime
> you use a 32-bit address out of the range 0x0000-FFFF. So no use there.
it's set to FFFFFF, 4K granular, it will not.
This is simply because the segmentation unit has no clue about real or
protected mode, it only knows about the descriptor caches.
If you search for 'Flat Real Mode' on the internet, you'll find some more
details.
H
Herman Dullink
> can't assume the processor is not in Virtual 8086 mode.
the CPU
is running in protected mode. If not, it'll use 'Real Big Mode' (aka Flat
Real
Mode, aka UnReal Mode), otherwise it'll use the 80286 compatible way,
which is calling INT 15h/AH=87h.
When running in a V86 task, the host will intercept this interrupt call,
and will do a simple REP MOVSD to emulate the interrupt.
H
Herman Dullink
> use a GP fault handler to fix up segment limits in unreal mode", rather
> than no one uses Unreal Mode itself.
If you look for 'Flat Real Mode' on the internet, you'll see an example
of an 'SOE' handler, and some examples in Turbo Pascal mixed with
Turbo Assembler using these techniques.
> Still I think the time to use this sort of hack has passed.
For normal applications, yes. This technique has no place in a multitasking
Windows/Unix platform.
For for special/research purposes however, people want full control
of the hardware, and no interference of other processes.
DOS + Flat Real/Real Big/UnReal is one of the options.
H
Dennis Bliefernicht
Well, looks like Intels manuals didn't want to cover unreal mode :)
Herman Dullink
more
> > details.
> Well, looks like Intels manuals didn't want to cover unreal mode :)
which is a 16-bit CPU.
Also, to Intel, it's normal Real Mode behaviour, but not with the default
(8086 compatible) values. Flat Real, Real Big and UnReal are no Intel terms,
but... System Management Mode is. SMM is meant for e.g. putting the
system to hibernate. In this mode, the code requires full access to all
hardware and memory. Looking at the states of the various units of
the CPU, you'll see that SMM is very like Flat Real/Real Big/UnReal.
H
Benjamin Kalytta
Yes as far as I know this was once a bug.
But many people used thios technique at this time, and intel decided
not to change the behaviour in future cpu's.
See following links for more information on SMM and URM:
SMM:
http://www.ddj.com/documents/s=945/ddj9701p/9701p.htm
http://www.ddj.com/documents/s=941/ddj9705o/9705o.htm
http://www.ddj.com/documents/s=943/ddj9703m/9703m.htm
URM:
http://x86.ddj.com/ddj/aug98/aug98.htm
Benjamin Kalytta
Vinzent Hoefler
>"Vinzent Hoefler" <JeLlyFish...@gmx.net> wrote in message news:bbq3mp$c881m$1...@ID-175126.news.dfncis.de...
>> Ben Peddell wrote:
>>
>> >> [Unreal mode]
>> >>
>> >Anyway, even if you are in 16-bit mode, I think you'd be able to make
>> >32-bit JMPs by prefixing them with the address size prefix.
>>
>> Yes, this is correct, you could do that explicitely. But from that
>> point on you probably break anything else. AFAICS, any Interrupt,
>> (16-Bit) call/return or (D)OS-Call would destroy your EIP.
>
>Yeah, anything that loads a segment register will set the limits back
>to 64K.
Nope. This is still a very common misunderstanding of how (un)real
mode works. Please look at some initialization code for unreal mode
and tell me why the segment limits don't get reset every time you load
the segment registers with zero then. The fact that unreal mode works
lies in that the processor does not touch the descriptor cache in real
mode. Once the limits are set, they remain set whatever you do with
your segment registers.
[32-Bit Unreal mode]
>Pushing and poping them from the stack would have
>the same effect.
No. The problem is that any 16-Bit instruction that fiddles around
with the IP (jmp, call, (i)ret, ...) only works on a 16-Bit EIP, so
the upper 16 bits are reset then, so that you wouldn't return to where
you wanted to, but instead to some code in the lower 64K of the
current codesegment.
>Even if it worked, all the prefix bytes would slow things down - to
>access 32 bit data in a segment above 64K, you'd need both
>operand size and data size. If you want to use FS or GS, that's
>another prefix. This will slow things down.
Yes and such it is a pretty neat idea to use DS/ES to save the
FS/GS-overrides and such is quite handy for a fast rep movsd
instruction.
>Having to handle a GP fault after every timer interrupt or Bios call
>wouldn't be quick either.
You wouldn't need to. Those are really very rare cases. This would
imply that the interrupt or BIOS has switched to protected mode and
reset the segment register limits upon return from it.
Vinzent Hoefler
It doesn't, simple as that. Just take a look at the source.
Vinzent.
H. Peter Anvin
Return-Path: <od...@wotan.crayne.org>
X-Comment-To: "Herman Dullink" <hdul...@hetnet.nl>
Disclaimer: Not speaking for Transmeta in any way, shape, or form.
Copyright: Copyright 2003 H. Peter Anvin - All Rights Reserved
X-MailScanner: Found to be clean
X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (score=-0.2,
required 5, NOSPAM_INC, REFERENCES, SIGNATURE_SHORT_DENSE,
SPAM_PHRASE_00_01)
X-UIDL: Wp;!!7+a"!Q~X"!kK<!!
Followup to: <bbse5d$lvf$2...@reader10.wxs.nl>
By author: "Herman Dullink" <hdul...@hetnet.nl>
In newsgroup: comp.lang.asm.x86
>
> > Hmm... well, in practice it doesn't make much difference, since HIMEM.SYS
> > can't assume the processor is not in Virtual 8086 mode.
> That's why it tests for it first. Using the SMSW instruction, it tests if
> the CPU is running in protected mode. If not, it'll use 'Real Big Mode' (aka Flat
> Real Mode, aka UnReal Mode)>
>
How daft -- of course, this is a Microsoft product, so it shouldn't be
a surprise they're being idiots. Since you can't allow interrupts
anyway, you might as well use plain old Protected Mode. All using
Unreal Mode does is gives you the extra performance penalty of an
extra mode transition on the way in and on the way out.
-hpa
--
<h...@transmeta.com> at work, <h...@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64
Vinzent Hoefler
>a surprise they're being idiots. Since you can't allow interrupts
>anyway,
Who says, you can't allow interrupts?
>you might as well use plain old Protected Mode.
You wouldn't stop interrupts for the whole time a 1 MiByte (or more)
memory-memory-copy needs, especially on an old 386, would you? I'd
consider this a bad idea.
> All using
>Unreal Mode does is gives you the extra performance penalty of an
>extra mode transition on the way in and on the way out.
Well, but even that switch is relatively fast[0]. :)
Plus you get the extra of the possibility to use interrupts without
needing to initialize a whole IDT.
Vinzent.
[0] IIRC, a plain 386/16 is capable of about 9000 transitions per
second.
Black Phantom
> > > Hmm... well, in practice it doesn't make much difference, since HIMEM.SYS
> > > can't assume the processor is not in Virtual 8086 mode.
> > That's why it tests for it first. Using the SMSW instruction, it tests if
> > the CPU is running in protected mode. If not, it'll use 'Real Big Mode' (aka Flat
> > Real Mode, aka UnReal Mode)>
> >
>
> How daft -- of course, this is a Microsoft product, so it shouldn't be
> a surprise they're being idiots.
I wish I understood this rant. And AFAIK, HIMEM.SYS never was a
separate product (?)
> Since you can't allow interrupts
> anyway, you might as well use plain old Protected Mode. All using
> Unreal Mode does is gives you the extra performance penalty of an
> extra mode transition on the way in and on the way out.
I don't think I understand this. It's all *not* using Unreal Mode
does is give you extra mode switching (RM to PM and PM to RM).
/BP
Herman Dullink
HIMEM.SYS
> > > can't assume the processor is not in Virtual 8086 mode.
> > That's why it tests for it first. Using the SMSW instruction, it tests
if
> > the CPU is running in protected mode. If not, it'll use 'Real Big Mode'
(aka Flat
> > Real Mode, aka UnReal Mode)>
> How daft -- of course, this is a Microsoft product, so it shouldn't be
> a surprise they're being idiots.
from Microsoft (of course :-).
> Since you can't allow interrupts anyway
Why not? You're still in Real Mode. The only difference between having set
the segment
limits to 4GB and not having set the limits is that you won't have an SOE
when an offset
is larger than 64KB - 1.
> All using Unreal Mode does is gives you the extra performance penalty of
an
> extra mode transition on the way in and on the way out.
No it doesn't, Flat Real/Real Big/UnReal mode is not actually a mode, it's
still Real
Mode, but with some segment descriptor cache registers set to a different
value than
default.
While these registers aren't reset to default, no extra penalty is given.
Flat Real (see
Flat.zip) and Real Big (Himem) have an SOE handler installed which set the
limits
(back) to 4GB in case something has reset them. In normal DOS operation,
such
a handler is triggered only once during the first >64KB addressing (by e.g.
SmartDrv :-).
H
Tom Thornhill
> > By "no one seems to use this technique", I meant "no one seems to
> > use a GP fault handler to fix up segment limits in unreal mode", rather
> > than no one uses Unreal Mode itself.
> OK, well Himem install a GP fault handler (temporary) :-)
If anyone is interested I found this code
http://www.himinbi.org/~will/fileformat.virtualave.net/programming/himem/xm386.asm
;*******************************************************************************
;
; Int13Handler
; Handler for int 13 during our rep moves
; If it is a real interrupt, jump to the old handler
; If it is a fault, set Real Big Mode and return