Explanations about unreal mode bug
Antoine Leca
bug report 2962575 in KVM about what is apparently the same issue.
http://sf.net/tracker/?func=detail&atid=893831&aid=2962575&group_id=180599
However I am failing to fully understand the technical matters involved;
I got the idea the problem is related to alignement matters (this is
pretty easy to grasp, I admit, just reading the patch); and I got the
impression it is caused by the virtual BIOS of bochs/qemu, but I do not
see the exact reason why 16-byte alignement is required, by which
software (any virtual BIOS? Only with KVM?) and when (Only on Intel-VT?)
Can someone shed a bit more light on this issue?
Antoine
Erik van der Kouwe
> I saw Erik's commit 6387 in the boot monitor loader, and I also found
> bug report 2962575 in KVM about what is apparently the same issue.http://sf.net/tracker/?func=detail&atid=893831&aid=2962575&group_id=1...
Correct.
> However I am failing to fully understand the technical matters involved;
> I got the idea the problem is related to alignement matters (this is
> pretty easy to grasp, I admit, just reading the patch); and I got the
> impression it is caused by the virtual BIOS of bochs/qemu, but I do not
> see the exact reason why 16-byte alignement is required, by which
> software (any virtual BIOS? Only with KVM?) and when (Only on Intel-VT?)
>
> Can someone shed a bit more light on this issue?
The patch is based on Avi Kivity's answer to my question about the
problem with MINIX on KVM 0.12.3 on a public mailing list:
http://www.mail-archive.com/k...@vger.kernel.org/msg30306.html
The explanation is fairly simple, although not immediately obvious
unless one knows what KVM can and cannot do.
The situation is as follows:
The boot monitor runs in real-address mode, but has to copy parts of
the boot image into high memory (>= 1 MB) which is not accessible from
that mode as only 20 bits are available. It calls the BIOS (int 0x15)
to perform the copy. This is done under the ext_copy label in boot/
boothead.s.
The BIOS switches to protected mode, loading a GDT which it receives
from the caller. Before returning to the caller, it copies data using
the segment descriptors in the GDT and switches back to real-address
mode. When doing switch, the cached segment selectors are preserved,
which allows one to use protected mode segments in real-address mode
(this is called unreal mode).
I knew these parts before, but this is where Avi's answer came in: KVM
on Intel does not yet support unreal mode and requires the cached
segment descriptors to be valid in real-address mode.
In real-address mode, segment descriptors are derived from segment
selectors as follows: base=selector*16 and limit=0xffff. Because MINIX
had unaligned segment bases in the GDT provided to the BIOS, it left
the cached segment descriptors invalid in real-address mode. This is
irrelevant if unreal mode is supported, but was a problem in KVM on
Intel.
The solution was to 16-byte align the pointers that end up in the GDT
used for the copy operation.
With kind regards,
Erik
Antoine Leca
> The patch is based on Avi Kivity's answer to my question about the
> problem with MINIX on KVM 0.12.3 on a public mailing list:
> http://www.mail-archive.com/k...@vger.kernel.org/msg30306.html
>
>
> The boot monitor runs in real-address mode, but has to copy parts of
> the boot image into high memory (>= 1 MB) which is not accessible from
> that mode as only 20 bits are available. It calls the BIOS (int 0x15)
> to perform the copy. This is done under the ext_copy label in boot/
> boothead.s.
Okay. It is my understanding this is where Minix' involvement stops.
The careful reader will note a similar call is in boot/doshead.s.
> The BIOS switches to protected mode, loading a GDT which it receives
> from the caller. Before returning to the caller, it copies data using
> the segment descriptors in the GDT and switches back to real-address
> mode.
This is the description of BIOS service 15/87, which have to be
implemented (using whatever solution it pleases) by the BIOS.
> When doing switch, the cached segment selectors are preserved,
> which allows one to use protected mode segments in real-address mode
> (this is called unreal mode).
Now this is a by-product of the implementation inside the BIOS.
In fact, even if the BIOS enters unreal mode (or the similar big real,
more useful with segmentation-less architectures), before turning back
to the client it (should) reset things to normal real mode, and service
15/87 is not an usual way to enter unreal mode (for example, this effect
is not even mentionned in Ralf Brown's list).
As a result (and also and foremost because of 80286 compatibility),
instead of directly using unreal or big real mode if possible (as done
eg. in himem.sys), Minix monitor goes to the great pain to going back to
square #1, and since blocks are at most 64 KB in size and several
iterations are needed, on the next block Minix sets up the (very
similar) GDT then does another call to the same BIOS service 15/87.
> I knew these parts before, but this is where Avi's answer came in: KVM
> on Intel does not yet support unreal mode and requires the cached
> segment descriptors to be valid in real-address mode.
I do not know which virtual BIOS is using KVM, but I notice while
reading http://bochs.sourceforge.net/cgi-bin/lxr/source/bios/rombios.c:
[ Slightly edited to fit the width of my post. AL. ]
3555 case 0x87:
3556 #if BX_CPU < 3
3557 # error "Int15 function 87h not supported on < 80386"
3558 #endif
3559 // +++ should probably have descriptor checks
3560 // +++ should have exception handlers
...
3640 mov eax, cr0
3641 or al, #0x01
3642 mov cr0, eax
3643 ;; far jump to flush CPU queue after transition to prot. mode
3644 JMP_AP(0x0020, protected_mode)
3645
3646 protected_mode:
3647 ;; GDT points to valid descriptor table, now load SS, DS, ES
3648 mov ax, #0x28 ;; 101 000 = 5th desc.in table, TI=GDT,RPL=00
3649 mov ss, ax
3650 mov ax, #0x10 ;; 010 000 = 2nd desc.in table, TI=GDT,RPL=00
3651 mov ds, ax
3652 mov ax, #0x18 ;; 011 000 = 3rd desc.in table, TI=GDT,RPL=00
3653 mov es, ax
3654 xor si, si
3655 xor di, di
3656 cld
3657 rep
3658 movsw ;; move CX words from DS:SI to ES:DI
3659
3660 ;; make sure DS and ES limits are 64KB
3661 mov ax, #0x28
3662 mov ds, ax
3663 mov es, ax
3664
3665 ;; reset PG bit in CR0 ???
3666 mov eax, cr0
3667 and al, #0xFE
3668 mov cr0, eax
I should be loosing something here... There is no unreal mode at any
moment, is it?
[ ... some web browsing occuring meanwhile ... Later: ]
Okay, now I got another picture. 8-|
Until recently, KVM (and qemu) used Bochs BIOS, showed above; but they
switched recently to SeaBIOS... where the applicable code is in
src/system.c, and looks like (now this is AT&T assembly):
83 static void
84 handle_1587(struct bregs *regs)
85 {
86 // +++ should probably have descriptor checks
87 // +++ should have exception handlers
....
127 // Enable protected mode
128 " movl %%cr0, %%eax\n"
129 " orl $" __stringify(CR0_PE) ", %%eax\n"
130 " movl %%eax, %%cr0\n"
131
132 // far jump to flush CPU queue after transition to prot. mode
133 " ljmpw $(4<<3), $1f\n"
134
135 // GDT points to valid descriptor table, now load DS, ES
136 "1:movw $(2<<3), %%ax\n"
// 2nd descriptor in table, TI=GDT, RPL=00
137 " movw %%ax, %%ds\n"
138 " movw $(3<<3), %%ax\n"
// 3rd descriptor in table, TI=GDT, RPL=00
139 " movw %%ax, %%es\n"
140
141 // move CX words from DS:SI to ES:DI
142 " xorw %%si, %%si\n"
143 " xorw %%di, %%di\n"
144 " rep movsw\n"
145
146 // Disable protected mode
147 " movl %%cr0, %%eax\n"
148 " andl $~" __stringify(CR0_PE) ", %%eax\n"
149 " movl %%eax, %%cr0\n"
Note that while the basic scheme is the same, the "cleaning up" of lines
3660-3663 "make sure DS and ES limits are 64KB" is not present.
> In real-address mode, segment descriptors are derived from segment
> selectors as follows: base=selector*16 and limit=0xffff. Because MINIX
> had unaligned segment bases in the GDT provided to the BIOS, it left
> the cached segment descriptors invalid in real-address mode.
Okay, this explains the bug you saw. But can we say the bug is specific
to KVM/SeaBIOS, in its current state? I mean, when/if KVM's BIOS is
modified, for example to clean up the big segments before switching
back, then the workaround would not be necessary, would it?
I am not saying the workaround should be reverted, of course it should
not (since it makes the monitor more stable at any rate); I am just
trying to emphatize what is the real problem which drives the added code
(moreover, since the current trend is aparently to follow the BSD
tradition and have very few comments remaining in the code.)
On the other hand I will try to draw attention on this on part of KVM
developpers...
Antoine
Erik van der Kouwe
> Note that while the basic scheme is the same, the "cleaning up" of lines
> 3660-3663 "make sure DS and ES limits are 64KB" is not present.
Good find! That explains why it was broken only recently. IMHO this is a
bad thing about the new BIOS then, it is more proper to clean up after
it is done. Nevertheless, it would still have worked if KVM had
supported unreal mode.
>> In real-address mode, segment descriptors are derived from segment
>> selectors as follows: base=selector*16 and limit=0xffff. Because MINIX
>> had unaligned segment bases in the GDT provided to the BIOS, it left
>> the cached segment descriptors invalid in real-address mode.
>
> Okay, this explains the bug you saw. But can we say the bug is specific
> to KVM/SeaBIOS, in its current state? I mean, when/if KVM's BIOS is
> modified, for example to clean up the big segments before switching
> back, then the workaround would not be necessary, would it?
Correct, I think this is how it was in KVM 0.11.0.
> On the other hand I will try to draw attention on this on part of KVM
> developpers...
Thanks.
With kind regards,
Erik