16-bit protected mode?

[James Harris]

Have any of you already worked out the following? Or do you want a way to run 16-bit code under a 64-bit OS?

I heard long ago that AMD did not support use of 16-bit code when in Long mode but I see that Long mode does in fact support some form of 16-bit code. Is that protected mode where the code and data segments have default address size and operand size set to 16 bits?

Is 16-bit Protected mode the right term?

Whatever it's called it seems an unusual mode to run in.

Would it be feasible to run old 16-bit apps in this mode? If not, what would cause them to fail?

Could the BIOS be run in that mode, perhaps with the OS emulating some operations?

I see that V8086 mode and Real mode won't work under Long mode but wonder how much can be done in the above 16-bit mode. Is it useful or not?

James

[Alexei A. Frounze]

On Monday, July 8, 2013 3:30:20 AM UTC+4, James Harris wrote:
> Have any of you already worked out the following? Or do you want a way to run 16-bit code under a 64-bit OS?
>
>
>
> I heard long ago that AMD did not support use of 16-bit code when in Long mode but I see that Long mode does in fact support some form of 16-bit code. Is that protected mode where the code and data segments have default address size and operand size set to 16 bits?
>

Protected. Not real or v86.

> Is 16-bit Protected mode the right term?

Yep.

> Whatever it's called it seems an unusual mode to run in.

What's unusual about it? If you want to support 16-bit segmented apps from Windows 3.xx/9x or OS/2 (I hear the latter also used to use segments), then you have the right CPU mode for that.

> Would it be feasible to run old 16-bit apps in this mode? If not, what would cause them to fail?

Depends on the apps. Bugs?

> Could the BIOS be run in that mode, perhaps with the OS emulating some operations?
>

It would be more like the OS emulating pretty much everything. Most of the BIOS code runs in the real address mode or the virtual 8086 mode.

> I see that V8086 mode and Real mode won't work under Long mode but wonder how much can be done in the above 16-bit mode. Is it useful or not?
>

I'd say it's of very little practical use these days. When you start parsing code and decoding instructions (in order to workaround GDT/LDT-based address translation absent in real/v86 mode), you've already got some 30%-50% of an emulator done.

AFAIK, Windows uses an emulator to run BIOS video code (int 10h) in 64-bit mode when the proper video driver isn't loaded yet.

Alex

[James Harris]

On Monday, July 8, 2013 8:32:59 AM UTC+1, Alexei A. Frounze wrote:
> On Monday, July 8, 2013 3:30:20 AM UTC+4, James Harris wrote:
>
> > Have any of you already worked out the following? Or do you want a way to run 16-bit code under a 64-bit OS?
>
> > I heard long ago that AMD did not support use of 16-bit code when in Long mode but I see that Long mode does in fact support some form of 16-bit code. Is that protected mode where the code and data segments have default address size and operand size set to 16 bits?
>
> Protected. Not real or v86.
>
> > Is 16-bit Protected mode the right term?
>
> Yep.
>
> > Whatever it's called it seems an unusual mode to run in.
>
> What's unusual about it? If you want to support 16-bit segmented apps from Windows 3.xx/9x or OS/2 (I hear the latter also used to use segments), then you have the right CPU mode for that.

Ah ... so that's where it was used. I don't think I ever wrote a 16-bit Windows app but now I can see what the mode was used for.

> > Would it be feasible to run old 16-bit apps in this mode? If not, what would cause them to fail?
>
> Depends on the apps. Bugs?
>
> > Could the BIOS be run in that mode, perhaps with the OS emulating some operations?
>
> It would be more like the OS emulating pretty much everything. Most of the BIOS code runs in the real address mode or the virtual 8086 mode.
>
> > I see that V8086 mode and Real mode won't work under Long mode but wonder how much can be done in the above 16-bit mode. Is it useful or not?
>
> I'd say it's of very little practical use these days. When you start parsing code and decoding instructions (in order to workaround GDT/LDT-based address translation absent in real/v86 mode), you've already got some 30%-50% of an emulator done.

Understood. An app could set anything it wanted in a selector and expect it to point to a specific 16-byte block of memory. The pmode descriptors would not match.

I was thinking more of letting the CPU execute the instructions - rather than parsing them - but that would work only if the CPU could get control back when needed such as when the 16-bit code loaded a segment selector.

That might be possible if the segment descriptors were initially marked as either privileged or not present. The OS would get control when one was loaded. That should work in most cases. It would be awkward if the 16-bit app wanted to use a segment selector that was designated for the OS but that too could probably be worked around.

Is segmentation the only impediment to running the BIOS in 16-bit pmode? I guess not as its IDT handling would also need to be accommodated. That could perhaps be done by page protections.

There would also be the need to simulate device access but that would be true whatever means of pmode-handling of the BIOS was used.

> AFAIK, Windows uses an emulator to run BIOS video code (int 10h) in 64-bit mode when the proper video driver isn't loaded yet.

That is similar to what I was thinking to do, i.e. run the video BIOS in an emulator. Rather than go to the bother of writing a V8086 monitor which would work in protected mode but not in long mode an emulator would work in both. It would not run as quickly but it would be portable. And for what I would want it to do speed would not be an issue.

The 16-bit pmode option is intriguing though as it would do a lot of the work automatically - as long as the OS gets control at the right times.

James

[Rod Pemberton]

"James Harris" <james.h...@gmail.com> wrote in message
news:82f23bda-1037-46ef...@googlegroups.com...

> Have any of you already worked out the following?

Not I.

> Or do you want a way to run 16-bit code under a 64-bit OS?

My long, long term goal, after I get back around to working on my
OS again, which could be never, was to find a way to code my code
once and have it work for 16-bit, 32-bit, and 64-bit code. I had
two ideas on this. The first was an interpreter which implements a
consistent data size, likely 32-bits, for all modes. A C compiler
that supported all three modes would be used to compile the
interpreter for each mode. The second was multi-mode assembly. A
while back, I found a limited way to implement 16-bit and 32-bit
code that executes equivalently. That's dual mode 16-bit and
32-bit without any operand and address size overrides. I was
hoping to find a way to do 16-bit and 32-bit and 64-bit. But, I'm
not sure that that is possible because of the REX prefix... You'd
have to find a way to nullify the effects of it's operation in
16-bit and 32-bit code.

> I heard long ago that AMD did not support use of 16-bit code when
> in Long mode but I see that Long mode does in fact support some
> form of 16-bit code.

I all I know about that at this point in time is what Sandpile.org
has:
http://www.sandpile.org/x86/mode.htm

They call it CM16. You can see CS.ar.L is 0 for CM16 and CM32
instead of 1 for PM64. You should be able jump to segment using a
new CS segment selector without CS.ar.L being set. CS.ar.D will
need to be set correctly for CM16 or CM32. This should be nearly
identical to jumping to a 16-bit code segment from a 32-bit code
segment.

> Is that protected mode where the code and data segments have
> default address size and operand size set to 16 bits?

You're asking if CM16 is like PM16 or VM16 or RM16? Yeah, I have
no idea. Sorry.

Although, the link above would indicate it's PM since it has CR0.PE
set. CM16 appears to have the same settings for PM16, with CS.ar.L
cleared (not long mode...), RFLAGS.VM ignored (so not VM16...), and
PAE is enabled.

> Is 16-bit Protected mode the right term?

From the link, it appears so.

> Whatever it's called it seems an unusual mode to run in.

... not long mode, but with PAE ...

> Would it be feasible to run old 16-bit apps in this mode?

Which mode? As Alexei pointed out there are RM16 (or VM16) and
PM16 16-bit apps. I'm assuming you meant RM16. If it's a PM,
which the chart indicates, then most 16-bit code probably wouldn't.

> If not, what would cause them to fail?

Probably, the same things as executing RM16 code in PM16 ...
whatever those are. At least, that's what I'm assuming. Addressing
is different. GDT is different. IDT is different. What else?
Otherwise, much of the 16-bit code is the same between modes. The
issue is whether PM16 could be setup to work execute RM16 code.
I'd suspect that most code could be, if coded correctly or
"cleanly" to work for both. You'd need code where the code didn't
manipulate the segment registers or do math to calculate addresses.
I.e., if you set a selector to a segment value, it'd be wrong. You
could setup an IDT that worked the same as the IVT. You'd need
appropriate selectors in the GDT for each segment used in the IVT.
The question is whether you have enough information to setup a GDT
without knowing what segments are use in RM code. With something
like a BIOS or option ROM (video, eth), you would know the segments
used in advance. With an application, you might not. Of course,
this is all speculation based on what I've seen of the other modes,
since I've not use PM16, nor CM16/32, nor PM64.

> Could the BIOS be run in that mode, perhaps with the OS emulating
> some operations?

I'm sure there are no BIOSes created prior to that mode which have
actually been tested in that mode. Whether this works or not
depends on how compatible CM16 is with PM16 and/or RM16 (or VM16).
AIUI, the BIOS calls are supposed to work for both RM16 and VM16,
but I've never seen anything which confirm this. Do 16-bit BIOS
calls work in PM16 too? I suspect they could if coded correctly.
PM16 has been around as long as VM16 and almost as long as RM16 ...
I.e., does 16-bit BIOS code mess with addressing or is it "clean"?

> I see that V8086 mode and Real mode won't work under Long mode
> but wonder how much can be done in the above 16-bit mode. Is it
> useful or not?

No idea... If BIOS code can be cleanly executed in PM16 when a PM
environment is correctly setup, then, yes, definately, it's useful.
But, I have no idea if that's possible or not.


Rod Pemberton

[James Harris]

On Monday, July 8, 2013 12:43:04 PM UTC+1, Rod Pemberton wrote:
> "James Harris" <james.h...@gmail.com> wrote in message
> news:82f23bda-1037-46ef...@googlegroups.com...
>
> > Have any of you already worked out the following?
>
> Not I.
>
> > Or do you want a way to run 16-bit code under a 64-bit OS?
>
> My long, long term goal, after I get back around to working on my
> OS again, which could be never, was to find a way to code my code
> once and have it work for 16-bit, 32-bit, and 64-bit code.

That's what I've been looking at. I figured that if I was going to make the move away from writing in assembly to writing in C then I would have the problem of having to start dealing with the vagaries of different compilers ... but to offset that there was a way to get a big gain. The one really big advantage of making the change was to be able to write OS code that works in any bit width.

In other words, I was thinking that it should be possible to write some of the OS code once and compile it in to three different object images - i.e. create large chunks of a 16-bit OS, a 32-bit OS and a 64-bit OS all from the same source. And the resulting code should be essentially the same OS but running in the different modes.

Some of the source code would need to be different but much of it could be the same.

For example, it should be possible to have a single set of source code for managing many of the OS's internal data structures and have that compiled to 16-, 32- and 64-bit object code. Also, device management, while it would work with the specific port widths needed - e.g. 8-bit ports - could pass values around as integers of a width suitable for the bit-width the OS was running ... so uint8_t would be used to access the ports and unsigned ints would be passed between modules.

It would not be possible to make all the HLL code bit-width agnostic. A number of data structures are very different for different bit widths. The PM IDT is very different from the RM IDT, for example. So there would need to be some bit-width-specific source code. But, hopefully, much of it could be written once and compiled to each mode.

There are some particular nasties, especially with RM. RM memory management would be very odd, given that two pointer types are possible and that one of them is wider than the natural integer. In RM, also, there is no security and no virtual addressing.

Nevertheless it seems to be a good way to go. If it can be made to work there are two other specific advantages:

1. Building the OS this way has to make the OS more modular and thus easier to maintain.

2. The OS should be able to run in real mode and thus have BIOS support. That could be a big help to get the ball rolling.

> I had
> two ideas on this. The first was an interpreter which implements a
> consistent data size, likely 32-bits, for all modes. A C compiler
> that supported all three modes would be used to compile the
> interpreter for each mode. The second was multi-mode assembly. A
> while back, I found a limited way to implement 16-bit and 32-bit
> code that executes equivalently. That's dual mode 16-bit and
> 32-bit without any operand and address size overrides. I was
> hoping to find a way to do 16-bit and 32-bit and 64-bit. But, I'm
> not sure that that is possible because of the REX prefix... You'd
> have to find a way to nullify the effects of it's operation in
> 16-bit and 32-bit code.

YMMV but I'm not sure that either of those is a good idea. The interpreter would be slow and might need to be pretty big to cope with all the instructions it might have to interpret. And requiring code to compile to some binary that can execute in either RM or PM could result it too many wrangles with the compilers.

Wouldn't it be simpler to use a 16-bit compiler to build 16-bit object files and a 32-bit compiler to build 32-bit object files? Both could be built from the same HLL source. (They would need to be linked with other object files which were mode specific in order to generate the complete OS.)

Speaking of compilers I have been using bcc (Bruce's C Compiler, not Borlands) to generate 16-bit code and gcc to generate 32-bit code. Would you recommend I change to another compiler for 16-bit code?

I haven't done much yet but have run code samples through both compilers to ensure it compiles portably.

James

[Alexei A. Frounze]

On Monday, July 8, 2013 2:28:12 PM UTC+4, James Harris wrote:
> On Monday, July 8, 2013 8:32:59 AM UTC+1, Alexei A. Frounze wrote:
>
> > On Monday, July 8, 2013 3:30:20 AM UTC+4, James Harris wrote:
>
> >
>
> > > Have any of you already worked out the following? Or do you want a way to run 16-bit code under a 64-bit OS?
>
> >
>
> > > I heard long ago that AMD did not support use of 16-bit code when in Long mode but I see that Long mode does in fact support some form of 16-bit code. Is that protected mode where the code and data segments have default address size and operand size set to 16 bits?
>
> >
>
> > Protected. Not real or v86.
>
> >
>
> > > Is 16-bit Protected mode the right term?
>
> >
>
> > Yep.
>
> >
>
> > > Whatever it's called it seems an unusual mode to run in.
>
> >
>
> > What's unusual about it? If you want to support 16-bit segmented apps from Windows 3.xx/9x or OS/2 (I hear the latter also used to use segments), then you have the right CPU mode for that.
>
>
>
> Ah ... so that's where it was used. I don't think I ever wrote a 16-bit Windows app but now I can see what the mode was used for.
>
>
>
> > > Would it be feasible to run old 16-bit apps in this mode? If not, what would cause them to fail?
>
> >
>
> > Depends on the apps. Bugs?
>
> >
>
> > > Could the BIOS be run in that mode, perhaps with the OS emulating some operations?
>
> >
>
> > It would be more like the OS emulating pretty much everything. Most of the BIOS code runs in the real address mode or the virtual 8086 mode.
>
> >
>
> > > I see that V8086 mode and Real mode won't work under Long mode but wonder how much can be done in the above 16-bit mode. Is it useful or not?
>
> >
>
> > I'd say it's of very little practical use these days. When you start parsing code and decoding instructions (in order to workaround GDT/LDT-based address translation absent in real/v86 mode), you've already got some 30%-50% of an emulator done.
>
>
>
> Understood. An app could set anything it wanted in a selector and expect it to point to a specific 16-byte block of memory. The pmode descriptors would not match.
>

Exactly.

> I was thinking more of letting the CPU execute the instructions - rather than parsing them - but that would work only if the CPU could get control back when needed such as when the 16-bit code loaded a segment selector.
>

But you first need to differentiate between what you can and can't execute as-is. So, you need to parse to make the decision.

> That might be possible if the segment descriptors were initially marked as either privileged or not present. The OS would get control when one was loaded. That should work in most cases. It would be awkward if the 16-bit app wanted to use a segment selector that was designated for the OS but that too could probably be worked around.
>

So, even if you get a #GP (or #SS, #NP, etc) to handle, what's next? You still need to figure out what needs to be done and that depends on the instruction that's caused the exception, which means you need to parse/decode it.

> Is segmentation the only impediment to running the BIOS in 16-bit pmode? I guess not as its IDT handling would also need to be accommodated. That could perhaps be done by page protections.
>

Right, interrupts and exceptions are an issue as well, but they aren't a big problem as the former are unlikely to occur in BIOS code and the latter can be emulated.

> There would also be the need to simulate device access but that would be true whatever means of pmode-handling of the BIOS was used.
>
>
>
> > AFAIK, Windows uses an emulator to run BIOS video code (int 10h) in 64-bit mode when the proper video driver isn't loaded yet.
>
>
>
> That is similar to what I was thinking to do, i.e. run the video BIOS in an emulator. Rather than go to the bother of writing a V8086 monitor which would work in protected mode but not in long mode an emulator would work in both. It would not run as quickly but it would be portable. And for what I would want it to do speed would not be an issue.
>
>
>
> The 16-bit pmode option is intriguing though as it would do a lot of the work automatically - as long as the OS gets control at the right times.
>

As long as, yes.

Another odd option would be to enable virtualization and let a VM do all BIOS work in the real address mode. For that you need a CPU that supports this mode in VMs directly (early intel CPUs didn't).

Alex

[wolfgang kern]

James Harris asked:

|Have any of you already worked out the following? Or do you want a way to
|run 16-bit code under a 64-bit OS?

My 64-bit OS variant is still under development...
But when I look at VESA-extensions (beside the opportunity to call
16-bit BIOS/VBIOS functions that will work on PM16 similar to RM16)
there would this easy switch from LONG to RM16 become quite handy.

|I heard long ago that AMD did not support use of 16-bit code when in Long
|mode but I see that Long mode does in fact support some form of 16-bit
code. |Is that protected mode where the code and data segments have default
address |size and operand size set to 16 bits?

|Is 16-bit Protected mode the right term?

yes, even PM16 is different to VM86 and BIG-Real/Unreal-modes.

|Whatever it's called it seems an unusual mode to run in.

The BIOS itself uses it on power-up, and I (my OS) have to use it whenever
a transition from PM32 to true RM is requested (just one far jump anyway).

|Would it be feasible to run old 16-bit apps in this mode?

Sure, as long all expected API/SYS-calls were supported then...

|If not, what would cause them to fail?

Bugs and wrong assumtions ? :)

|Could the BIOS be run in that mode, perhaps with the OS emulating some
|operations?

Some/many BIOS-functions would work both, RM and PM16 as long they wont
alter any segments by itself (see INT0x10 or better its INT0x6D alias).

|I see that V8086 mode and Real mode won't work under Long mode but wonder
|how much can be done in the above 16-bit mode. Is it useful or not?

Methink it's not for nothing that AMD supports PM16 code in LONG-mode
while I'm not happy that all PM32-support is gone then.
I also would appreciate if I could disable paging in LONG-mode.

__
wolfgang

[Rod Pemberton]

"James Harris" <james.h...@gmail.com> wrote in message

news:911de716-4485-493c...@googlegroups.com...

On Monday, July 8, 2013 12:43:04 PM UTC+1, Rod Pemberton wrote:
> "James Harris" <james.h...@gmail.com> wrote in message
> news:82f23bda-1037-46ef...@googlegroups.com...
>

> > [SNIP]

>
> It would not be possible to make all the HLL code bit-width
> agnostic. A number of data structures are very different for
> different bit widths. The PM IDT is very different from the RM
> IDT, for example. So there would need to be some
> bit-width-specific source code. But, hopefully, much of it could
> be written once and compiled to each mode.

RM has an IVT. IDTs are for PM. But, yes, a RM IVT and a 32-bit
IDT, have differently sized data. Does a 16-bit PM IDT have the
same _size_ data as a RM IVT, not _type_? I'm not sure. I'd guess
that they do. So, maybe, mode-specific code, not bit-width
agnostic, would be more appropriate? I.e., RM is 16-bits and one
of the PMs is 16-bits.

> There are some particular nasties, especially with RM. RM memory
> management would be very odd, given that two pointer types are
> possible and that one of them is wider than the natural integer.

I don't see how that's any different from PM. RM has a segment and
offset. PM has a base address and offset. Segmentation exists for
both. It's just a question of whether the offset is large enough
to address all of memory. That's true for 32-bit PM memory upto
4GB. It's not true for RM. It's probably not true for 16-bit PM
either. I'm not up on 64-bit, but I seem to recall that the offset
is limited to a smaller range than the maximum hardware memory for
it also.

> In RM, also, there is no security and no virtual addressing.

Yeah, I'm not using security so far... Paging and VMM wasn't used
for many of the older processors and computers I learned from.
AIR, the 68000 series had just introduced a paging coprocessor for
the 68020 when I stopped learning about processors, until I
restarted a few years ago. Yes, paging is an area of the Intel and
AMD manuals I've been avoiding. I had to learn some virtual
addressing for x86.

> Nevertheless it seems to be a good way to go. If it can be made
> to work there are two other specific advantages:
>
> 1. Building the OS this way has to make the OS more modular and
> thus easier to maintain.
>
> 2. The OS should be able to run in real mode and thus have BIOS
> support. That could be a big help to get the ball rolling.

...

> > I had two ideas on this. The first was an interpreter which
> > implements a consistent data size, likely 32-bits, for all
> > modes. A C compiler that supported all three modes would
> > be used to compile the interpreter for each mode. The second
> > was multi-mode assembly. A while back, I found a limited way
> > to implement 16-bit and 32-bit code that executes equivalently.
> > That's dual mode 16-bit and 32-bit without any operand and
> > address size overrides. I was hoping to find a way to do
> > 16-bit and 32-bit and 64-bit. But, I'm not sure that that is
> > possible because of the REX prefix... You'd have to find a
> > way to nullify the effects of it's operation in 16-bit and
> > 32-bit code.
>
> YMMV but I'm not sure that either of those is a good idea. The
> interpreter would be slow

True.

> and might need to be pretty big to cope with all the instructions
> it might have to interpret.

Well, I wasn't thinking an x86 instruction set interpreter for that
reply, although I had considered it previously. That would be
complicated. I was thinking more of byte code in my reply. Of
course, that wouldn't be powerful enough, or correct, to execute
BIOS in 64-bit mode... The big issue is do you need BIOS support
in the OS or not? I.e., can BIOS usage be limited to the MBR/VBR
and bootloader code? I think it can. I.e., 16-bit code can be
used until transferring execution to the OS.

> And requiring code to compile to some binary that can execute in
> either RM or PM could result it too many wrangles with the
> compilers.

That's true. Assembly would be the choice for this. A good
assembler with a powerful pre-processor, like NASM, might help too.

> Wouldn't it be simpler to use a 16-bit compiler to build 16-bit
> object files and a 32-bit compiler to build 32-bit object files?

What exactly would be simpler? Two compilers instead of one?
Or, building 16-bit object files and 32-bit object files
separately?

Two compilers is definately more work than just one. It's easier
to build for just one mode. But, if you need to support three
modes, you need three binaries... That's why I was talking about
an interpreter. I can restrict the custom instruction size to
something easy to support across multiple modes, say 8-bits or
16-bits.

> Both could be built from the same HLL source.

That's great, where possible. It's never 100%.

> (They would need to be linked with other object files which
> were mode specific in order to generate the complete OS.)

Is this binary mixed-mode code? Both 16-bit and 32-bit in one
binary? Yeah, I just haven't considered that. I'd prefer entirely
16-bit, or entirely 32-bit, etc.

> Speaking of compilers I have been using bcc (Bruce's C Compiler,
> not Borlands) to generate 16-bit code and gcc to generate 32-bit
> code. Would you recommend I change to another compiler for 16-bit
> code?

I don't know.

Can the 16-bit code be converted to NASM? I.e., move as much C
code into the 32-bit code as possible, and implement the remainder
in assembly.

Are you using GCC on Linux or via DJGPP for DOS? DJGPP did
release a minimal 16-bit backend. I've not used it.
http://delorie.com/djgpp/16bit/

Also, GAS and GCC inlined assembly in C, supports some 16-bit
directives:

.data16
.addr16
.code16
.code16gcc

You can also use Intel syntax, instead of AT&T syntax:

.intel_syntax

I can produce 16-bit exe's for a number of my DOS applications
without many changes with OpenWatcom. However, certain pointers
and addressing related code requires C #if-def sections to adjust
for each compiler. There is nothing you can do about that except
do what the compiler wants done.

My OS is 32-bit. It has a small amount of 16-bit code in NASM
assembly which is separate, standalone from the C code. I was
using two DOS C compilers: DJGPP (GCC) which supports 32-bit DPMI
and OpenWatcom which supports 32-bit DPMI and 16-bit. When I need
assembly combined with the C code, I use inline assembly (GAS,
WASM) in C, instead of linking in NASM object files. I was
planning on dropping OW support going forward. It does produce
faster code, and catch a few extra coding errors. But, supporting
two compilers, when DJGPP seems to be my preferred compiler, is
probably not worth it...


Rod Pemberton

[Rod Pemberton]

"wolfgang kern" <now...@never.at> wrote in message
news:krhhuu$kb2$1...@newsreader2.utanet.at...
> James Harris asked:

>

> |Is 16-bit Protected mode the right term?
>
> yes, even PM16 is different to VM86 and BIG-Real/Unreal-modes.
>
> |Whatever it's called it seems an unusual mode to run in.
>
> The BIOS itself uses it on power-up, and I (my OS) have to use it
> whenever a transition from PM32 to true RM is requested (just
> one far jump anyway).
>

Wolfgang,

I might be mistaken, but I think you're talking about the old PM16
while James is talking about the new CM16... (?)

Well, PM16 and CM16 are the names Sandpile.org has assigned to
these modes... CM16 is the "new" 16-bit long mode (EFER.LMA=1).
Go here:
http://www.sandpile.org/x86/mode.htm

Or, are you saying that new BIOSes use CM16? If so, what for?
Couldn't they just use PM16?


Rod Pemberton

[s_dub...@yahoo.com]

On Sunday, July 7, 2013 6:30:20 PM UTC-5, James Harris wrote:
> Have any of you already worked out the following? Or do you want a way to run 16-bit code under a 64-bit OS?
>
>
>
> I heard long ago that AMD did not support use of 16-bit code when in Long mode but I see that Long mode does in fact support some form of 16-bit code. Is that protected mode where the code and data segments have default address size and operand size set to 16 bits?
>
>
>
> Is 16-bit Protected mode the right term?
>
>
>
> Whatever it's called it seems an unusual mode to run in.
>
>
>
> Would it be feasible to run old 16-bit apps in this mode? If not, what would cause them to fail?
>
>

I'm curious to know. I don't have a 64bit machine available to test on. I have tried 16bit PM on a 32bit machine to explore some of 'what would cause them to fail'. You can switch in and out of PM without the need for setting CS to a selector of a descriptor, this is counter to what the manuals say: (switch to a selector using jmp far selector:offset at the earliest opportunity). You can co-mingle RM and 16bit PM code and data easy enough. I alias RM data with a 16bit PM data selector in FS, and use FS segment over-ride in the PM code to access the same data offsets. In the PM code governed by the RM CS the data references require a PM selector (as I say I used FS:same_offset for that), but this extends to the stack as well, for 'calls', requiring a selector for SS, so a stack switch is what I did for that to allow 'calls' to happen despite the fact that CS is not changed. I wanted to use a small PM stack for that. Trying to do that reveals an interesting 'gotcha': a fault occurs if the current SP value is outside the limit of the descriptor when its selector is to be loaded into SS, even tho the programmer is doing FS: LSS SP, [Location].

There is also a minimum PM stack size for a small stack. (I wish I had my notes and example code, but I lost them in a hard drive crash).

Considering the above, 16bit RM <-> 16bit PM, isn't quite what you are aiming for with Long Mode, also note that Long Mode drops support for some instructions; gone are the BCD ones. I use 'aam 16' often, and that is gone.

Compatibility Mode isn't compatible with the cpu task-switching mechanisms of old.

It sounds like a virtual machine or emulator/monitor is the only avenue, as others have suggested.

>
> Could the BIOS be run in that mode, perhaps with the OS emulating some operations?
>
>

In the scheme above, I drop out of PM to call a BIOS function, even in repeating looping code where the heart of the loop is in PM. That is a mode switch to call the BIOS, not what you are after it seems.

Steve

[James Harris]

On Tuesday, July 9, 2013 10:11:08 PM UTC+1, Rod Pemberton wrote:
> "James Harris" <james.h...@gmail.com> wrote in message
> news:911de716-4485-493c...@googlegroups.com...
> On Monday, July 8, 2013 12:43:04 PM UTC+1, Rod Pemberton wrote:
> > "James Harris" <james.h...@gmail.com> wrote in message
> > news:82f23bda-1037-46ef...@googlegroups.com...
> >
>
> > > [SNIP]
> >
> > It would not be possible to make all the HLL code bit-width
> > agnostic. A number of data structures are very different for
> > different bit widths. The PM IDT is very different from the RM
> > IDT, for example. So there would need to be some
> > bit-width-specific source code. But, hopefully, much of it could
> > be written once and compiled to each mode.
>
> RM has an IVT. IDTs are for PM. But, yes, a RM IVT and a 32-bit
> IDT, have differently sized data. Does a 16-bit PM IDT have the
> same _size_ data as a RM IVT, not _type_? I'm not sure. I'd guess
> that they do. So, maybe, mode-specific code, not bit-width
> agnostic, would be more appropriate? I.e., RM is 16-bits and one
> of the PMs is 16-bits.

Yes, IVT is a better acronym. The key thing is the formats of IVT and 32-bit IDT are completely different and so would need some different source code to manage them.

To try to illustrate what I had in mind the following could be the code for reading the CMOS/RTC chip.

int cmos_read(int addr)
{
if (addr > ADDR_MAX)
return -1;
wait_if_needed(addr);
addr_write(addr);
return port_8_in(DATA_PORT);
}

It's untested but the idea is that the same source code should compile to three different outputs: 16-bit, 32-bit and 64-bit object code and work in the same way on each of them. The int sizes would differ but not what the code does.

> > There are some particular nasties, especially with RM. RM memory
> > management would be very odd, given that two pointer types are
> > possible and that one of them is wider than the natural integer.
>
> I don't see how that's any different from PM. RM has a segment and
> offset. PM has a base address and offset. Segmentation exists for
> both. It's just a question of whether the offset is large enough
> to address all of memory. That's true for 32-bit PM memory upto
> 4GB. It's not true for RM. It's probably not true for 16-bit PM
> either. I'm not up on 64-bit, but I seem to recall that the offset
> is limited to a smaller range than the maximum hardware memory for
> it also.

The thing is that even with segments and whether paging is turned on or not, on x86-32 all the PM addresses that programs use are ultimately 32-bits wide. RM addresses are 16-bit or 20-bit.

...

> > > I had two ideas on this. The first was an interpreter which
> > > implements a consistent data size, likely 32-bits, for all
> > > modes. A C compiler that supported all three modes would
> > > be used to compile the interpreter for each mode. The second
> > > was multi-mode assembly. A while back, I found a limited way
> > > to implement 16-bit and 32-bit code that executes equivalently.
> > > That's dual mode 16-bit and 32-bit without any operand and
> > > address size overrides. I was hoping to find a way to do
> > > 16-bit and 32-bit and 64-bit. But, I'm not sure that that is
> > > possible because of the REX prefix... You'd have to find a
> > > way to nullify the effects of it's operation in 16-bit and
> > > 32-bit code.
> >
> > YMMV but I'm not sure that either of those is a good idea. The
> > interpreter would be slow
>
> True.
>
> > and might need to be pretty big to cope with all the instructions
> > it might have to interpret.
>
> Well, I wasn't thinking an x86 instruction set interpreter for that
> reply, although I had considered it previously. That would be
> complicated. I was thinking more of byte code in my reply.

OK. Of course, you need something to generate that byte code but you personally might be able to do that with your Forth translator. Anyone could generate 8086 code with a suitable compiler or assembler.

> Of
> course, that wouldn't be powerful enough, or correct, to execute
> BIOS in 64-bit mode... The big issue is do you need BIOS support
> in the OS or not? I.e., can BIOS usage be limited to the MBR/VBR
> and bootloader code? I think it can. I.e., 16-bit code can be
> used until transferring execution to the OS.

Two answers to that:

1. Yes, I think I might need to be able to execute the BIOS to adjust video settings. It seems impossible in a homebrew OS to do that portably otherwise.

2. I had an eye on a distant 64-bit OS being able to support very old 16-bit real-mode apps. So an emulator would still be needed.

> > And requiring code to compile to some binary that can execute in
> > either RM or PM could result it too many wrangles with the
> > compilers.
>
> That's true. Assembly would be the choice for this. A good
> assembler with a powerful pre-processor, like NASM, might help too.

If I follow then as above I had a different approach in mind, i.e. that the source code would be compiled to three different versions of the object code, one for each bit width of the OS. So there would be

/ --> 16-bit obj
Portable C Source ---> | --> 32-bit obj
\ --> 64-bit obj

16-bit C and asm ---> 16-bit obj
32-bit C and asm ---> 32-bit obj
64-bit C and asm ---> 64-bit obj

Then the obj files would be combined to form three working OS images.

My intention is that none of the above source files should contain conditionally included code. For a number of reasons ISTM better to write separate modules than have a given module with conditional code, if the modules can be kept small and their interfaces are well defined.

> > Wouldn't it be simpler to use a 16-bit compiler to build 16-bit
> > object files and a 32-bit compiler to build 32-bit object files?
>
> What exactly would be simpler? Two compilers instead of one?
> Or, building 16-bit object files and 32-bit object files
> separately?
>
> Two compilers is definately more work than just one. It's easier
> to build for just one mode. But, if you need to support three
> modes, you need three binaries... That's why I was talking about
> an interpreter. I can restrict the custom instruction size to
> something easy to support across multiple modes, say 8-bits or
> 16-bits.

Could you say some more about why two compilers are more work than one? You might save me going down a bad road.

> > Both could be built from the same HLL source.
>
> That's great, where possible. It's never 100%.

Would having some modules be mode-specific allow the bulk of code to be truly portable?

> > (They would need to be linked with other object files which
> > were mode specific in order to generate the complete OS.)
>
> Is this binary mixed-mode code? Both 16-bit and 32-bit in one
> binary? Yeah, I just haven't considered that. I'd prefer entirely
> 16-bit, or entirely 32-bit, etc.

No, I don't mean the binaries to run in any more than one mode. I thought you did based on your comment:

"That's dual mode 16-bit and 32-bit without any operand and
address size overrides. I was hoping to find a way to do
16-bit and 32-bit and 64-bit. But, I'm not sure that that is
possible because of the REX prefix... You'd have to find a
way to nullify the effects of it's operation in 16-bit and
32-bit code."

Sorry if I misunderstood it.

> > Speaking of compilers I have been using bcc (Bruce's C Compiler,
> > not Borlands) to generate 16-bit code and gcc to generate 32-bit
> > code. Would you recommend I change to another compiler for 16-bit
> > code?
>
> I don't know.
>
> Can the 16-bit code be converted to NASM? I.e., move as much C
> code into the 32-bit code as possible, and implement the remainder
> in assembly.

Thanks for the suggestion. I don't think I need to do that. I hope it will be possible to write quite a lot of code in C and have it compile as mentioned above.

> Are you using GCC on Linux or via DJGPP for DOS? DJGPP did
> release a minimal 16-bit backend. I've not used it.
> http://delorie.com/djgpp/16bit/

I have made a start with gcc and bcc on Linux but don't want the C code to use any pragmas or specific features of either of them. I don't want to get locked into any specific compiler.

Thanks for the pointer to the backend for DJGPP though it looks too experimental to depend on.

...

> I can produce 16-bit exe's for a number of my DOS applications
> without many changes with OpenWatcom. However, certain pointers
> and addressing related code requires C #if-def sections to adjust
> for each compiler. There is nothing you can do about that except
> do what the compiler wants done.

Open Watcom would be good if the source it was compiling used none of OW's special features. It seems to have many!

> My OS is 32-bit. It has a small amount of 16-bit code in NASM
> assembly which is separate, standalone from the C code. I was
> using two DOS C compilers: DJGPP (GCC) which supports 32-bit DPMI
> and OpenWatcom which supports 32-bit DPMI and 16-bit. When I need
> assembly combined with the C code, I use inline assembly (GAS,
> WASM) in C, instead of linking in NASM object files. I was
> planning on dropping OW support going forward. It does produce
> faster code, and catch a few extra coding errors. But, supporting
> two compilers, when DJGPP seems to be my preferred compiler, is
> probably not worth it...

If you wanted to would it be practical to change the C source so it would compile under any compiler? Could the stuff that makes some code compiler-specific including possibly inline assembly be written in separate modules and linked to the C?

James

[James Harris]

On Tuesday, July 9, 2013 6:38:23 PM UTC+1, wolfgang kern wrote:
> James Harris asked:
>
> |Have any of you already worked out the following? Or do you want a way to
> |run 16-bit code under a 64-bit OS?
>
> My 64-bit OS variant is still under development...
> But when I look at VESA-extensions (beside the opportunity to call
> 16-bit BIOS/VBIOS functions that will work on PM16 similar to RM16)
> there would this easy switch from LONG to RM16 become quite handy.

I think the big problem with switching back down to Real Mode and back up again is that the RM code will run without the security of the rest of the OS.

> |I heard long ago that AMD did not support use of 16-bit code when in Long
> |mode but I see that Long mode does in fact support some form of 16-bit
> code. |Is that protected mode where the code and data segments have default
> address |size and operand size set to 16 bits?
>
> |Is 16-bit Protected mode the right term?
>
> yes, even PM16 is different to VM86 and BIG-Real/Unreal-modes.
>
> |Whatever it's called it seems an unusual mode to run in.
>
> The BIOS itself uses it on power-up, and I (my OS) have to use it whenever
> a transition from PM32 to true RM is requested (just one far jump anyway).

Ah, no, slightly different thing. Real mode is the boot mode but PM16 uses 16-bit values but with PM segment registers. I didn't know anything about it until recently.

...

> |I see that V8086 mode and Real mode won't work under Long mode but wonder
> |how much can be done in the above 16-bit mode. Is it useful or not?
>
> Methink it's not for nothing that AMD supports PM16 code in LONG-mode
> while I'm not happy that all PM32-support is gone then.
> I also would appreciate if I could disable paging in LONG-mode.

PM32 is still supported in Long mode. So is PM16. But RM and V8086 are disabled.

As you say, you are stuck with paging!

James

[James Harris]

On Wednesday, July 10, 2013 2:57:43 AM UTC+1, s_dub...@yahoo.com wrote:
> On Sunday, July 7, 2013 6:30:20 PM UTC-5, James Harris wrote:
> > Have any of you already worked out the following? Or do you want a way to run 16-bit code under a 64-bit OS?
> >
> >
> >
> > I heard long ago that AMD did not support use of 16-bit code when in Long mode but I see that Long mode does in fact support some form of 16-bit code. Is that protected mode where the code and data segments have default address size and operand size set to 16 bits?
> >
> >
> >
> > Is 16-bit Protected mode the right term?
> >
> >
> >
> > Whatever it's called it seems an unusual mode to run in.
> >
> >
> >
> > Would it be feasible to run old 16-bit apps in this mode? If not, what would cause them to fail?
> >
> >
>
> I'm curious to know. I don't have a 64bit machine available to test on.

You could set up a 64-bit OS in an emulator - at least in theory. I have been trying that today without success but I was trying to set up a minimal distribution. I'll try again with something easier to work with.

> I have tried 16bit PM on a 32bit machine to explore some of 'what would cause them to fail'. You can switch in and out of PM without the need for setting CS to a selector of a descriptor, this is counter to what the manuals say: (switch to a selector using jmp far selector:offset at the earliest opportunity). You can co-mingle RM and 16bit PM code and data easy enough. I alias RM data with a 16bit PM data selector in FS, and use FS segment over-ride in the PM code to access the same data offsets. In the PM code governed by the RM CS the data references require a PM selector (as I say I used FS:same_offset for that), but this extends to the stack as well, for 'calls', requiring a selector for SS, so a stack switch is what I did for that to allow 'calls' to happen despite the fact that CS is not changed. I wanted to use a small PM stack for that. Trying to do that reveals an interesting 'gotcha': a fault occurs if the current SP value is outside the limit of the descriptor when its selector is to be loaded into SS, even tho the programmer is doing FS: LSS SP, [Location].
>
> There is also a minimum PM stack size for a small stack. (I wish I had my notes and example code, but I lost them in a hard drive crash).

Sounds like you are setting up a stack segment. These days it might be more normal to overlay the stack segment and the data segment so that languages can address both interchangeably with non-segmented pointers. In that case minimum stack is possibly 4k. Just a thought.

> Considering the above, 16bit RM <-> 16bit PM, isn't quite what you are aiming for with Long Mode, also note that Long Mode drops support for some instructions; gone are the BCD ones. I use 'aam 16' often, and that is gone.

aam 16 is a funny one to miss. It is probably exceptionally slow and is easy to replace by shifting or even a division, isn't it?

...

> > Could the BIOS be run in that mode, perhaps with the OS emulating some operations?
> >
> In the scheme above, I drop out of PM to call a BIOS function, even in repeating looping code where the heart of the loop is in PM. That is a mode switch to call the BIOS, not what you are after it seems.

Right, the BIOS *should* be safe to call but because dropping back to RM makes the OS lose its security I don't want to do that. (For the same reason I have no intention of supporting APM for power management - not a great loss.)

James

[Rod Pemberton]

"James Harris" <james.h...@gmail.com> wrote in message

news:be55406f-2cf3-4097...@googlegroups.com...

> I use 'aam 16' often, [...]

For what?

> It is probably exceptionally slow and is easy to replace by
> shifting or even a division, isn't it?

Won't DIV or IDIV do? They return quotient and remainder too.
Both have 8-bit and 16-bit/32-bit versions. I would guess that the
8-bit versions should be the same or very similar, especially the
unsigned 8-bit DIV... It looks you might just need an "AND AX,
00Fh" or "MOVZX AX, AL" to clear AH, and move of the imm8 into a
8-bit register. That should make them "the same" or very close...
Personally, I'd have to setup some code to find out for sure. So,
I'm not going to do that. It would be interesting to know though.

> In the scheme above, I drop out of PM to call a BIOS function,
> even in repeating looping code where the heart of the loop is in
> PM. That is a mode switch to call the BIOS, not what you are
> after it seems.

I'm beginning to think that I need a map as to what BIOS functions
are available for PM and what isn't... It appears there is far
more available than I realized for many years. I recently found a
memory call in Phoenix BIOSes is for PM and whether it's for 16-bit
or 32-bit. Apparently, a number of BIOS extensions, like perhaps
PnP, APM, ACPI, VESA 2.0, VESA 3.0, BIOS32, etc have PM support.
There are also a couple of standard BIOS calls, i.e., should work
in RM, for setting up and switching to PM and moving memory. I
guess I should start combing the specifications and RBIL ...

> Right, the BIOS *should* be safe to call but because dropping
> back to RM makes the OS lose its security I don't want to do
> that.

Does anyone have any familiarity of how 16-bit PM was used in other
OSes? Say, perhaps Windows 3.1 or OS/2? I'm wondering *if* they
called the BIOS in 16-bit PM. If so, that could present some
circumstantial evidence that BIOS are mostly safe for 16-bit PM.

I saw a post by the author of SeaBIOS where he was asking whether
he could remove 16-bit PM support or not... He wanted to know
which OSes and applications called the BIOS in 16-bit PM. I don't
know whether I should read into that and think some OS called the
BIOS in 16-bit PM, or whether I should take his old post as meaning
he didn't know if anything at all called the BIOS in 16-bit PM.
That's what prompted the question above.

> (For the same reason I have no intention of supporting APM for
> power management - not a great loss.)

Does it have a PM interface? A number of such specifications do.
I thought I saw APM listed as one that did.


Rod Pemberton

[Rod Pemberton]

"James Harris" <james.h...@gmail.com> wrote in message

news:70bcdacb-8024-408b...@googlegroups.com...

> On Tuesday, July 9, 2013 10:11:08 PM UTC+1, Rod Pemberton wrote:
> > "James Harris" <james.h...@gmail.com> wrote in message
> > news:911de716-4485-493c...@googlegroups.com...

> > > [snip]
> > [snip]

> The key thing is the formats of IVT and 32-bit IDT are completely
> different and so would need some different source code to manage
> them.

Are you trying to manage them using the exact same code? #if-def's
do work...

There is no IDT until you set it up. So, you need code to install
it and manage it. The IVT is setup by the BIOS. Do you need code
to manage the IVT? How many RM interrupts does a PM OS need to
"fix" ... ? I'm assuming that for security, which you mentioned
somewhere, if you have a multi-mode 16-bit/32-bit/64-bit OS, then
all three modes will be PM, including the 16-bit mode. I.e., no
v86, no 16-bit RM. In which case, you have an IDT for all three,
but the sizes of data are likely different. I don't know what
you'd do about needing v86 or 16-bit RM for BIOS calls. We need to
find out if 16-bit PM can be used for BIOS calls.

> To try to illustrate what I had in mind the following could be
> the code for reading the CMOS/RTC chip.
>
> int cmos_read(int addr)
> {
> if (addr > ADDR_MAX)
> return -1;
> wait_if_needed(addr);
> addr_write(addr);
> return port_8_in(DATA_PORT);
> }
>

I'd expect "you", i.e., the OS you're creating or planning, to be
the _only_ caller of cmos_read(). I wouldn't expect a user to be
able to call it since it accesses a port. You can't be as
successful as MS without angering your users. Access denied! So,
we'll say that's for security reasons... ;-) So, is there any
reason at all to check for "addr>ADDR_MAX"? That seems to be
saying you can't trust your own OS which you coded to pass in the
correct value to cmos_read()... :-) If hackers do manage to call
it somehow, the 'if' would block bad port numbers. The question is
what happens if you attempt to write to a non-existant port number?
Does it get masked or wrapped to be within the valid address range?
I want to say it does... But, I'd need to pull out the manuals.

> It's untested but the idea is that the same source code should
> compile to three different outputs: 16-bit, 32-bit and 64-bit
> object code and work in the same way on each of them. The
> int sizes would differ but not what the code does.

8-bits, AFAIK, are supported in all three mode sizes on x86 as
integers and for ports. Of course, I'm not so familiar with
64-bits... As long as the int size is sufficiently large for all
modes for the returned data, that seems correct to me. I'm
assuming the data size is 8-bits from "port_8_in". I.e, returning
8-bits "promoted" or "implicitly" cast by the compiler to the
default integer size, most likely larger in size, should work for
16-bit, 32-bit, 64-bit without issue. Now, if there was a 32-bit
value that needs to be used in all three modes, it'll result in
splitting or joining operations for 16-bit mode, if you're using C.
If using assembly, you could use a size override. Then, the
question becomes, if you're using C, do you want to use the lowest
common denominator by spliting or joining for all three modes, or
do you want to use mode based #if-def code?

> The thing is that even with segments and whether paging is turned
> on or not, on x86-32 all the PM addresses that programs use are
> ultimately 32-bits wide. RM addresses are 16-bit or 20-bit.

v86, (and probably) 16-bit PM, are not 32-bits... For 32-bits, the
offset is large enough to address typical maximum hardware memory.
IIRC, for 64-bits, the offset is limited also, perhaps to 32-bits?
I.e., 64-bits has same issue as RM addresses: you can't address
*all* physical memory with a single offset, you may need to change
the selector or segment.

> [x86 interpreter]

Although, there definately are merits to a full 16-bit x86 binary
translator... It's just nobody really wants to code one for their
OS. A 32-bit PM monitor with v86 mode support and with a built-in
16-bit to 32-bit binary translator would progressively convert all,
or nearly all, 16-bit RM/v86 code from the BIOS, video BIOS, option
roms, DOS, DOS apps, DOS TSRs, to 32-bit. I.e., you'd have a
32-bit DOS OS in no time from an old 16-bit OS. However, you'd
still be limited to whatever drivers are available to DOS
community... I.e., if there is no 16-bit DOS SATA driver, there is
no SATA support.

> > Of course, that wouldn't be powerful enough, or correct, to
> > execute BIOS in 64-bit mode... The big issue is do you need
> > BIOS support in the OS or not? I.e., can BIOS usage be limited
> > to the MBR/VBR and bootloader code? I think it can. I.e.,
> > 16-bit code can be used until transferring execution to the OS.
>
> Two answers to that:
>
> 1. Yes, I think I might need to be able to execute the BIOS to
> adjust video settings. It seems impossible in a homebrew OS
> to do that portably otherwise.

You can do the VGA and lower stuff, because that hardware register
set is defined. I'm thinking you're probably aware of this, but
it's the non-standardized SVGA interfaces and VESA modes that are
an issue. First, their hardware interfaces aren't defined, i.e.,
must call BIOS to program the hardware. Second, the newest hi-res
modes have different mode numbers for each video card.

A while back I mentioned to Alexei that one could possibly use v86
mode to trap the ports and values for each required video BIOS
call. Then, you could program the card. He pointed out that there
could be timing issues or unknown wait loops etc. But, I think
it's still a valid concept, although it may need tweaking to
actually work correctly all the time, on every machine, because of
such issues. It's possible such an idea could work on very many or
very few machines, but if it worked on 50% to 80%, that might be
good enough. Obviously, an incompatibility disclaimer or DOS test
program should be offered so an interested user could verify
compliance of his machine with your OS, etc.

> 2. I had an eye on a distant 64-bit OS being able to support very
> old 16-bit real-mode apps. So an emulator would still be needed.

...

> If I follow then as above I had a different approach in mind,
> i.e. that the source code would be compiled to three different
> versions of the object code, one for each bit width of the OS.
>
> So there
> would be
>
> / --> 16-bit obj
> Portable C Source ---> | --> 32-bit obj
> \ --> 64-bit obj
>
> 16-bit C and asm ---> 16-bit obj
> 32-bit C and asm ---> 32-bit obj
> 64-bit C and asm ---> 64-bit obj
>
> Then the obj files would be combined to form three
> working OS images.

Um, what I understand you to be saying, and what the diagram tells
me are two different things. This is what I understand you to be
saying:

/ --> 16-bit obj
Portable C Source ---> | --> 32-bit obj
\ --> 64-bit obj

16-bit asm and Port. C Src. 16-bit obj ---> 16-bit OS image
32-bit asm and Port. C Src. 32-bit obj ---> 32-bit OS image
64-bit asm and Port. C Src. 64-bit obj ---> 64-bit OS image

Where the obj from the Portable C Source is taken and combined with
additional asm to produce the image. Is that correct?

Or, is your diagram correct? I take your diagram to mean you have
an obj from Portable C Source plus *another* obj from a mix of
non-portable 16-bit C and non-portable 16-bit asm. That seemed odd
to me. Why is there more C that's not in the Portable C Source?
I.e., avoid non-portable C. So, I would think all or most
non-portable code would be in asm (or could be...) or is inlined
assembly in C.

BTW, by "non-portable" I've been assuming this is _just_ across x86
cpu modes, i.e., non-portable from 16-bit to 32-bit to 64-bit x86.
If you meant "non-portable" in the C sense of portability across
different platforms, e.g., from x86 to ARM to Cray to Vax to 6502
etc, I misunderstood.

> My intention is that none of the above source files should
> contain conditionally included code. For a number of reasons
> ISTM better to write separate modules than have a given module
> with conditional code, if the modules can be kept small and their
> interfaces are well defined.

Creator's choice! Bonus.

> Could you say some more about why two compilers are more
> work than one? You might save me going down a bad road.
>

The two compilers I used have some very different ways to do
things. Of course, it's very possible I wasn't doing some things
the correct way. So, searching my code for my OS (long list):

-They call DPMI functions differently. That's not important to
you.
-The inline assembly syntax is different for each compiler, i.e.,
#if-defs. You'll need inline assembly for various instructions etc
like: lgdt, sgdt, lidt, interrupt vectors, cr0, lar, lsl, cs, ds,
es, fs, gs, ss, sp, wbinvd, rdtsc, hlt, int, smsw, etc.
-The inline assembly of one compiler doesn't support forward
references! (nightmare...)
-The two compilers have different naming conventions, i.e., those
underscores on names.
-They address registers in register structures differently. You
can fix this with conditional code or defines.
-They need different include files to load register structures. Of
course, you could define your own.
-They both have different naming for their inport and outport C
routines. They are in different include files. Of course, you
could define your own, but each has different inline assembly
syntaxes... I.e., no way to make uniform.
-They use a different method address C objects. One compiler uses
physical addressing. The addresses correspond to the physical
memory locations. This is true of memory below 1MB and the
application space. The other compiler uses relative addressing.
So, the application space starts at offset zero, no matter where
it's loaded. This means you have to adjust any address to memory
below 1MB by the load offset.
-The code they generate is for DOS DPMI, i.e., 32-bit code. Each
DPMI host set's up it's own GDT. However, when using these
compilers for a custom OS that won't have DPMI available, the
compiled C code is dependent on the GDT entries each compiler needs
for it's compiled code. I.e., one compiler needs 4 GDT entries
while the other needs 5.
-One compiler packs structs by single bytes. The other needs a
#pragma to do so.
-I'm not using them, but I found out the format of the C jmpbuf
structure was different for each.
-Each compiler has their own cli and sti _routines_, not
instruction... You need to use the compiler's routine, not roll
your own. Each of these is in a different include file too.
-I had to code some code to replace the initialization of various
data items that were set as part of the executable startup which
were no longer present once I abused the compiler to produce code
for my OS...
-The routines that will move and copy to memory below 1MB are
different.
-The compiler methods to inline routines is different for each.
One uses an declaration with an attribute and the other uses a
compiler keyword.
-One compiler had a way to make "naked" C routines, i.e., no
prolog(ue), no epilog(ue), no stack frame code, no code clean,
i.e., "clean" assembly. The other compiler would only support
"void func(void)" which leaves a minimal stack frame. This
eliminated the prolog and epilog but keeps the stackframe code. It
required use of the 'leave' instruction to remove the stack frame.
I suspect I'm overlooking or unaware of some compiler feature...
You're likely going to need a naked function for an interrupt
wrapper.
-One compiler could generate both near and far returns
appropriately for procedures while the other would only generate
near. That prevented adjusting the "naked" function's return code
via a far ret. Optimization also prevented adjustment of the
procedure return instruction for that compiler. Additional
instructions were required to manually adjust the IP.

> > > Both could be built from the same HLL source.
>
> > That's great, where possible. It's never 100%.
>
> Would having some modules be mode-specific allow
> the bulk of code to be truly portable?

30% of C is portable.
30% of C appears portable, i.e., close equivalent.
40% of C is not portable and never will be.

:-)

Your asking if, say the BIOS video routines are always 16-bit RM,
can the rest of the OS be portable between modes? I've not
compiled my OS for 16-bit. I'm assuming all the C code could be
made to work for 16-bits fairly easily, but not without some
changes. Off hand, I can't think of anything that's 32-bit
specific, but I haven't worked on it in some years now... Of
course, there are the addressing limitations for 16-bit code that
have to be fixed by small #if-defs. The few inline assembly
_routines_ would need new routines coded as 16-bit, e.g., interrupt
wrappers, instead of 32-bit. The standalone inline assembly
_instructions_ should probably compile as-is.

> > > (They would need to be linked with other object files which
> > > were mode specific in order to generate the complete OS.)
>
> > Is this binary mixed-mode code? Both 16-bit and 32-bit in one
> > binary? Yeah, I just haven't considered that. I'd prefer
> > entirely 16-bit, or entirely 32-bit, etc.
>
> No, I don't mean the binaries to run in any more than one mode.
> I thought you did based on your comment:
>
> "That's dual mode 16-bit and 32-bit without any operand and
> address size overrides. I was hoping to find a way to do
> 16-bit and 32-bit and 64-bit. But, I'm not sure that that is
> possible because of the REX prefix... You'd have to find a
> way to nullify the effects of it's operation in 16-bit and
> 32-bit code."
>
> Sorry if I misunderstood it.

Sorry, taken together that's not especially clear, is it?

The part I hadn't considered was differently sized code segments in
the same file. This would be like an OS that supports both 32-bit
and 16-bit code segments at the same time, instead of a pure 32-bit
OS, etc. The dual mode idea (multi-mode now...) was intended for
minimal assembly startup, bootloader, bootstrap interpreter, etc.
There is no way I'd want to code anything of sufficient size in
that. There is no way to get a compiler to emit such code either.
I.e., develop the technique, assemble code for 16-bit, disassemble
for 16-bit, 32-bit, and 64-bit, confirm code functions
equivalently. PIA.

> I have made a start with gcc and bcc on Linux but don't want the
> C code to use any pragmas or specific features of either of them.

You may have a hard time without using a pragma. GCC needs one to
pack structures correctly, or an attribute... Some of the x86
system structures need to be aligned correctly too. That requires
a pragma, or directive, or attribute too.

> I don't want to get locked into any specific compiler.

I'd say "good luck"! But, that could just be my experience or
incorrect path.

> If you wanted to would it be practical to change the C source
> so it would compile under any compiler?

Any compiler, any C compiler, or any compiler or assembler?

There is a bit of inline assembly. That syntax is custom to the
compiler. E.g., OpenWatcom uses WASM which is MASM compatible
syntax, DJGPP (GCC) uses GNU AS (GAS) which is AT&T syntax. If I
move the assembly out of the C code into assembly files, I could
convert it to say NASM. Then, the C code would be purely C code,
but it'll still need compiler specific tweaks because of the x86
assembly produced, different include files, differently named
functions, functions which work differently, etc.

Of course, I'm not up on name demangling, so I might be overlooking
some feature that solves a few more of these issues. IIRC, Chris
Giese in his code used many #defines to make the main body of C
code in each file identical. However, there was a block of
compiler specific defines at the top of each file. My code has
compiler specific includes at the top and a bunch of #if-defs
throughout to adjust for each compiler.

> Could the stuff that makes some code compiler-specific
> including possibly inline assembly be written in separate
> modules and linked to the C?

Yes. But, you can't eliminate all of it, AFAIK. E.g., 32-bit code
produced by DJGPP needs 5 descriptors, whereas OW needs 4. The
setup is compiler specific. Of course, I'm not using "standalone"
or "freestanding" compiler options, or compiler flags to select
independence from the C library, or linker scripts. I'm using a
standard DOS DPMI executable intended to be executed under DOS.
This would be like using a standard ELF executable on Linux or
standard PE on Win32 for your OS. Most people don't do that. :-)
So, those things may be a big factor that helps you eliminate some
of the issues I have.


Rod Pemberton

[s_dub...@yahoo.com]

On Wednesday, July 10, 2013 7:34:27 PM UTC-5, Rod Pemberton wrote:
> "James Harris" <james.h...@gmail.com> wrote in message
>
> news:be55406f-2cf3-4097...@googlegroups.com...
>
>
>
> > I use 'aam 16' often, [...]
>
>
>
> For what?
>

;;-----------------------------------------------------------------------75
;; -= Convert Byte in AL to ascii chrs AH, AL =-
;;-----------------------------------------------------------------------75

[SECTION .code]

;;--- convert byte value in AL to ascii hex ---
;; only AX is affected.

Byte2_Ascii:
.conv:
aam 16 ;; split byte into nibbles, AH, AL

cmp ah, 9
ja .conv_alpha_h ;; it is A..F
;; otherwise a digit
add ah, 30h ;; make ascii number
jmp short .conv_l

.conv_alpha_h:
add ah, 37h ;; 37h + Ah..Fh = ascii 'A'..'F'

.conv_l:
cmp al, 9
ja .conv_alpha_l ;; it is A..F
;; otherwise a digit
add al, 30h ;; make ascii number
jmp short .cont

.conv_alpha_l:
add al, 37h ;; 37h + Ah..Fh = ascii 'A'..'F'

.cont: ;; AX has ascii characters for original byte value nibbles.

RET

So this is a subroutine to hexadecimal ascii-fy a byte (a word, a double word,..)..which can be poked into a string for console output.

I know you debug by message output, and this is used to message a runtime value, among other uses involving hexadecimal ascii conversions.

>
>
> > It is probably exceptionally slow and is easy to replace by
>
> > shifting or even a division, isn't it?
>
>
>
> Won't DIV or IDIV do? They return quotient and remainder too.
>
> Both have 8-bit and 16-bit/32-bit versions. I would guess that the
>
> 8-bit versions should be the same or very similar, especially the
>
> unsigned 8-bit DIV... It looks you might just need an "AND AX,
>
> 00Fh" or "MOVZX AX, AL" to clear AH, and move of the imm8 into a
>
> 8-bit register. That should make them "the same" or very close...
>
> Personally, I'd have to setup some code to find out for sure. So,
>
> I'm not going to do that. It would be interesting to know though.
>
>
>
> > In the scheme above, I drop out of PM to call a BIOS function,
>
> > even in repeating looping code where the heart of the loop is in
>
> > PM. That is a mode switch to call the BIOS, not what you are
>
> > after it seems.
>
>
>
> I'm beginning to think that I need a map as to what BIOS functions
>
> are available for PM and what isn't... It appears there is far
>
> more available than I realized for many years. I recently found a
>
> memory call in Phoenix BIOSes is for PM and whether it's for 16-bit
>
> or 32-bit. Apparently, a number of BIOS extensions, like perhaps
>
> PnP, APM, ACPI, VESA 2.0, VESA 3.0, BIOS32, etc have PM support.
>
> There are also a couple of standard BIOS calls, i.e., should work
>
> in RM, for setting up and switching to PM and moving memory. I
>
> guess I should start combing the specifications and RBIL ...
>
>

Please let us know what you find.

I hadn't found that much, mostly service specific to check service specific stuff, like version, or service entry. AIR, those that support PM entry require 16bit PM selector(s) setup prior to making the call.

After all this time, it wouldn't surprise me at all to learn of undocumented PM32/64 interface to the RomBios services.

>
> > Right, the BIOS *should* be safe to call but because dropping
>
> > back to RM makes the OS lose its security I don't want to do
>
> > that.
>

Oh, security, yes I see what you mean. (to James)

>
>
> Does anyone have any familiarity of how 16-bit PM was used in other
>
> OSes? Say, perhaps Windows 3.1 or OS/2? I'm wondering *if* they
>
> called the BIOS in 16-bit PM. If so, that could present some
>
> circumstantial evidence that BIOS are mostly safe for 16-bit PM.
>

I managed to install and run graphical Windows 3.0 on an XT clone, so no PM was absolutely necessary, but it did use PM if available.

AIUI, the first OS/2 used PM16 and cpu task-switching.

>
>
> I saw a post by the author of SeaBIOS where he was asking whether
>
> he could remove 16-bit PM support or not... He wanted to know
>
> which OSes and applications called the BIOS in 16-bit PM. I don't
>
> know whether I should read into that and think some OS called the
>
> BIOS in 16-bit PM, or whether I should take his old post as meaning
>
> he didn't know if anything at all called the BIOS in 16-bit PM.
>
> That's what prompted the question above.
>

Is SeaBIOS mature, is it offered by any OEM for their motherboard, out of the box? Is it open source, (source code available)?

Steve

[wolfgang kern]

Rod Pemberton figured ...

...

>> The BIOS itself uses it on power-up, and I (my OS) have to use it
>> whenever a transition from PM32 to true RM is requested (just
>> one far jump anyway).

> I might be mistaken, but I think you're talking about the old PM16
> while James is talking about the new CM16... (?)
>
> Well, PM16 and CM16 are the names Sandpile.org has assigned to
> these modes... CM16 is the "new" 16-bit long mode (EFER.LMA=1).
> Go here:
> http://www.sandpile.org/x86/mode.htm
>
> Or, are you saying that new BIOSes use CM16? If so, what for?
> Couldn't they just use PM16?

You're right, I interpreted 'PM16' in the old meaning :)

__
wolfgang

[wolfgang kern]

Rod Pemberton wrote in part:

...

> RM has an IVT. IDTs are for PM. But, yes, a RM IVT and a 32-bit
> IDT, have differently sized data. Does a 16-bit PM IDT have the
> same _size_ data as a RM IVT, not _type_? I'm not sure. I'd guess
> that they do. So, maybe, mode-specific code, not bit-width
> agnostic, would be more appropriate? I.e., RM is 16-bits and one
> of the PMs is 16-bits.

IDT and GDT seem to show the same layout for both, PM16 and PM32.
while the RM code range is limited to 0x0010_ffef, PM16 may reside
anywhere within 4GB.

My code is a mix of RM(fast), PM16(short) and PM32(convenient)-routines.
And with a few tricks they can even call each other.
But I got only one IDT which contains PM16 and PM32 interrupt-gates.
Legacy hardware like RTCL, keybd and mouse use the same routines for
PM16 and RM (with different entry-points to set DS for equal range).

IIRC: win3.11 ran on i286 in PM16 only ... and needed to crash if
it wanted RM again :)
__
wolfgang

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message
news:krl67p$a48$1...@speranza.aioe.org...

> "James Harris" <james.h...@gmail.com> wrote in message
> news:70bcdacb-8024-408b...@googlegroups.com...
>> On Tuesday, July 9, 2013 10:11:08 PM UTC+1, Rod Pemberton wrote:
>> > "James Harris" <james.h...@gmail.com> wrote in message
>> > news:911de716-4485-493c...@googlegroups.com...
>
>> > > [snip]
>> > [snip]
>> The key thing is the formats of IVT and 32-bit IDT are completely
>> different and so would need some different source code to manage
>> them.
>
> Are you trying to manage them using the exact same code? #if-def's
> do work...

No, the 16-bit IVT and the 32-bit IDT would be managed by different source
code modules but they should provide the same interface to the rest of the
OS.

BTW, when you say #if-defs do you mean one of these:

Opt 1: #if (cond) code #elif (cond) code #else code #endif
Opt 2: #if (cond) #define ... #endif
Opt 3: #ifndef #include ... #endif

> There is no IDT until you set it up. So, you need code to install
> it and manage it. The IVT is setup by the BIOS. Do you need code
> to manage the IVT?

The idea would be to have a 16-bit source-code module which manages an IVT
(i.e. a piece of software written specifically to execute in 16-bit real
mode to deal with the interrupt vector table that exists in a real-mode
environment) and another module of source code which manages a 32-bit IDT
and another module of source code which manages a 64-bit IDT. The first
would appear only in the 16-bit form of the OS, the second would only appear
in the 32-bit OS etc. Only one OS would run at a time. The 16-bit OS could
not be fully secure because that mode of the hardware does not permit the OS
to restrict what user programs do. The others would be secure.

> How many RM interrupts does a PM OS need to
> "fix" ... ?

I'm not sure I understand the question but from memory I think a 32-bit PM
OS would only deal with 32-bit PM interrupts. Even if it is executing RM
code in a virtual 8086 monitor or executing a PM16 code segment the
interrupts would still, I think, fire as full 32-bit ones.

> I'm assuming that for security, which you mentioned
> somewhere, if you have a multi-mode 16-bit/32-bit/64-bit OS, then
> all three modes will be PM, including the 16-bit mode.

To be clear, I wasn't thinking of there being one OS operating in three
modes but that there would be three OSes (a real mode one, a PM32 one and a
64-bit one). However, they would all be built from source code that included
a lot of code that worked on all three. Was that what you were talking
about?

> I.e., no
> v86, no 16-bit RM. In which case, you have an IDT for all three,
> but the sizes of data are likely different. I don't know what
> you'd do about needing v86 or 16-bit RM for BIOS calls. We need to
> find out if 16-bit PM can be used for BIOS calls.

Based on comments made in this thread it's probably safest to run real mode
code such as the BIOS in an emulator, unless running the 16-bit OS when the
BIOS etc can be run directly.

>> To try to illustrate what I had in mind the following could be
>> the code for reading the CMOS/RTC chip.
>>
>> int cmos_read(int addr)
>> {
>> if (addr > ADDR_MAX)
>> return -1;
>> wait_if_needed(addr);
>> addr_write(addr);
>> return port_8_in(DATA_PORT);
>> }
>>
>
> I'd expect "you", i.e., the OS you're creating or planning, to be
> the _only_ caller of cmos_read(). I wouldn't expect a user to be
> able to call it since it accesses a port. You can't be as
> successful as MS without angering your users. Access denied! So,
> we'll say that's for security reasons... ;-) So, is there any
> reason at all to check for "addr>ADDR_MAX"? That seems to be
> saying you can't trust your own OS which you coded to pass in the
> correct value to cmos_read()... :-)

Quite right. Defensive code!

> If hackers do manage to call
> it somehow, the 'if' would block bad port numbers. The question is
> what happens if you attempt to write to a non-existant port number?
> Does it get masked or wrapped to be within the valid address range?
> I want to say it does... But, I'd need to pull out the manuals.
>
>> It's untested but the idea is that the same source code should
>> compile to three different outputs: 16-bit, 32-bit and 64-bit
>> object code and work in the same way on each of them. The
>> int sizes would differ but not what the code does.
>
> 8-bits, AFAIK, are supported in all three mode sizes on x86 as
> integers and for ports. Of course, I'm not so familiar with
> 64-bits... As long as the int size is sufficiently large for all
> modes for the returned data, that seems correct to me. I'm
> assuming the data size is 8-bits from "port_8_in". I.e, returning
> 8-bits "promoted" or "implicitly" cast by the compiler to the
> default integer size, most likely larger in size, should work for
> 16-bit, 32-bit, 64-bit without issue.

Yes.

> Now, if there was a 32-bit
> value that needs to be used in all three modes, it'll result in
> splitting or joining operations for 16-bit mode, if you're using C.

Yes. As an example, I wondered how a compiler would return a double-width
integer. On the two compilers I tried an integer which is twice the size of
an int gets returned in dx:ax or edx:eax.

Notably, someone pointed out to me that the default int size for a 64-bit
machine may not be 64 bits but 32. I haven't tested that yet. Something to
watch out for.

> If using assembly, you could use a size override. Then, the
> question becomes, if you're using C, do you want to use the lowest
> common denominator by spliting or joining for all three modes, or
> do you want to use mode based #if-def code?
>
>> The thing is that even with segments and whether paging is turned
>> on or not, on x86-32 all the PM addresses that programs use are
>> ultimately 32-bits wide. RM addresses are 16-bit or 20-bit.
>
> v86, (and probably) 16-bit PM, are not 32-bits... For 32-bits, the
> offset is large enough to address typical maximum hardware memory.
> IIRC, for 64-bits, the offset is limited also, perhaps to 32-bits?
> I.e., 64-bits has same issue as RM addresses: you can't address
> *all* physical memory with a single offset, you may need to change
> the selector or segment.

No, not at all. AIUI in long mode the addresses are 64 bits wide.

>
>> [x86 interpreter]
>
> Although, there definately are merits to a full 16-bit x86 binary
> translator... It's just nobody really wants to code one for their
> OS. A 32-bit PM monitor with v86 mode support and with a built-in
> 16-bit to 32-bit binary translator would progressively convert all,
> or nearly all, 16-bit RM/v86 code from the BIOS, video BIOS, option
> roms, DOS, DOS apps, DOS TSRs, to 32-bit. I.e., you'd have a
> 32-bit DOS OS in no time from an old 16-bit OS. However, you'd
> still be limited to whatever drivers are available to DOS
> community... I.e., if there is no 16-bit DOS SATA driver, there is
> no SATA support.

I'm not sure I would completely trust a program which translated 16-bit code
to 32-bit. Assembly programmers are free to play tricks that may not
translate reliably. Emulating the machine should be safer - even though it
would be a lot of work. (It may be feasible to work with an existing 8086
emulator.)

>> > Of course, that wouldn't be powerful enough, or correct, to
>> > execute BIOS in 64-bit mode... The big issue is do you need
>> > BIOS support in the OS or not? I.e., can BIOS usage be limited
>> > to the MBR/VBR and bootloader code? I think it can. I.e.,
>> > 16-bit code can be used until transferring execution to the OS.
>>
>> Two answers to that:
>>
>> 1. Yes, I think I might need to be able to execute the BIOS to
>> adjust video settings. It seems impossible in a homebrew OS
>> to do that portably otherwise.
>
> You can do the VGA and lower stuff, because that hardware register
> set is defined. I'm thinking you're probably aware of this, but
> it's the non-standardized SVGA interfaces and VESA modes that are
> an issue. First, their hardware interfaces aren't defined, i.e.,
> must call BIOS to program the hardware. Second, the newest hi-res
> modes have different mode numbers for each video card.

Yes, the non-VGA stuff is the issue.

> A while back I mentioned to Alexei that one could possibly use v86
> mode to trap the ports and values for each required video BIOS
> call. Then, you could program the card. He pointed out that there
> could be timing issues or unknown wait loops etc. But, I think
> it's still a valid concept, although it may need tweaking to
> actually work correctly all the time, on every machine, because of
> such issues. It's possible such an idea could work on very many or
> very few machines, but if it worked on 50% to 80%, that might be
> good enough. Obviously, an incompatibility disclaimer or DOS test
> program should be offered so an interested user could verify
> compliance of his machine with your OS, etc.

OK. Timing could be an issue under an emulator also. I'll keep it in mind.

Ah, no. Each OS image would be made from, effecively, three components:
portable C source, mode-specific C source and mode-specific assembly source.
If I redo the diagram to specify that more clearly:

/ --> 16-bit obj1
Portable C Source ---> | --> 32-bit obj1
\ --> 64-bit obj1

asm and non-portable 16-bit C ---> 16-bit obj2
asm and non-portable 32-bit C ---> 32-bit obj2
asm and non-portable 64-bit C ---> 64-bit obj2

Then 16-bit obj1 and 16-bit obj2 would be linked to form the 16-bit OS.
Similar for the other bit widths.

> Or, is your diagram correct? I take your diagram to mean you have
> an obj from Portable C Source plus *another* obj from a mix of
> non-portable 16-bit C and non-portable 16-bit asm.

Yes, that's what I intended. By "portable" and "non-portable" I mean code
that can or cannot be compiled to the different modes. Hopefully the bulk of
the code would be portable (in the sense that it would work in any of the
three modes). The non-portable code would only work in one mode, i.e. in one
bit-width.

> That seemed odd
> to me. Why is there more C that's not in the Portable C Source?

The use of non-portable C (non-portable in the sense mentioned above) is
intended to allow much of the OS to be written in a HLL. I figured that if I
was going to write in C I might as well see how far I could go to take
advantage of it.

> I.e., avoid non-portable C. So, I would think all or most
> non-portable code would be in asm (or could be...) or is inlined
> assembly in C.

All the non-portable stuff could be written in asm. I was thinking to see
how effectively it could be done in C, though.

ISTM that inline asm is to be avoided like the plague. I fear it may tie the
code to specific compilers. That is something I got to from discussions with
you and from trying some 16-bit code under OpenWatcom. I get the impression
that once the source code starts to include specifics that make a certain
compiler behave as we would want it to then the code can get increasingly
dependent on that compiler. Instead I would rather call separate assembly
routines where needed. Will have to see how it works out in practice.

> BTW, by "non-portable" I've been assuming this is _just_ across x86
> cpu modes, i.e., non-portable from 16-bit to 32-bit to 64-bit x86.
> If you meant "non-portable" in the C sense of portability across
> different platforms, e.g., from x86 to ARM to Cray to Vax to 6502
> etc, I misunderstood.

That's good. You are right. The only portability I have in mind just now is
between different x86 CPU modes. Anything else is too difficult to include
at this time. I did think about trying to write C code that would port to
different architectures but I don't know enough about them to do that yet.
Something for the back burner.

I'll reply to the rest of your post separately.

James

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message
news:krl67p$a48$1...@speranza.aioe.org...

...

Thanks for the info below - it's great to get the details to consider. Will
reply inline.

>> Could you say some more about why two compilers are more
>> work than one? You might save me going down a bad road.
>>
>
> The two compilers I used have some very different ways to do
> things. Of course, it's very possible I wasn't doing some things
> the correct way. So, searching my code for my OS (long list):
>
> -They call DPMI functions differently. That's not important to
> you.

True.

> -The inline assembly syntax is different for each compiler, i.e.,
> #if-defs. You'll need inline assembly for various instructions etc
> like: lgdt, sgdt, lidt, interrupt vectors, cr0, lar, lsl, cs, ds,
> es, fs, gs, ss, sp, wbinvd, rdtsc, hlt, int, smsw, etc.
> -The inline assembly of one compiler doesn't support forward
> references! (nightmare...)

I want to avoid inline asm (because different compilers implement it
differently) so the above two points should not be an issue.

> -The two compilers have different naming conventions, i.e., those
> underscores on names.

Ah, yes. I already came across this. It's a pain having to write the
assembly to work with comilers which do and which don't add underscores to
names. And some (OpenWatcom IIRC) change where they put the underscore -
before or after the name - depending on how the symbol was declared.

I tried writing some Nasm macros which would adapt to the naming conventions
of different compilers. That's probably a good way to go but it does make
the assembly source less readable.

> -They address registers in register structures differently. You
> can fix this with conditional code or defines.

Do you mean things like

struct regs {
int ax;
int bx;
};

> -They need different include files to load register structures. Of
> course, you could define your own.

If I understand, here's what I have been trying for the register save code
of the scheduler. The *C* code for task switching is CPU-agnostic in that it
doesn't know what type of CPU it is running on but it saves and restores any
thread data that it knows to differ between threads. It works with an
assembly routine which saves and restores the CPU's register set. This
should make the scheduler portable between x86 CPU modes. (In this case it
also makes it portable to different architectures but that's not the current
intention.)

In other words, the code which knows about registers and other CPU specifics
is in assembly routines. I would hope I can avoid any knowledge of the CPU
registers in the C code but I'll have to see how it pans out.

> -They both have different naming for their inport and outport C
> routines. They are in different include files. Of course, you
> could define your own, but each has different inline assembly
> syntaxes... I.e., no way to make uniform.

OK. I think this would be avoided by avoiding inline asm but do tell me if
I'm wrong.

> -They use a different method address C objects. One compiler uses
> physical addressing. The addresses correspond to the physical
> memory locations. This is true of memory below 1MB and the
> application space. The other compiler uses relative addressing.
> So, the application space starts at offset zero, no matter where
> it's loaded. This means you have to adjust any address to memory
> below 1MB by the load offset.

That sounds odd.

> -The code they generate is for DOS DPMI, i.e., 32-bit code. Each
> DPMI host set's up it's own GDT. However, when using these
> compilers for a custom OS that won't have DPMI available, the
> compiled C code is dependent on the GDT entries each compiler needs
> for it's compiled code. I.e., one compiler needs 4 GDT entries
> while the other needs 5.

OK. The code I have so far (compiled as test cases but not much of it
actually running) starts in assembly rather than C. To an extent the C code
is just subroutines even if C makes up the bulk of the source. I had to
start in assembly so that the C code did not depend on (and the linker did
not pull in) the C startup boilerplate.

> -One compiler packs structs by single bytes. The other needs a
> #pragma to do so.

This sounds important. I know C only guarantees that the first field begins
at offset zero so officially we cannot know where other fields of a struct
will sit but did you have problems only when you wanted fields to be
unaligned? I am hoping that if I put the fields of a struct at their natural
alignment that any compiler would do the expected thing.

> -I'm not using them, but I found out the format of the C jmpbuf
> structure was different for each.

OK.

> -Each compiler has their own cli and sti _routines_, not
> instruction... You need to use the compiler's routine, not roll
> your own. Each of these is in a different include file too.

I'll bear this in mind. I might simply avoid calling cli and sti from C.

> -I had to code some code to replace the initialization of various
> data items that were set as part of the executable startup which
> were no longer present once I abused the compiler to produce code
> for my OS...
> -The routines that will move and copy to memory below 1MB are
> different.
> -The compiler methods to inline routines is different for each.
> One uses an declaration with an attribute and the other uses a
> compiler keyword.

OK.

> -One compiler had a way to make "naked" C routines, i.e., no
> prolog(ue), no epilog(ue), no stack frame code, no code clean,
> i.e., "clean" assembly. The other compiler would only support
> "void func(void)" which leaves a minimal stack frame. This
> eliminated the prolog and epilog but keeps the stackframe code. It
> required use of the 'leave' instruction to remove the stack frame.
> I suspect I'm overlooking or unaware of some compiler feature...

Why did you want to avoid the stack frame? Was it just for performance or
did it interfere with some other code?

> You're likely going to need a naked function for an interrupt
> wrapper.

OK. I haven't written any of this stuff yet but I'll say what I have in
mind. Two options:

First option is to write all interrupt handlers in assembly and none in C.
If I do that I could plan to keep then very short. Each would do the minimum
to keep the device happy, send a message to a hander process for that device
and ensure that the handler was on the ready queue.

Second option is to have some assembly code which, when it receives an
interrupt, sets up a small environment and loops through calling a C handler
for each device listening on that interrupt. I'm wary about this option
because (as you mention) it's hard to be sure that the C compiler would
generate code that would work safely in an interrupt.

> -One compiler could generate both near and far returns
> appropriately for procedures while the other would only generate
> near. That prevented adjusting the "naked" function's return code
> via a far ret. Optimization also prevented adjustment of the
> procedure return instruction for that compiler. Additional
> instructions were required to manually adjust the IP.

This seems like a big point. The bane of calling conventions again!

...

>> I have made a start with gcc and bcc on Linux but don't want the
>> C code to use any pragmas or specific features of either of them.
>
> You may have a hard time without using a pragma. GCC needs one to
> pack structures correctly, or an attribute... Some of the x86
> system structures need to be aligned correctly too. That requires
> a pragma, or directive, or attribute too.

For those structures that had to pass between C and asm I think that if I
couldn't get the compilers to lay them out in a way that I could rely on
without using a pragma I might resort to having little accessor routines.
For example, instead of

x = p->field;

if "field" was a word-sized integer and "FIELD" was its offset the code
could say something like

x = *word_addr(p, FIELD);

Not nice to look at but portable. There would only need to be one accessor
routine per data type.

>> I don't want to get locked into any specific compiler.
>
> I'd say "good luck"! But, that could just be my experience or
> incorrect path.

I wouldn't call it "incorrect" but I know from experience that one small
acceptance of the compiler's awkwardness can lead to another. I spent a lot
of time trying to get OpenWatcom to interface with assembly. In the end the
C source looked nothing like C! It was really a totally different language
which was loosely based on C. It was difficult to express and hard to read
and totally non-portable. So I have an aversion to getting stuck in the same
vein again.

>> If you wanted to would it be practical to change the C source
>> so it would compile under any compiler?
>
> Any compiler, any C compiler, or any compiler or assembler?

I meant any C compiler. No need to reply now as I understand the issues
given what you have explained.

...

> Yes. But, you can't eliminate all of it, AFAIK. E.g., 32-bit code
> produced by DJGPP needs 5 descriptors, whereas OW needs 4.

Is there any reason for the compilers to require those descriptors apart
from because they are running with a DPMI startup? I mean does the code they
generate for subroutines - i.e. routines other than main() - depend on any
specific descriptors?

> The
> setup is compiler specific. Of course, I'm not using "standalone"
> or "freestanding" compiler options, or compiler flags to select
> independence from the C library, or linker scripts. I'm using a
> standard DOS DPMI executable intended to be executed under DOS.
> This would be like using a standard ELF executable on Linux or
> standard PE on Win32 for your OS. Most people don't do that. :-)
> So, those things may be a big factor that helps you eliminate some
> of the issues I have.

Again, thanks for the info.

James

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message

news:krkua7$pmc$1...@speranza.aioe.org...

...

> I'm beginning to think that I need a map as to what BIOS functions
> are available for PM and what isn't...

It may be best to avoid the BIOS as much as possible because

1. Cannot be sure a given BIOS will support many of the more modern
functions
2. Some BIOS functions are brought in on adapter cards, not inbuilt, so of
uncertain origin.
3. BIOS may have bugs. Reputed for APM (see below) and many others. I have
personally seen A20 enable BIOS code to be unreliable.

...

>> (For the same reason I have no intention of supporting APM for
>> power management - not a great loss.)
>
> Does it have a PM interface? A number of such specifications do.
> I thought I saw APM listed as one that did.

There is a good writeup here:

http://wiki.osdev.org/APM

The functions are defined but they may be unreliable or unsafe.

James

[Rod Pemberton]

"James Harris (es)" <james.h...@gmail.com> wrote in message
news:krmpop$sus$1...@dont-email.me...

> "Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message
> news:krkua7$pmc$1...@speranza.aioe.org...

> > I'm beginning to think that I need a map as to what BIOS
> > functions are available for PM and what isn't...
>
> It may be best to avoid the BIOS as much as possible because
>
> 1. Cannot be sure a given BIOS will support many of the more
> modern functions
> 2. Some BIOS functions are brought in on adapter cards, not
> inbuilt, so of uncertain origin.
> 3. BIOS may have bugs. Reputed for APM (see below) and many
> others. I have personally seen A20 enable BIOS code to be
> unreliable.
>

Well, I did that for my OS. It is simple and doesn't need the
BIOS. The hardware I stuff I programmed was all available and
documented information.

If you don't have proprietary drivers, some of the hardware, like
video, AIUI, can only be done that way. Of course, my original
goals didn't include newer hardware features, or even VGA... Some
recent work with VESA has made me consider how I would implement
higher resolution modes. So far, BIOS is the answer. What do you
do if each device is programmed differently and the specifications
are secret? So, VESA and SATA, I think, will become issues for me,
if I ever resume work on it.


Rod Pemberton

[Rod Pemberton]

<s_dub...@yahoo.com> wrote in message
news:7330b3b0-b379-43f5...@googlegroups.com...

> Is SeaBIOS mature, is it offered by any OEM for their
> motherboard, out of the box? Is it open source, (source
> code available)?

I just found SeaBIOS a day or two ago. It's apparently part of
Coreboot, formerly LinuxBIOS, formerly FreeBIOS. I've seen
coreboot a few times over the years. It had a number of
interesting sub-projects such as FILO and ADLO. However, I don't
recall the name SeaBIOS. According to a past post of mine,
Coreboot used a modified Bochs BIOS image.

I found SeaBIOS because I was trying to (re)find a BIOS project I
thought was called "svgabios". But, the code I was actually
looking for was is part of the Bochs emulator. It's a project
called the Plex/Bochs VGABios.

In a different post, I mentioned that Scitech's x86emu was part of
Coreboot. It may still be.

In the other past post, I stated the following, which may give an
idea of how Coreboot+SeaBIOS works:

"It uses motherboard specific initialization routines for the
motherboard hardware (hardware dependent), followed by running a
modified BOCH's BIOS (hardware independent), and uses specific
motherboard's video BIOS (hardware dependent)."

I have another post that goes into more detail on how LinuxBIOS
loaded either FILO or ADLO in 2007. I'm not sure if it still works
the same way today. Even if not, it's worth a read for the
techniques they used.
https://groups.google.com/d/msg/alt.os.development/OxIRneBBVqw/0Szve5c0aqIJ

SeaBIOS is open source, LGPLv3.
http://www.seabios.org/SeaBIOS

Coreboot on SeaBIOS
http://www.coreboot.org/SeaBIOS

"payloads" Coreboot will load
http://www.coreboot.org/Payloads

Plex/Bochs VGABios is open source, LGPLv2.
http://www.nongnu.org/vgabios/

Bochs
http://bochs.sourceforge.net/

OpenBIOS
http://www.openfirmware.info/Welcome_to_OpenBIOS

FILO - PM bootloader, BIOS independent
http://www.coreboot.org/FILO

ADLO - PM loader of ROM images
http://www.coreboot.org/ADLO

Libpayload - basically, a minimal OS kernel C library...
http://www.coreboot.org/Libpayload

HTH,


Rod Pemberton

[Rod Pemberton]

"James Harris (es)" <james.h...@gmail.com> wrote in message
news:krmh6a$c20$1...@dont-email.me...

> "Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message
> news:krl67p$a48$1...@speranza.aioe.org...

> > [#if-defs]

>
> BTW, when you say #if-defs do you mean one of these:
>
> Opt 1: #if (cond) code #elif (cond) code #else code #endif
> Opt 2: #if (cond) #define ... #endif
> Opt 3: #ifndef #include ... #endif

Yes, all of those and other combinationss. I just meant to use
simple pre-processor #if sections to include or exclude code. I
was being terse. I tend to use only three or four basic #if-defs.

I can add or remove code quickly/simply:

#if N /* where N is 0 for disable or 1 for enabled as needed */
/* code here */
#endif

I do use an #else too on occasion. I tend to *not* nest these. I
can delete them easily.

Or, I can control a bunch of conditional sections like so:

/* at top of file set N to include DEBUG code */
#if N
#define DEBUG
#endif
...
/* later in file */
#ifdef DEBUG
/* printf ... */
#endif
...
/* even later in file */
#ifdef DEBUG
/* printf ... */
#endif
...
/* ditto */

Typically, compilers have a define that is only available on that
compiler to allow compiler specific code. They usually have a
define for the processor mode too. So, for the two I use, those
two defines are __WATCOMC__ and __DJGPP__. So, if I
need code which is different for each compiler and mode, I
just do, as an example:

void some_proc(int arg0, int arg1, int arg2)
{
/* common code */
#ifdef __WATCOMC__
/* OpenWatcom code here */
#ifdef __386__
/* OpenWatcom 32-bit code */
#else
/* OpenWatcom 16-bit code */
#endif
/* OpenWatcom code here */
#endif
#ifdef __DJGPP__
/* DJGPP code here - only 32-bit */
#endif
/* common code */
}

Obviously, for my OS there is no 16-bit C code for OW.

> > How many RM interrupts does a PM OS need to
> > "fix" ... ?
>

> I'm not sure I understand the question [...]

I was "foreshadowing" my comment on all three OS modes being PM for
security, no RM, no v86. I.e., a PM OS, be it 16-bit or 32-bit or
64-bit, doesn't need RM interrupts (unless you're using v86 or need
VESA...).

> [...] but from memory I think a 32-bit PM OS would only

> deal with 32-bit PM interrupts. Even if it is executing RM
> code in a virtual 8086 monitor or executing a PM16 code
> segment the interrupts would still, I think, fire as full
> 32-bit ones.
>

If you're executing a PM16 code segment for a mostly 32-bit PM OS,
how does the processor know the mode is 32-bits before triggering
an interrupt? If the interrupt tables are different sizes, the
processor would need to know. CR0.PE is set for both 16-bit PM and
32-bit PM. The CS descriptor sets the code size. PM is not
entered until a jump to the code segment. AIR, LGDT and LIDT do
nothing special in regards to that. I'm not sure about the format
of 16-bit PM IDT and GDT structures. But, this train-of-thought
makes me think the IDT and GDT are dual-mode and 16-bit or 32-bit
mode are sub-modes of PM. I.e., I don't recall what, if anything,
would tell the processor that the GDT and IDT are both 16-bit or
both 32-bit. That would be required if they're differently sized.
I'm not saying there isn't something. If there is, I don't know
off-hand what it is. I'd definately have to pull out the manuals
on this to find out.

> AIUI in long mode the addresses are 64 bits wide.

...

> Ah, no. Each OS image would be made from, effecively, three
> components: portable C source, mode-specific C source and
> mode-specific assembly source.

Ok.

> All the non-portable stuff could be written in asm. I was
> thinking to see how effectively it could be done in C, though.

Well, I did it in C, unless I couldn't. (C fanatic) Inlined
assembly in C was my first choice for assembly. I only used an
external assembler (NASM) when required, or when it was too much
work to do it for both compilers, or just when I decided too...

> ISTM that inline asm is to be avoided like the plague. I fear it
> may tie the code to specific compilers.

You need assembly for the special processor instructions. The
question becomes where, when, or how you're going to implement
them.

If you make all the assembly code separate from C, you need at
least one assembler. But, each compiler has it's own assembler
with it's own syntax. GAS or WASM use the same syntax for external
assembly as inlined C code for GCC or OpenWatcom, respectively.
So, moving the assembly out of C doesn't help. As mentioned
previously, GAS can use Intel syntax. That can be used to
eliminate AT&T syntax some of the time. However, other compilers
use MASM syntax assembly. So, to not write the assembly code in
multiple formats, you're going to tie the assembly code to a
specific "universal" assembler, e.g., NASM, or need to be able to
select a single syntax for all of them, e.g., Intel. I don't think
you can select a single assembly syntax across more than a few x86
assemblers. (Oh, I probably shouldn't say that so concretely,
Herbert Kleebauer might post a solution to that in reply.)

Another option is something radical, low-level, or rudimentary,
e.g., a small ANSI C program to un-dump (convert back to binary
from hex) a pre-compiled library or module with the special
routines and processor instructions that are coded in assembly.
Then, the only environment that would need the assembly code is
yours, to create the binary. All other users would just compile
the utility and execute it. I.e., it'd be a bit like a distributed
.obj file without source, except it'd be in hex text.

I'm working on the list reply...


Rod Pemberton

[Rod Pemberton]

"James Harris (es)" <james.h...@gmail.com> wrote in message
news:krmmlm$b54$1...@dont-email.me...

> "Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message
> news:krl67p$a48$1...@speranza.aioe.org...

> > -They address registers in register structures differently.
> > You can fix this with conditional code or defines.
>
> Do you mean things like
>
> struct regs {
> int ax;
> int bx;
> };
>

E.g., for DJGPP ax is r.x.ax and OpenWatcom is r.w.ax within each
compiler's provided register unions. How do you handle the 'x'
and 'w' being different? #defines work. #if-def's work. Compiler
independence? Yeah, I didn't figure it out...

Um... So, this might not apply to you either. These structs are
really needed to call DPMI functions. Well, you might need them or
your own for task switching... Of course, you wouldn't have this
problem with your own code.

Anyway, both compilers have a number of "structs", unions actually,
which are comprised of all or most of the registers per mode.
Think of creating a struct for all the registers for the PUSHA
instruction. So, for a 16-bit register struct, it'll have all or
most of the 16-bit registers as struct items. However, these
structs are usually setup as unions of multiple register structs.
So, the union will have 16-bit and 32-bit register structs.

> > -They both have different naming for their inport and outport C
> > routines. They are in different include files. Of course, you
> > could define your own, but each has different inline assembly
> > syntaxes... I.e., no way to make uniform.
>
> OK. I think this would be avoided by avoiding inline asm but do
> tell me if I'm wrong.

OpenWatcom names them inp and outp.

DJGPP names them inportb and outportb.

I.e., each C compiler has a different name for the port functions.
#defines fix this easily, but obviously it's not compiler
independent code.

Of course, the inline assembly syntax for the instructions is
different for each too.

Finally, they're in different include files: <pc.h> DJGPP and
<conio.h> OW, which you can only select between via #if-defs...

I.e., I don't know how to make these remotely identical across
multiple compilers without something compiler specific.

Yes, you could put all stuff like this to one configuration .h
file. Then, there is just one non-portable .h file.

> OK. The code I have so far (compiled as test cases but not much
> of it actually running) starts in assembly rather than C. To an
> extent the C code is just subroutines even if C makes up the bulk
> of the source. I had to start in assembly so that the C code did
> not depend on (and the linker did not pull in) the C startup
> boilerplate.

Yes.

But, each compiler generates code for specific selectors. E.g.,
the DJGPP compiler uses a different selector for C library code
versus the code in the program.

So, you may need those freestanding and no standard library options
to make the code more independent of issues like this or even
relocatable. I didn't use any of those options, so I may be
fighting an issue I didn't need to fight. I'm not sure whether
each compiler is similar or different in regards to those options.

> > -One compiler packs structs by single bytes. The other needs a
> > #pragma to do so.
>
> This sounds important. I know C only guarantees that the first
> field begins at offset zero so officially we cannot know where
> other fields of a struct will sit but did you have problems only
> when you wanted fields to be unaligned? I am hoping that if I
> put the fields of a struct at their natural alignment that any
> compiler would do the expected thing.
>

The issue is with structures used directly by the processor and
accessed via C. I.e., if the structure is only being used within
your OS or application, the alignment and packing doesn't matter.
The compiler sets the alignment and packing however it wants or you
select. However, the processor requires certain alignments and
packing. E.g., if the compiler inserts alignment padding into the
middle of the GDT, you've got a problem. The C code will access
the "correct" item taking into account the padding, and the
processor will attempt to access the "correct" item without being
aware of any padding.

> > -One compiler had a way to make "naked" C routines, i.e., no
> > prolog(ue), no epilog(ue), no stack frame code, no code clean,
> > i.e., "clean" assembly. The other compiler would only support
> > "void func(void)" which leaves a minimal stack frame. This
> > eliminated the prolog and epilog but keeps the stackframe code.
> > It required use of the 'leave' instruction to remove the stack
> > frame. I suspect I'm overlooking or unaware of some compiler
> > feature...
>
> Why did you want to avoid the stack frame? Was it just for
> performance or did it interfere with some other code?
>

Darn! I just completed this *entire* reply when I realized I said
"stack frame" somewhere when I meant the procedure code for passing
and cleaning up parameter's. OK, I see what I did. "stack frame
code" and "stackframe code" above are correct, since such code
implements the passing and cleanup of parameters. The "minimal
stack frame" above should've had the word "code" added to the end
of it. I.e., the voided function minimizes the prolog or epilog,
but doesn't entirely eliminate them. "Naked" is without any of the
code that normally wraps the procedure body, i.e., completely
eliminated.

Early DOS compilers had the "interrupt" keyword to support
interrupts. OpenWatcom has it too, but I didn't use it. However,
DJGPP (GCC) doesn't have support for interrupts ... AFAIK. If you
find it or I missed the obvious, let me know. So, you need a
mechanism to cleanly transfer execution from executing code - which
is incompatible with your C code - to your C code. I.e., you have
to setup a stack frame and parameters etc and call, not jump, to
the procedure.

In more depth, 98% of the work of all my interrupt routines are
handled by a central routine coded in C. Basically, all interrupts
come into this routine which then breaks out to code specific to
each interrupt. So, each interrupt vector must transfer to this
single C procedure. However, you can't jump directly to a C
procedure from assembly. It's wrapped in a prolog(ue), epilog(ue),
stack setup, and stack cleanup code. You can't jump into a
procedure either, because the epilog(ue) will execute. To "jump"
to the C code, you need to setup a stack frame and associated
registers and then call the procedure. So, I needed a way to
transfer from all interrupt vectors to my central C code. If you
have a "naked" C procedure, when compiled (or assembled), it's just
like a procedure-less piece of assembly code. This means you can
jump directly to it from other assembly code, like from an
interrupt vector. So, all my interrupt vectors jump to the same
single "naked" C routine. The good thing about inline assembly
inside a C procedure is that the compiler inserts the inline
assembly into the C code for the procedure as if it was creating by
compiling C code. I.e., from the perspective of a "naked" C
procedure, you can't tell the difference between inline assembly
and compiled code. Next, that routine then sets up a stack frame,
registers, etc and transfers execution to the central C interrupt
routine, which is just a normal C procedure. I refer to this
mechanism as an interrupt "wrapper". I'm not sure if that's the
correct term. It might be something else like "trampoline" or who
knows what...(?)

> First option is to write all interrupt handlers in assembly and
> none in C.

Good choice, if you can keep them really short and simple.

(Wolfgang might be able to help with that.)

> Second option is to have some assembly code which, when it
> receives an interrupt, sets up a small environment and loops
> through calling a C handler for each device listening on that
> interrupt. I'm wary about this option because (as you mention)
> it's hard to be sure that the C compiler would generate code
> that would work safely in an interrupt.

...

> For those structures that had to pass between C and asm I think
> that if I couldn't get the compilers to lay them out in a way
> that I could rely on without using a pragma I might resort to
> having little accessor routines.
>
> For example, instead of
>
> x = p->field;
>
> if "field" was a word-sized integer and "FIELD" was its offset
> the code could say something like
>
> x = *word_addr(p, FIELD);
>
> Not nice to look at but portable. There would only need to be one
> accessor routine per data type.
>

It's not the C side that's the problem. It's the processor side.
You can adjust the C side to fit the processor's alignment and
packing. You can't adjust the processor to use the C compiler's
alignment and packing.

This is similar to the struct being a "volatile". With a
"volatile," something other than your code is modifying the data
where it's stored (or read). You have something outside your code
space that is doing something or has different expectations about
how something is done. The compiler only controls the code space.

> > Yes. But, you can't eliminate all of it, AFAIK. E.g., 32-bit
> > code produced by DJGPP needs 5 descriptors, whereas OW
> > needs 4.
>
> Is there any reason for the compilers to require those
> descriptors apart from because they are running with
> a DPMI startup?

A 32-bit compiler generates code for PM. PM uses selectors and
descriptors. The compiler will generate code that uses some
selectors. At a minimum, a compiler must use a code selector.
Typically, it also uses a data selector too, because that requires
fewer segment overrides than using just a code selector. It's
likely the stack segment is set too. Other than that, it's
entirely up to the compiler's code optimizer, or the compiler's
authors, and really, those are too, except CS.

> I mean does the code they generate for subroutines - i.e.
> routines other than main() - depend on any specific
> descriptors?

It's PM. It should. Most likely CS and DS, at a minimum. What I
did was compile some code to dump all the selector values as hex.
I.e., print hex values for CS, DS, ES, FS, GS, SS. Some of these
are likely to be the same value, i.e., same selector. The total
number of different values is likely the number of selectors you
need to setup for your code. CS must be different from the other
selectors. But, ES, FS, GS, may be the same as DS or not. SS may
be the same as DS or not. Of course, that worked for OpenWatcom.
It used a simpler model. DJGPP (GCC) has a number of other
selectors which are not permanently loaded into the
segment/selector registers. These are used intermittently, i.e.,
switched to when needed. I.e., I had to track those down in the
compiler source code and figure out how to initialize them without
the CRT startup or "boilerplate" present (as you called it).

Oh, yeah, OpenWatcom allows you to select a register passing
parameter convention instead of the stack, DJGPP (GCC) is stack
only. This only matters if you get into modifying the "nitty
gritty", i.e., each is different, but stack for both is more
similar.

I mentioned the freestanding option previously for GCC. There is
also a fomit-frame-pointer option that affects the generated stack
frame code too.


Rod Pemberton

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message

news:krnoi0$t0g$1...@speranza.aioe.org...

...

From some comments you made above I wondered if you were thinking that there
was a version of protected mode where 16-bit descriptor tables were used. I
might be wrong but I suspect that once in CPU is in Pmode it will always use
the current (32-bit) IDT - even if a 16-bit code segment is executing when
an interrupt occurs. Ditto for GDT and the current LDT when segment
registers are loaded.

...

>> ISTM that inline asm is to be avoided like the plague. I fear it
>> may tie the code to specific compilers.
>
> You need assembly for the special processor instructions. The
> question becomes where, when, or how you're going to implement
> them.
>
> If you make all the assembly code separate from C, you need at
> least one assembler.

Agreed.

> But, each compiler has it's own assembler
> with it's own syntax. GAS or WASM use the same syntax for external
> assembly as inlined C code for GCC or OpenWatcom, respectively.
> So, moving the assembly out of C doesn't help.

I don't understand the part of this where a compiler's associated assembler
becomes important. Here's what I had in mind:

c file ---> (GCC) ---> 32-bit object code file
asm file ---> (NASM) ---> 32-bit object code file

then both those 32-bit object code files get linked together. The only
assembler used is Nasm. No need for a compiler-specific assembler.

> previously, GAS can use Intel syntax. That can be used to
> eliminate AT&T syntax some of the time. However, other compilers
> use MASM syntax assembly. So, to not write the assembly code in
> multiple formats, you're going to tie the assembly code to a
> specific "universal" assembler, e.g., NASM, or need to be able to
> select a single syntax for all of them, e.g., Intel. I don't think
> you can select a single assembly syntax across more than a few x86
> assemblers. (Oh, I probably shouldn't say that so concretely,
> Herbert Kleebauer might post a solution to that in reply.)

Agreed. Practically speaking it is necessary to stick to one assembler.
Assembly code which would adapt to many different assemblers would probably
look like it had been obfuscated!

As above I cannot see a need to do anything but ignore the assemblers
associated with the compilers. Am I missing something?

James

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message

news:kro18q$dis$1...@speranza.aioe.org...

...

>> > -They both have different naming for their inport and outport C
>> > routines. They are in different include files. Of course, you
>> > could define your own, but each has different inline assembly
>> > syntaxes... I.e., no way to make uniform.
>>
>> OK. I think this would be avoided by avoiding inline asm but do
>> tell me if I'm wrong.
>
> OpenWatcom names them inp and outp.
>
> DJGPP names them inportb and outportb.
>
> I.e., each C compiler has a different name for the port functions.
> #defines fix this easily, but obviously it's not compiler
> independent code.
>
> Of course, the inline assembly syntax for the instructions is
> different for each too.
>
> Finally, they're in different include files: <pc.h> DJGPP and
> <conio.h> OW, which you can only select between via #if-defs...
>
> I.e., I don't know how to make these remotely identical across
> multiple compilers without something compiler specific.
>
> Yes, you could put all stuff like this to one configuration .h
> file. Then, there is just one non-portable .h file.

Rather than changing the C code, or telling the compiler anything specific,
how about writing, in C,

x = port_8_in(port1);

The port_8_in routine would be implemented in 32-bit assembly something like

port_8_in:
push edx
mov edx, [esp + 8 + 0] ;Fetch the port number
xor eax, eax
in al, dx
pop edx
ret

(As an aside the 16-bit assembly code would be similar but so that it can
run in real mode it would set up a stack frame and address the parameter off
bp. Real mode doesn't allow addressing off sp.)

...

Thanks for all the points about segment registers and other things
(snipped).

I may have been OK so far because:

1. I don't use compiler-supplied startup or runtime code
2. I don't use the standard library

It's also possible that the compilers I've used have been well-behaved or
because I have asked C to do only simple stuff. It sounds like something
I'll have to keep in mind.

James

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message

news:krnobq$sj6$1...@speranza.aioe.org...

Sure. I mentioned avoiding the BIOS as much as possible, not avoiding it
altogether. In particular, while there may be some tempting features in
newer BIOSes use of them could decrease portability and, possibly,
reliability as well.

James

[Rod Pemberton]

"James Harris (es)" <james.h...@gmail.com> wrote in message

news:krpblu$hbr$1...@dont-email.me...

> "Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message
> news:krnoi0$t0g$1...@speranza.aioe.org...
>

> > [...]

> As above I cannot see a need to do anything but ignore
> the assemblers associated with the compilers. Am I
> missing something?
>

As long as it all links together correctly, that works. If you're
going to extremes to avoid *any* type of compiler specific
features, then that code will likely need to be NASM too. E.g.,
let's say the GDT is an example of that. The GDT is not hard to
code in C, nor is code to access it. So, you might give up some of
C's strengths to avoid compiler specific features.


RP

[Mike Gonta]

"James Harris (es)" wrote:

> Rather than changing the C code, or telling the compiler anything
> specific, how about writing, in C,
>
> x = port_8_in(port1);
>
> The port_8_in routine would be implemented in 32-bit assembly something
> like
>
> port_8_in:
> push edx
> mov edx, [esp + 8 + 0] ;Fetch the port number
> xor eax, eax
> in al, dx
> pop edx
> ret
>
> (As an aside the 16-bit assembly code would be similar but
> so that it can run in real mode it would set up a stack frame
> and address the parameter off bp. Real mode doesn't allow
> addressing off sp.)

Only the _long gone and forgotten_ microprocessors before the
80386 don't allow addressing off sp. Everthing since allows
32 bit registers (still limited to 16 bit) in real mode to
access these _advanced features_. Even the latest x86 offering
can run in real mode, so why cripple the code for the sake of
8086 (or 186, 286)?


Mike Gonta
look and see - many look but few see

http://mikegonta.com

[Mike Gonta]

"James Harris" wrote:
> Have any of you already worked out the following? Or do
> you want a way to run 16-bit code under a 64-bit OS?

> I heard long ago that AMD did not support use of 16-bit
> code when in Long mode but I see that Long mode does in
> fact support some form of 16-bit code. Is that protected
> mode where the code and data segments have default
> address size and operand size set to 16 bits?

> Is 16-bit Protected mode the right term?

Yes.

> Whatever it's called it seems an unusual mode to run in.

It was the _new_ Protected Virtual Address Mode (also
referred to as Protected Mode) of the 80208.
http://ragestorm.net/downloads/286intel.txt
It wasn't much used because of the 16 bit size restrictions
and no easy way to return to real mode.
http://en.wikipedia.org/wiki/Intel_80286
The 386 solved these problems and thus we have 32 bit protected
mode.

> Would it be feasible to run old 16-bit apps in this mode?

No. Some historical attempts were made, but not feasible today.

> If not, what would cause them to fail?

Hard-coding F000h as a data segment/selector.

> Could the BIOS be run in that mode, perhaps with the OS emulating
> some operations?

No. A quick peek at a current (Decemeber 2011) EUFI CSM BIOS reveals
hard-coding 0xF000.

> I see that V8086 mode and Real mode won't work under Long mode but
> wonder how much can be done in the above 16-bit mode. Is it useful
> or not?

Not useful.
Compac patented (US6105101) the idea of using the 16 bit PM mode to
run ROM BIOS code from 32 bit PM. This required re-writing the BIOS
to eliminate the above restiction.

[Rod Pemberton]

"Mike Gonta" <mike...@gmail.com> wrote in message
news:krrgrd$555$1...@speranza.aioe.org...

> "James Harris" wrote:

"look and see - many look but few see"

Yup, "not seeing" applies to you twice on this one Mike... You
need to stop the drive-by posting! Talk a while. ;-)

> > Have any of you already worked out the following? Or do
> > you want a way to run 16-bit code under a 64-bit OS?
>
> > I heard long ago that AMD did not support use of 16-bit
> > code when in Long mode but I see that Long mode does in
> > fact support some form of 16-bit code. Is that protected
> > mode where the code and data segments have default
> > address size and operand size set to 16 bits?
>
> > Is 16-bit Protected mode the right term?
>
> Yes.
>

That's PM16, not CM16. We've gone over this elsewhere in the
thread already. Look here:
http://www.sandpile.org/x86/mode.htm

> Compa[q] patented (US6105101) the idea [...]

You can't patent or copyright *ideas*. You can only protect
specific creations, i.e., implementations or expressions of an
idea. This means there can be multiple patents on "using the 16
bit PM mode to run ROM BIOS code from 32 bit PM" by different
people or companies because each process, software, and/or hardware
implementation would be different. Anyway, Mike, I'm not a lawyer,
but I do take issue with your _repeated_ statements that someone
can patent an idea. They simply can't.


RP

[Mike Gonta]

"Rod Pemberton" wrote:
> "Mike Gonta" wrote:

>> "James Harris" wrote:

>> > Have any of you already worked out the following? Or do
>> > you want a way to run 16-bit code under a 64-bit OS?
>>
>> > I heard long ago that AMD did not support use of 16-bit
>> > code when in Long mode but I see that Long mode does in
>> > fact support some form of 16-bit code. Is that protected
>> > mode where the code and data segments have default
>> > address size and operand size set to 16 bits?
>>
>> > Is 16-bit Protected mode the right term?
>>
>> Yes.
>>
>
> That's PM16, not CM16. We've gone over this elsewhere in the
> thread already. Look here:
> http://www.sandpile.org/x86/mode.htm

CM16 is exactly the same as PM16. The only difference is semantics.
With reference to the chart in the above link:

The native mode of x86-64 is LM - long mode. In long mode there is
no real mode. By setting flags the mode can be changed to
CM - compatibility mode. (There is also legacy mode which is the
equivalent of a 32 bit x86). In compatibility mode there are two
sizes, 32 and 16, but since there is no real mode in long mode both
of these modes (CM16 and CM32) are equivalent to protected mode. The
only difference (one of semantics) is that in legacy mode it is
referred to as protected mode and in long mode as compatibility mode.
It can be seen in this chart that PM16 and CM16 each have the same flags
set (just as PM32 and CM32 also do). The only difference is EFER.LMA
which is the difference between long mode and legacy mode.
There is however, one small insignificant difference - paging. In legacy
mode (PM16) paging is optional - just as it is in x86-32, while in
compatibility mode (CM16) paging is required (this is a sub-mode of long
mode, like it or not - paging is required). The reason that this is
insignificant is that paging can be enabled in PM16 or all memory can be
identity mapped in CM16 (which is the equivalent of no paging) and thus
both PM16 and CM16 will be _exactly_ the same. Paging by itself is not
a differentiating factor.


Mike Gonta

look and see - many look but few see

http://mikegonta.com

[Alexei A. Frounze]

On Sunday, July 14, 2013 3:30:27 AM UTC+4, Rod Pemberton wrote:
> "Mike Gonta" <> wrote in message

Um, don't they just throw in a spell, "aparatus/machine", to work-around the unpatentability of ideas/algorithms/etc?

Alex

[James Harris (es)]

"Mike Gonta" <mike...@gmail.com> wrote in message

news:krre6u$tg6$1...@speranza.aioe.org...

...

>> (As an aside the 16-bit assembly code would be similar but
>> so that it can run in real mode it would set up a stack frame
>> and address the parameter off bp. Real mode doesn't allow
>> addressing off sp.)
>
> Only the _long gone and forgotten_ microprocessors before the
> 80386 don't allow addressing off sp. Everthing since allows
> 32 bit registers (still limited to 16 bit) in real mode to
> access these _advanced features_.

Take an example,

mov ax, [sp + 2]

Are you saying that in real mode the 80386 and later can execute that
instruction and similar?

> Even the latest x86 offering
> can run in real mode, so why cripple the code for the sake of
> 8086 (or 186, 286)?

Code which accesses off bp is just as effective in real mode as code which
accesses off esp in protected mode so no effect is lost by having the stack
frame.

If the question is, "why consider pre-80386," ISTM the more hardware an OS
can run on the better. I would like a 16-bit version of the OS to run on an
8086 machine and 640k of RAM.

I did for a while consider a form that would run in just 64k of RAM but
found that led to too many constraints on data structures that would make
them inefficient on bigger machines. I may revisit this. In fact I cannot
see a problem with running a limited version of the OS on a Z80 or 6502!

James

[Alexei A. Frounze]

On Monday, July 15, 2013 3:45:35 PM UTC+4, James Harris wrote:
> "Mike Gonta" wrote in message
>
>
>

> ...
>
>
>
> >> (As an aside the 16-bit assembly code would be similar but
>
> >> so that it can run in real mode it would set up a stack frame
>
> >> and address the parameter off bp. Real mode doesn't allow
>
> >> addressing off sp.)
>
> >
>
> > Only the _long gone and forgotten_ microprocessors before the
>
> > 80386 don't allow addressing off sp. Everthing since allows
>
> > 32 bit registers (still limited to 16 bit) in real mode to
>
> > access these _advanced features_.
>
>
>
> Take an example,
>
>
>
> mov ax, [sp + 2]
>
>
>
> Are you saying that in real mode the 80386 and later can execute that
>
> instruction and similar?
>
>
>

mov ax, [esp + 2]

is valid and executable as long as 2 bytes at esp + 2 are within the segment limits.

> > Even the latest x86 offering
>
> > can run in real mode, so why cripple the code for the sake of
>
> > 8086 (or 186, 286)?
>
>
>
> Code which accesses off bp is just as effective in real mode as code which
>
> accesses off esp in protected mode so no effect is lost by having the stack
>
> frame.
>
>
>
> If the question is, "why consider pre-80386," ISTM the more hardware an OS
>
> can run on the better. I would like a 16-bit version of the OS to run on an
>
> 8086 machine and 640k of RAM.
>
>
>
> I did for a while consider a form that would run in just 64k of RAM but
>
> found that led to too many constraints on data structures that would make
>
> them inefficient on bigger machines. I may revisit this. In fact I cannot
>
> see a problem with running a limited version of the OS on a Z80 or 6502!
>
>
>
> James

Alex

[Rod Pemberton]

"James Harris (es)" <james.h...@gmail.com> wrote in message

news:ks0n1r$g5i$1...@dont-email.me...

> "Mike Gonta" <mike...@gmail.com> wrote in message
> news:krre6u$tg6$1...@speranza.aioe.org...
...

> > Even the latest x86 offering can run in real mode,
> > so why cripple the code for the sake of 8086 (or 186, 286)?
>

> [...]

>
> If the question is, "why consider pre-80386," ISTM the more
> hardware an OS can run on the better. I would like a 16-bit
> version of the OS to run on an 8086 machine and 640k of RAM.
>

Why? Do you actually have working hardware that old? Who does?

It's 2013 not 1992. Electronics die. So, you understand pre-486
processors are collector's items now, yes? I.e., you could pay a
fortune for a _non-working_ pre-486, x86 class processor.

> I did for a while consider a form that would run in just 64k of
> RAM but found that led to too many constraints on data structures
> that would make them inefficient on bigger machines. I may
> revisit this. In fact I cannot see a problem with running a
> limited version of the OS on a Z80 or 6502!
>

I'm fond of the 6502 because I learned on a variant of it. The
6502 had a large set of support chips which made it popular with
some electronic hobbyists too, especially those that needed
terminal video. I'm also fond of the Z80 because the electronic
hobbyist magazines used it. The Z80 could refresh DRAM without
support chips. So, electronic projects using the Z80 were simpler,
smaller.

So, the electronic hobbyists liked the Z80 and 6502 in the 1980's.
Today, the "maker" community embraces the Arduino and Raspberry Pi.


Rod Pemberton

[Ivan Shmakov]

>>>>> Rod Pemberton <do_no...@notemailnotq.cpm> writes:
>>>>> "James Harris (es)" wrote in message news:ks0n1r$g5i$1...@dont-email.me...

[...]

>> If the question is, "why consider pre-80386," ISTM the more hardware
>> an OS can run on the better. I would like a 16-bit version of the
>> OS to run on an 8086 machine and 640k of RAM.

> Why? Do you actually have working hardware that old? Who does?

> It's 2013 not 1992. Electronics die. So, you understand pre-486

> processors are collector's items now, yes? I .e., you could pay a

> fortune for a _non-working_ pre-486, x86 class processor.

Curiously enough, I seem to have a Am386-based computer.
(I guess I have to power it on sometime to check if it still
works. But the last time I did, -- it worked.)

And there's a Russia-made ZX Spectrum clone on the shelves, BTW.

[...]

> So, the electronic hobbyists liked the Z80 and 6502 in the 1980's.
> Today, the "maker" community embraces the Arduino and Raspberry Pi.

Yes.

Sadly, Raspberry Pi is not open hardware, and neither is it
particularly friendly to free software (as in: Debian.) And
while Arduino Due is 32-bit, it's still too low on cycles for
its cost.

(Any volunteers to design a decent board "for us, the makers"?)

--
FSF associate member #7257

[wolfgang kern]

Rod Pemberton wrote:

> ...
>> > Even the latest x86 offering can run in real mode,
>> > so why cripple the code for the sake of 8086 (or 186, 286)?
>> [...]
>> If the question is, "why consider pre-80386," ISTM the more
>> hardware an OS can run on the better. I would like a 16-bit
>> version of the OS to run on an 8086 machine and 640k of RAM.

> Why? Do you actually have working hardware that old? Who does?

You're right, backwards compatible issues got a new border every
few years...

> It's 2013 not 1992. Electronics die. So, you understand pre-486
> processors are collector's items now, yes? I.e., you could pay a
> fortune for a _non-working_ pre-486, x86 class processor.

I'd assume a processor which know about SIMD(up to VEX,XOP,) today
as 'standard of recent availabe technics', while I still use and
support only this already olde SSE2 instructions in my OS.

>> I did for a while consider a form that would run in just 64k of
>> RAM but found that led to too many constraints on data structures
>> that would make them inefficient on bigger machines. I may
>> revisit this. In fact I cannot see a problem with running a
>> limited version of the OS on a Z80 or 6502!

> I'm fond of the 6502 because I learned on a variant of it. The
> 6502 had a large set of support chips which made it popular with
> some electronic hobbyists too, especially those that needed
> terminal video. I'm also fond of the Z80 because the electronic
> hobbyist magazines used it. The Z80 could refresh DRAM without
> support chips. So, electronic projects using the Z80 were simpler,
> smaller.

Yeah, C64 and Sir Clive Sinclair's ZX81 were the most sold during
this days then. And I learned a lot on how to (ZX81) or better how
to avoid (like many C64 apps showed up as) high-level-programming.

> So, the electronic hobbyists liked the Z80 and 6502 in the 1980's.
> Today, the "maker" community embraces the Arduino and Raspberry Pi.

Dunno this two but Yes, all hardware oriented folks prefered Z80
coding style above Intel's shit [starting with MOV instead LD/ST]
and the absolutely wrong (inpracticable) theory from CS-schools.

Today I'd like to make my very own CPU: (if I had 4M Euro aside I'd do)
no REX no VEX but nibble based register addressing (for 16 regs)
available in all modes and I really could announce paging in 64-bit
mode.

__
wolfgang

[Alexei A. Frounze]

On Tuesday, July 16, 2013 9:30:31 PM UTC+4, wolfgang kern wrote:
...

> Dunno this two but Yes, all hardware oriented folks prefered Z80
>
> coding style above Intel's shit [starting with MOV instead LD/ST]
>
> and the absolutely wrong (inpracticable) theory from CS-schools.
>

I hate to have to operate with multitudes of slightly differing LD and ST mnemonics instead of the plain and simple MOV. It's still a move/copy/load/store/etc, so why have many names for one concept? And why have "AND reg1, reg2" and at the same time "ANI reg, const", why not just use "AND" in both cases? And so on and so forth.

Alex

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message

news:ks25cv$kt4$1...@speranza.aioe.org...

> "James Harris (es)" <james.h...@gmail.com> wrote in message
> news:ks0n1r$g5i$1...@dont-email.me...
>> "Mike Gonta" <mike...@gmail.com> wrote in message
>> news:krre6u$tg6$1...@speranza.aioe.org...
> ...
>
>> > Even the latest x86 offering can run in real mode,
>> > so why cripple the code for the sake of 8086 (or 186, 286)?
>>
>> [...]
>>
>> If the question is, "why consider pre-80386," ISTM the more
>> hardware an OS can run on the better. I would like a 16-bit
>> version of the OS to run on an 8086 machine and 640k of RAM.
>>
>
> Why? Do you actually have working hardware that old? Who does?

In answer to the first question, yes, I have some specific reasons No, I
don't have hardware that old (but there are emulators).

As I've commented before (and I know YMMV) IMO an OS is principally defined
by the environment it presents to apps. Having some different targets for
the OS to run on can be a big help in keeping the design of that environment
isolated from the hardware. Real mode is not a vast difference but it is
convenient and easy to access.

There's also a potential performance benefit of considering the 8086 as a
target. Modern machines are really ridiculously fast but you'd never know
that judging by the OSes that run on them. I suspect that's down to very
poor software. The machines we use now are orders of magnitude faster than
the early PCs yet the OSes can appear to users to be slow. I like the
challenge of writing an OS that would be seen as fast even on an old
computer such as an 8086 or an 80386. Having the old stuff as one target is
more of a challenge but I think it helps keep my feet on the ground in terms
both of CPU and memory usage.

> It's 2013 not 1992. Electronics die. So, you understand pre-486
> processors are collector's items now, yes? I.e., you could pay a
> fortune for a _non-working_ pre-486, x86 class processor.

I see that they are very expensive. Bizarre, isn't it!

>> I did for a while consider a form that would run in just 64k of
>> RAM but found that led to too many constraints on data structures
>> that would make them inefficient on bigger machines. I may
>> revisit this. In fact I cannot see a problem with running a
>> limited version of the OS on a Z80 or 6502!
>>

Here's an example to illustrate the result of deciding to support old CPUs.
Feel free to criticise it.

I currently have functions like

unsigned int uint_tsc_or(unsigned int);

It's job is to get the low bits of the time stamp counter. The "_or" part is
because some CPUs that the OS will run on will not have a TSC or anything
else suitable (high resolution and, more importantly, extremely low
latency). On hardware without a TSC the routine will return the integer it
is called with.

It is thus hardly more involved than a simple call to read the low bits of
the TSC and is more flexible. The design decision (which was forced on me
because of considering old CPUs) has, I think, led to a more modular design
in that other parts of the OS will be able to use the function regardless of
the age of the CPU. The name is a reminder that callers have to take into
account that the returned value may not be a real timer count.

In terms of implementations there would be one uint_tsc_or() routine for
CPUs with a TSC and another such routine for CPUs without.

I could have chosen to avoid CPUs prior to the Pentium (or whereever the TSC
was introduced) but ISTM that including them in the process has led to a
better design. It might even be portable to other architectures.

> I'm fond of the 6502 because I learned on a variant of it. The
> 6502 had a large set of support chips which made it popular with
> some electronic hobbyists too, especially those that needed
> terminal video. I'm also fond of the Z80 because the electronic
> hobbyist magazines used it. The Z80 could refresh DRAM without
> support chips. So, electronic projects using the Z80 were simpler,
> smaller.

Happy memories! My first exposure to machine code was the 6502 on the PET.
There were very few zero page locations available. You may remember that
they were needed in order to form 16-bit addresses off register Y.

Moving to the Z80 on the Sinclair Spectrum was a luxury with all of its
sixteen-bittery but both CPUs were fun to learn on.

> So, the electronic hobbyists liked the Z80 and 6502 in the 1980's.
> Today, the "maker" community embraces the Arduino and Raspberry Pi.

At some point I would like to spend some time looking at how Arm differs
from Intel. That would help with design decisions such as that for the TSC
above. But that's for the future.

James

[James Harris (es)]

"Ivan Shmakov" <onei...@gmail.com> wrote in message
news:87ehayvh...@violet.siamics.net...

>>>>>> Rod Pemberton <do_no...@notemailnotq.cpm> writes:
>>>>>> "James Harris (es)" wrote in message
>>>>>> news:ks0n1r$g5i$1...@dont-email.me...
>
> [...]
>
> >> If the question is, "why consider pre-80386," ISTM the more hardware
> >> an OS can run on the better. I would like a 16-bit version of the
> >> OS to run on an 8086 machine and 640k of RAM.
>
> > Why? Do you actually have working hardware that old? Who does?
>
> > It's 2013 not 1992. Electronics die. So, you understand pre-486
> > processors are collector's items now, yes? I .e., you could pay a
> > fortune for a _non-working_ pre-486, x86 class processor.
>
> Curiously enough, I seem to have a Am386-based computer.
> (I guess I have to power it on sometime to check if it still
> works. But the last time I did, -- it worked.)
>
> And there's a Russia-made ZX Spectrum clone on the shelves, BTW.
>
> [...]
>
> > So, the electronic hobbyists liked the Z80 and 6502 in the 1980's.
> > Today, the "maker" community embraces the Arduino and Raspberry Pi.
>
> Yes.
>
> Sadly, Raspberry Pi is not open hardware, and neither is it
> particularly friendly to free software (as in: Debian.)

Not sure what you mean by Open but Broadcom has published a hardware
document for the Raspberry Pi. It should be enough to know where the device
register addresses sit in the memory map. There is a copy linked from

http://aodfaq.wikispaces.com/raspberry-pi

> And
> while Arduino Due is 32-bit, it's still too low on cycles for
> its cost.
>
> (Any volunteers to design a decent board "for us, the makers"?)

There seem to be lots of options. Here's an unusual one that I don't see
mentioned much:

http://www.xmos.com/

James

[James Harris (es)]

"wolfgang kern" <now...@never.at> wrote in message
news:ks4047$trd$1...@newsreader2.utanet.at...

...

> Today I'd like to make my very own CPU: (if I had 4M Euro aside I'd do)
> no REX no VEX but nibble based register addressing (for 16 regs)
> available in all modes and I really could announce paging in 64-bit
> mode.

4M Euro not needed. You could design your own CPU and write it using VHDL or
Verilog and implement it on an FPGA. Happy coding!

James

[James Harris (es)]

"Alexei A. Frounze" <alexf...@gmail.com> wrote in message
news:3a3997d5-f4cc-4c84...@googlegroups.com...

On Tuesday, July 16, 2013 9:30:31 PM UTC+4, wolfgang kern wrote:

...

I hate to have to operate with multitudes of slightly differing LD and ST
mnemonics instead of the plain and simple MOV. It's still a
move/copy/load/store/etc, so why have many names for one concept? And why
have "AND reg1, reg2" and at the same time "ANI reg, const", why not just
use "AND" in both cases? And so on and so forth.

The mnemonic is relatively unimportant but didn't the Z80 use "LD" for all
transfers including what other machines call stores? I quite liked the
regularity of that. It makes more sense than "move" which doesn't really
move but copies.

James

[wolfgang kern]

Alexei A. Frounze replied:

...
> Dunno this two but Yes, all hardware oriented folks prefered Z80
> coding style above Intel's shit [starting with MOV instead LD/ST]
> and the absolutely wrong (inpracticable) theory from CS-schools.

|I hate to have to operate with multitudes of slightly differing LD and ST
|mnemonics instead of the plain and simple MOV. It's still a
|move/copy/load/store/etc, so why have many names for one concept? And why
|have "AND reg1, reg2" and at the same time "ANI reg, const", why not just
|use "AND" in both cases? And so on and so forth.

As I worked on several different 8/16-bit MCs often in parallel, I once
created my own common mnemonic to cover almost all instructions. There
were only a few vendor specific... but this changed a lot with x86+++.

My personal disassembler/debug-tools still show LD and ST instead of MOV,
and when I type code I always use the LD variant (8A,8B) for register-
destination while I use ST (88,89) for memory-destination only.

__
wolfgang

[wolfgang kern]

James Harris(es) mentioned:

> ...
>> Today I'd like to make my very own CPU: (if I had 4M Euro aside I'd do)
>> no REX no VEX but nibble based register addressing (for 16 regs)
>> available in all modes and I really could announce paging in 64-bit
>> mode.

> 4M Euro not needed. You could design your own CPU and write it using VHDL
> or Verilog and implement it on an FPGA. Happy coding!

Yeah, I already played around with Lattice products. FPGA-functionblocks
must be connected via gates which slow the story down even more.
And it would need a really huge FPGA to compete a quad x86-64 3.8GHz :)
__
wolfgang

[Rod Pemberton]

"James Harris (es)" <james.h...@gmail.com> wrote in message

news:ks4di0$5gr$1...@dont-email.me...

> "Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message
> news:ks25cv$kt4$1...@speranza.aioe.org...
> > "James Harris (es)" <james.h...@gmail.com> wrote in message
> > news:ks0n1r$g5i$1...@dont-email.me...
> >> "Mike Gonta" <mike...@gmail.com> wrote in message
> >> news:krre6u$tg6$1...@speranza.aioe.org...

> >> > Even the latest x86 offering can run in real mode,
> >> > so why cripple the code for the sake of 8086 (or 186, 286)?
> >>
> >> [...]
> >>
> >> If the question is, "why consider pre-80386," ISTM the more
> >> hardware an OS can run on the better. I would like a 16-bit
> >> version of the OS to run on an 8086 machine and 640k of RAM.
> >>
> >
> > Why? Do you actually have working hardware that old? Who
> > does?
>
> In answer to the first question, yes, I have some specific
> reasons No, I don't have hardware that old (but there are
> emulators).
>
> As I've commented before (and I know YMMV) IMO an OS is
> principally defined by the environment it presents to apps.

Okay, that's interesting concept. It's in a similar vein to the
web browser as OS concept, which MS got slapped for, but Google
embraced a few years later.

> Having some different targets for the OS to run on can be a big
> help in keeping the design of that environment isolated from the
> hardware.

True.

> Real mode is not a vast difference but it is convenient and easy
> to access.
>
> There's also a potential performance benefit of considering the
> 8086 as a target. Modern machines are really ridiculously fast
> but you'd never know that judging by the OSes that run on them.

I agree.

Win98/SE runs real fast as compared to newer Windows, but it's
falling apart at the seams...

> I suspect that's down to very poor software.

That could be a factor. But, hardware and processor capabilities
are issues too.

> The machines we use now are orders of magnitude faster than
> the early PCs yet the OSes can appear to users to be slow.

Yes, it's called software bloat.

> I like the challenge of writing an OS that would be seen as fast
> even on an old computer such as an 8086 or an 80386. Having
> the old stuff as one target is more of a challenge but I think it
> helps keep my feet on the ground in terms both of CPU and
> memory usage.

Ok.

> > It's 2013 not 1992. Electronics die. So, you understand
> > pre-486 processors are collector's items now, yes? I.e., you
> > could pay a fortune for a _non-working_ pre-486, x86 class
> > processor.
>
> I see that they are very expensive. Bizarre, isn't it!
>

...

> Here's an example to illustrate the result of deciding to support
> old CPUs.
> Feel free to criticise it.
>
> I currently have functions like
>
> unsigned int uint_tsc_or(unsigned int);
>
> It's job is to get the low bits of the time stamp counter. The
> "_or" part is because some CPUs that the OS will run on will
> not have a TSC or anything else suitable (high resolution and,
> more importantly, extremely low latency). On hardware without
> a TSC the routine will return the integer it is called with.
>
> It is thus hardly more involved than a simple call to read the
> low bits of the TSC and is more flexible. The design decision
> (which was forced on me because of considering old CPUs)
> has, I think, led to a more modular design in that other parts
> of the OS will be able to use the function regardless of the age
> of the CPU. The name is a reminder that callers have to take
> into account that the returned value may not be a real timer
> count.
>
> In terms of implementations there would be one uint_tsc_or()
> routine for CPUs with a TSC and another such routine for CPUs
> without.
>
> I could have chosen to avoid CPUs prior to the Pentium (or

> where ever the TSC was introduced) but ISTM that including

> them in the process has led to a better design. It might even be
> portable to other architectures.
>

You're looking for coherence. So, why not have the uint_tsc_or()
routine called for all machines? Let the failure codes decide what
you should do with the TSC value returned.

E.g.,

/* in header file .h */
#define unknown 0
#define 8086 1
#define 80186 2
#define 80286 3
#define 80386 4
#define 80486 5
#define PENT 6 /* for all newer */
#define AMIGA 7
#define DECVAX 8
/* etc */

/* in setup C code .c */

int get_system(void)
{ /* use compiler define for compile environment */
#ifdef is8086
return(8086);
#endif
#ifdef is80186
return(80186);
#endif
#ifdef is80286
return(80286);
#endif
#ifdef is80386
return(80386);
#endif
#ifdef is80486
return(80486);
#endif
/* more ... */
}

int uint_tsc(unsigned int *tsc)
{
int ret;

ret=-1; /* default to failure */

switch(get_system())
{
case 8086:
case 80186:
case 80286:
case 80386:
ret=-1; /* in case someone changes it above */
/* failure, not present */
break;
case 80486:
if(test_for_X86_TSC())
{
get_X86_TSC(tsc);
ret=0;
}
break;
case PENT:
/* always good */
get_TSC(tsc);
ret=0;
break;
case AMIGA:
if(test_for_AMIGA_TSC())
{
get_AMIGA_TSC(tsc);
ret=0;
}
break;
/* ... */

}
/* if ret is not set above, should be -1 here */

return(ret);
}


/* elsewhere in OS code .c */

sts=uint_tsc(&TSC);
if(sts!=-1)
{ /* TSC is available */
if(!sts) /* sts==0 */
{ /* TSC value is good */
/* use TSC */
}
else
{ /* TSC value is bad */
/* error message or retry code */
switch(sts)
{
/*...*/
}
}
}
else
{ /* TSC is not available */
/* simulate TSC and/or set flags to ignore TSC dependent code */
}


Too messy? You could put the if thens into use_uint_tsc():

sts=uint_tsc(&TSC);
use_uint_tsc(TSC,sts);


Rod Pemberton

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message

news:ks7h63$21u$1...@speranza.aioe.org...

...

>> As I've commented before (and I know YMMV) IMO an OS is
>> principally defined by the environment it presents to apps.
>
> Okay, that's interesting concept. It's in a similar vein to the
> web browser as OS concept, which MS got slapped for, but Google
> embraced a few years later.

I don't know whether the MS or Google concept as as low level. (It might
be.) I was thinking that programs (applications) are written to work with
certain system calls and a certain environment. Something like the following
chain where the apps call the OS, the OS invokes device drivers and the
device drivers control the hardware.

apps
os
device drivers
hardware

I was thinking that it should be possible to change the lower levels without
altering the upper levels, i.e. to run a given app on different hardware. As
long as the app's interface to the OS is the same the app can make the same
calls it would on any other machine and get responses that it understands.
In that sense the most important or defining part of the OS is the interface
it presents to apps. Just an opinion.

Thanks for going to the trouble of writing this code.

I'm fascinated by the idea of writing for something that I've no experience
with like the Amiga. Have you tried? I took a look and see there are lots of
models, some of which may be cheaply available. Would you say there are some
which are more suitable as OS targets than others? Any C compiler
recommendations? Any assembler recommendations? I see they used Motorola
68xxx and other CPUs which would be fun to play with. I know I haven't the
time and really shouldn't but I am tempted to buy a cheap one and try it
out.

Specifically on the TSC issue, rather than having some code with selections
in it how about doing the following?

1. Still have each app call uint_tsc_or(), as above.

2. Have that call linked to a piece of code which suits the hardware. For
example, on a machine without a TSC such as the 386 the code would be

uint_tsc_or:
mov eax, [esp + 4 + 0] ;Fetch the supplied value
ret

However, on something like the Pentium which has a TSC the code would be
more like

uint_tsc_or:
push edx ;Save the caller's EDX
rdtsc
pop edx ;Restore the caller's EDX
ret

How does that idea look? Neither machine has to go through any set of tests
so the code would be fast (which is particularly important when reading the
TSC but is always a good idea). Further, in order to port the OS to a
different machine there would be no need to change the main part of the
source code. All that would be required is to supply a version of
uint_tsc_or() that suits the machine.

James

[Rod Pemberton]

"James Harris (es)" <james.h...@gmail.com> wrote in message

news:ksbkke$h5n$1...@dont-email.me...

You need a bunch of stuff, like:
0) common programming language
1) compiler independent code
2) hardware independent code
3) common API
4) (possibly) a portable "binary" or executable format, or
interpreted code

There are some notable attempts that make code as independent as
possible, e.g., Java, Tendra's ANDF, X11, possibly MS CIL, etc.

> I'm fascinated by the idea of writing for something that I've
> no experience with like the Amiga. Have you tried?

I never programmed an Amiga.

I did do some DEC VAX coding years ago.

> Specifically on the TSC issue, rather than having some code with
> selections in it how about doing the following?
>
> 1. Still have each app call uint_tsc_or(), as above.
>
> 2. Have that call linked to a piece of code which suits the
> hardware. For example, on a machine without a TSC such

> as the 386 the code would be [...]
>

How do you select what code gets compiled and linked for a
specific machine? Makefile? Linker script? If so, you're moving
into compiler specific territory.

> How does that idea look?

It's purely implementor's choice. It does have the advantage
you've listed below.

> Neither machine has to go through any set of tests
> so the code would be fast (which is particularly important
> when reading the TSC but is always a good idea).

True.


Rod Pemberton

[James Harris (es)]

"Rod Pemberton" <do_no...@notemailnotq.cpm> wrote in message

news:kshs80$131$1...@speranza.aioe.org...

Yes. C is ideal.

> 1) compiler independent code

Yes. I am being careful to write the C code to be compiler independent.

> 2) hardware independent code

Yes, for the most part. There would be a limited amount of hardware-specific
code for each target.

> 3) common API

Yes. This has to be the key requirement.

> 4) (possibly) a portable "binary" or executable format, or
> interpreted code

It may be possible but I don't think it would be a good idea for the bulk of
an OS as it would be too slow or require specific support. Also, I don't
think it would be necessary. The code could be easily built for different
targets.

> There are some notable attempts that make code as independent as
> possible, e.g., Java, Tendra's ANDF, X11, possibly MS CIL, etc.

...

>> Specifically on the TSC issue, rather than having some code with
>> selections in it how about doing the following?
>>
>> 1. Still have each app call uint_tsc_or(), as above.
>>
>> 2. Have that call linked to a piece of code which suits the
>> hardware. For example, on a machine without a TSC such
>> as the 386 the code would be [...]
>>
>
> How do you select what code gets compiled and linked for a
> specific machine? Makefile? Linker script?

They would be fine if building on a Unix system. At the moment I am using a
makefile.

> If so, you're moving
> into compiler specific territory.

Well, the C source code would be indepdent of the compiler. Only the
commands to build images would have anything specific about them.

I was going to post some of my current makefile but makefiles can be
cryptic. The main idea, however, is simple. What I am doing at the moment is
forming two lists of object files: one for 16-bit targets and one for 32-bit
targets. Then I build the source files into those object files. This all
happens within the makefile.

At the command level I can just add a new C file to the appropriate source
directory and type

make

As is normal, make will build only those files which are new or have changed
so it is very fast.

I have separate sections so I can build just sets of objects.

make objs (the current default, builds all the object files)
make objs_16 (builds just the 16-bit object files)
make objs_32 (builds just the 32-bit object files)

This is very much experimental. I've not used a makefile in quite the same
way before but so far it works well. It is certainly convenient.

James

[malloc]

Hello.
I'm interested in 16-bit protected mode, as I would like to develop some 16-bit applications (an operating system actually, rather different from an application I guess), and I was wondering if there is any easy way forward to switching to it? The only reference I've found is:

This is the only protected mode available on 80286 processors. Segments can have any length between 1 and 216 = 64 kilobytes. A segment base has 24 bits on an 80286 CPU, limiting the available address space to 16 megabytes. On 386 and higher CPUs, a segment base can have 32 bits. Thus, even in 16-bit protected mode, the complete 32-bit address space of 4 gigabytes is available (although many segments are required to use the complete address space).
Near pointers are 16-bit offsets interpreted relative to a segment register. Far pointers consist of a 16-bit selector and a 16-bit offset.

From OnTime (http://www.on-time.com/rtos-32-docs/rttarget-32/programming-manual/x86-cpu/protected-mode/16-bit.htm).

This is leading me to taking a look at some datasheets for old CPUs (mainly the 286, as mentioned above, as it would have to be documented, because it is the only available protected mode. I recall Windows 3 used a protected mode kernel? Sorry to resurrect an old topic, and this may be a pointless question as I might find an answer anyway, but is there anyone here who has done something like this before (16-bit PM switch)?

[Rod Pemberton]

I haven't ever used 16-bit PM, but the 286 only had 24 address lines.
It could only address 16MB total. I believe that limitation continued
even for 32-bit processors with a 32-bit address space. I.e., 16-bit PM
on a processor after the 286 *cannot* address 4GB - if I'm correct. The
size of the segment base for 16-bit PM would need to be increased from
24-bits to 32-bits to support 4GB. 32-bit PM has segment bases which
are 32-bit, but I think 16-bit PM is still 24-bit.


Rod Pemberton

--
Cars kill more people than guns in the U.S.
Yet, no one is trying to take away your car.

[NimbUs]

Rod Pemberton dit dans news:op.xxa724tlwa0t4m@localhost:

I designed a protected mode "supervisor" as it was called
for an i80286 system in 1984-85. All development was thrown to
the garbage due to the unexpectedly(to me) quick arrival of 32-
bit systems to market.

> I haven't ever used 16-bit PM, but the 286 only had 24 address
lines.
> It could only address 16MB total. I believe that limitation
continued
> even for 32-bit processors with a 32-bit address space. I.e.,
16-bit PM
> on a processor after the 286 *cannot* address 4GB - if I'm
correct.

No, Rod, you are making things up - as you too often do.

(snipping Rod's fairy tales...)

O.P : "16 bit protected address mode" is NOT limited to 24-bit
addresses. Any segment can be as large as 4 gigabytes - or as
small as 1, or even 0 ("expand-down") byte :=)

You should look up and study sections on memory addressing and
"systems programming" of any Intel (or AMD) manual for the
"ia32" architecture. In particular everything GDT (whose format
is unchanged between 16 and 32 bit modes), the different kind of
descriptors, gates etc.

Why do you want to write a 16-bit protected address mode OS
however I'm curious to know : you have a particular goal or is
it just for the learning ?

--
NimbUs

[malloc]

> Why do you want to write a 16-bit protected address mode OS
> however I'm curious to know : you have a particular goal or is
> it just for the learning ?

Mostly for learning. I'd like to create a small DOS (not a weekend project, I know ;-), and 16-bit PM seems better than RM (or am I mistaken?). So far, as I understand, it's exactly the same as 32-bit PM, however with the operand size set to 0 in the GDT? And I think I'll keep the granularity at 0 for 16MB, not 4GB

> You should look up and study sections on memory addressing and
> "systems programming" of any Intel (or AMD) manual for the
> "ia32" architecture. In particular everything GDT (whose format
> is unchanged between 16 and 32 bit modes), the different kind of
> descriptors, gates etc.

Is it just me, or has it become harder to find old CPU datasheets? Seemed much easier last year. I still can't get my head around the idea of segments, even after reading a few tutorials, but I'm still going ;-). I get the basic idea. I'll see if I can get a little assembly to switch to 16-bit PM sometime tonight then.

[malloc]

Also, the source for MS-DOS 1.1 & 2 is freely avaliable, so I can take a look at that to understand how DOS works internally. Plus there are projects like FreeDOS. My initial fantasy was a DOS with the Linux kernel - that's why I thought of using 16-bit PM, to get the 24-bit address space (or even 32-bit if it was really necessary), but then one of the first things the kernel does is switch to 32-bit PM. Not ideal for anything, as getting it to work without having to switch to 32-bit would be nigh-on impossible (too much code). So I decided that writing my own would be "easier". Would using C be a good idea for the higher-level bits, after all the low-level stuff such as I/O is handled by some assembly code?

[malloc]

Actually, there is ELKS - I'll take a look and consider using it as a base. It will probably produce better results https://github.com/jbruchon/elks

[wolfgang kern]

"malloc" wrote:
<q>

</q>

My OS is a code mix of RM,PM16,PM32 and soon also 64-bit long mode.

If you just need a switch from RM to PM16:

first setup GDT and IDT as desired and define at least on code-
and one data-segment, the switch is:

CLI ;please don't disturb
MOV eax,CR0
OR AL,1
MOV CR0,eax
; jmpf next the assembler cannot know how to do this!
db EA
next_offset:
dw next
seg:
dw ?
; so only the manually way will work:
: offset of next to 'next_offset'and
; your new 16 bit code-segment descriptor, which better point
; to this current RM segment yet goes to 'seg'.
next: ;in protected mode from now on
mov ax, dataseg_descriptor ;either 16- or even flat 32-bit
mov ds,ax
mov ss,ax
mov sp,...
STI ;allow IRQs now, if you brave enough ...

__
wolfgang

[Rod Pemberton]

On Sun, 19 Apr 2015 06:57:22 -0400, NimbUs <nim...@xxx.invalid> wrote:
> Rod Pemberton dit dans news:op.xxa724tlwa0t4m@localhost:

>> [...]

>
> I designed a protected mode "supervisor" as it was called
> for an i80286 system in 1984-85. All development was thrown to
> the garbage due to the unexpectedly(to me) quick arrival of 32-
> bit systems to market.
>

Why do you think it was wasted?

It's still useful even to this day. E.g., you can trap ports used
by the BIOS, e.g., to code new 32-bit routines. You could use it
to create a DOS console window for DOS. This would allow 32-bit
routines to be used for faster disk and graphics, i.e., you could
use it to speed up DOS. It would allow anyone to use the DOS of
their choice and, possibly, have the performance of a 32-bit OS
like Windows 98/SE. You could use it to trace the interrupts used
by DOS for it's IOCTL or IFS or network redirector, etc.

>> I haven't ever used 16-bit PM, but the 286 only had 24 address
>> lines. It could only address 16MB total. I believe that
>> limitation continued even for 32-bit processors with a 32-bit
>> address space. I.e., 16-bit PM on a processor after the 286
>> *cannot* address 4GB - if I'm correct.
>

> No, Rod, you are making things up - [...]

I clarified that statement. Usually, Intel/AMD preserve
past behavior to ensure 100% compatibility. I'd be rather
surprised that they would allow 16-bit PM code to break,
by increasing the segment base for 16-bit PM on later
processors.

> No, Rod, you are making things up - as you too often do.

I'm not sure where the basis for this insult comes from.

This is about the only speculative post I've ever posted
and it was clarified that it was based on the fact that
Intel doesn't break things.

> O.P : "16 bit protected address mode" is NOT limited to 24-bit
> addresses. Any segment can be as large as 4 gigabytes - or as
> small as 1, or even 0 ("expand-down") byte :=)

Well, as I said, I'm not familiar with 16-bit PM, so let's
look it up with the original 286 and 386 hardware and programmers
reference manuals.

According to the 386 Programmer's Reference manual, you're wrong,
at least, partially, perhaps, totally.

16-bit addresses and operands, i.e., needed for 16-bit PM, is only
available when the D (default) bit is clear on the code segment's
descriptor. A clear D bit also restricts the code segment size to
64K. So, "any segment," as you claimed, can't be 4G for 16-bit PM,
since the code segment can't be for 16-bit PM. It can only be 64K.

The B (big) bit, if clear, will limit data segment size to 64K.
The 386 manual shows the B and D bits of PM descriptors as always
being set to '1' for normal 386 operation, i.e., 32-bit PM. The
286 Programmer's Reference manual says the upper word of *ALL* 286
descriptors is reserved and "MUST BE SET TO ZERO", i.e., this is for
16-bit PM. This clears the B bit for 16-bit PM limiting the data
segment to 64K. The 386 Programmer's Reference manual says any 286
programs, i.e., 16-bit PM, using bits in the reserved area of any
descriptor (PM) may cause the programs to "not run correctly" on
the 386 processor.

Can you use the B (big) bit set for 16-bit PM to get 4G of address
space for the data segment? Possibly, but the 286 and 386 manuals
seem to *strongly* warn against doing so. You'd also still be
restricted to 16-bit address and operands due to the cleared D
(default) bit needed for 16-bit PM code segment. So, I don't see
how this combination could work to access more than 64K even with
the B (bit) bit set to 4G since the instructions are still
restricted to 64K ... Perhaps, there is some way to overcome this,
since "unreal" mode does something similar, but for 16-bit RM, but
it clearly wouldn't be _normal_ 16-bit PM code. It'd be custom.

There is a section covering the differences between 286 16-bit PM
and 386 32-bit PM in the 386 manual. The 286 manual shows the
differences in the descriptors between 286 and 386, i.e., the
reserved upper bits which must be cleared, and those that aren't
used on 386 for 32-bit code.

> You should look up and study sections on memory addressing and
> "systems programming" of any Intel (or AMD) manual for the
> "ia32" architecture. In particular everything GDT (whose format
> is unchanged between 16 and 32 bit modes), the different kind of
> descriptors, gates etc.

They're (GDT descriptors) not entirely unchanged from 286 to 386.
The 286, i.e., for 16-bit PM, has some constraints. See above.

Any manual? No, I don't believe so.

16-bit PM is not covered in modern x86 manuals, by AMD or Intel,
at least for manuals between 2006 to 2012 or so. It was 2006 or
maybe 2007 when an asshole on Usenet pointed out to me that the
x86 had a 16-bit protected mode, which wasn't covered in the
official manuals, ... anywhere.

[Rod Pemberton]

On Sun, 19 Apr 2015 08:25:05 -0400, malloc <joe.t....@gmail.com> wrote:

> Is it just me, or has it become harder to find old CPU datasheets?

Please see the "address or value?" thread.

Wolfgang, Herbert, and James and I posted links to manuals.
I also posted a link to another post of mine ("boot code"
thread) with links to just about every AMD or Intel manual.

[Rod Pemberton]

On Sun, 19 Apr 2015 08:31:58 -0400, malloc <joe.t....@gmail.com> wrote:

> Also, the source for MS-DOS 1.1 & 2 is freely avaliable, so I can take
> a look at that to understand how DOS works internally. Plus there are
> projects like FreeDOS.

DR-DOS/OpenDOS has source too. It's compatible with DOS 5.0, IIRC.
AIR, there was some lawsuit which was settled with MS where DR-DOS
got rights to compatibility up to a certain point. I.e., you'd be
better to learn from a more up to date version of DOS. FreeDOS seems
to still have some incompatibilities, etc.

> My initial fantasy was a DOS with the Linux kernel -

By "with" do you mean Linux on DOS or DOS on Linux?

If DOS on Linux, you have DOSBox, dosemu, and Qemu. dosemu and Qemu
will need a DOS to execute. DOSBox has a minimal simulation.

If Linux on DOS, yes, I was intrigued by that idea some years ago ...

The DJGPP project ported GCC to DOS, but DJGPP uses it's own custom
DOS C library, i.e., not GLIBC. So, it's possible to port some Linux
code to DOS using DJGPP's port of GCC. The first problem is that
GLIBC is not available for DOS. This was because the DJGPP project
didn't want their applications to fall under GNU GPL. This makes
building regular Linux apps for DOS a pain, and probably, almost
prevents the compilation of the Linux kernel. The second problem
is that DOS doesn't have the filesystem drivers and support that
Linux OS requires. DJGPP simulates some of this in their C library,
but you really need full POSIX support and a raw filesystem driver
layer added to DOS. If I was doing this, I'd port GLIBC to DOS first.
This would require filling in the missing low-level routines. Next,
I'd use code from a Linux derived project, such as Coreboot's (a.k.a.
LinuxBios) FILO to obtain the filesystem driver code for Linux. FILO
has BIOS independent PM disk routines derived from Linux code for IDE,
USB, SATA, ext2, FAT, iso9660, multi-boot, and ELF.

Of course, you can start Linux from DOS using "linld" or "loadlin".
You just can't return to DOS once Linux has started.

> that's why I thought of using 16-bit PM, to get the 24-bit address space
> (or even 32-bit if it was really necessary), but then one of the first
> things the kernel does is switch to 32-bit PM. Not ideal for anything,
> as getting it to work without having to switch to 32-bit would be
> nigh-on impossible (too much code). So I decided that writing my own
> would be "easier".

...

> Would using C be a good idea for the higher-level bits,

IMO, C is a good idea in general, however, some people prefer assembly.

C allows for fast code development, IMO, and DOS allows direct access
to the hardware. However, you need to keep in mind that C compilers
only support certain modes, e.g., some support 16-bit and 32-bit, others
32-bit only, others 32-bit and 64-bit, etc. You also need to keep in
mind that you become slightly dependent on that compiler and it's
libraries. If they have special code or setup, you may need to mimic
those aspects. You'll also be subject to any bugs in the C libraries, etc.

> after all the low-level stuff such as I/O is handled by some assembly
> code?

C can do this too. GCC and OpenWatcom etc support inline assembly,
and generally have macro's or functions for things like port I/O.

[Wildman]

On Sun, 19 Apr 2015 15:21:41 -0400
"Rod Pemberton" <bo...@lllljunkqwer.cpm> wrote:

> If Linux on DOS, yes, I was intrigued by that idea some years ago ...

Have you seen this? http://www.andlinux.org/
Gives you the ability to run Linux programs in Windows.
The Linux kernel is loaded as a service.

Not exactly DOS though.

--
<Wildman> GNU/Linux user #557453
The cow died so I don't need your bull!

[malloc]

The initial idea was linux, but stripped down and providing the interface and functionality of DOS. I did not think that the compiler would automatically compile in 32-bit, thanks for pointing that out ;-). I could use program such as Borland Turbo Assembler in a DOS VM if I really needed to. I'm writing a kind of reference for FAT 16, using a DOS formatted disk as an example. I'll post it up when I'm done.

[Rod Pemberton]

On Mon, 20 Apr 2015 08:26:04 -0400, malloc <joe.t....@gmail.com> wrote:

> The initial idea was linux, but stripped down and providing
> the interface and functionality of DOS.

An updated version of very early Linux is here.
It's simple enough that it *might* compile for DOS ...

DrACOnUx
http://draconux.free.fr/os_dev/linux0.01.html


Other Linux links that might interest you:

ELKS
http://elks.sourceforge.net/

TinyCore
http://distro.ibiblio.org/tinycorelinux/welcome.html

uClinux
http://www.uclinux.org/


Also, DJGPP project already compiled BASH shell, and many
other GNU/FSF and FOSS utilities for DOS. So, if you just
need a shell, it's available via DJGPP.

[malloc]

OK, I'll take a look at those. Thanks ;-)