FX!16 ?
Rod Pemberton
BIOS, DOS, and Video calls without spending our lifetimes writing
replacement routines which are obsolete by the time we complete them.
What we need is something to leverage the code already written for us by
others: BIOS,DOS, VESA. What we need is binary translator and emulator.
Not a slow emulator like NTVDM, not an moderate "dynamic translator" like
QEMU, but a fast emulator/binary translator like DEC's ancient FX!32. Once
the code was emulated once, FX!32 produced native Alpha .dll's for each
executed code piece.
What that means is, if we had a program which _extremely_ thoroughly tested
16-bit BIOS, DOS, and Video functions and such a translator, we could
produce functionally equivalent 32-bit or 64-bit .dll's for BIOS, DOS, and
Video. All done automatically. All coded properly for our MB chipsets. We
wouldn't have to wean our PM programs of these 16-bit functions. We'd just
have to load the 32/64-bit version of them and call it.
What we need is an FX!16 which would produce native x86 .dll's. Imagine
a 64-bit OS using 64-bit VESA routines or BIOS disk routines created from
your BIOS' 16-bit routines! Imagine a 32-bit OS using 32-bit DOS calls
created from your DOS' 16-bit routines!
Is there a way to save the code snippets which are translated by a "dynamic
translator" such as QEMU? It'd be slow to convert all BIOS calls, etc., but
it is also already programmed...
As a worst case scenario we could insert an address and operand override
prefixes on any instruction that will take them (or switch the current state
of overrides). And, emulate any instruction which won't take the prefixes,
fails to set bits properly (shifts), or uses segment registers, or
branches...
The FX!32 system is made up of seven or so Windows and Alpha specific
modules. AFAICT, the FX!32 "runtime and emulator" are deciding which linear
code snippets (entry and exit address, no branching code) are to be
disassembled, translated, and recompiled by the FX!32 "binary translator".
They do the work. All of this can be recreated differently on x86 with
assembly and disassembly and with Intel's trap or debug register tracing.
Of course, they take things to a level or extreme we won't be approaching.
Most of the other complicated FX!32 stuff appears to be necessary only if
you choose to do the conversion incremementally over many executions of the
program. If an _extremely_ thorough test program was written to properly
test all known BIOS calls and sub-functions, the entire BIOS would be
converted at once. There'd only be a problem if some unknown function
wasn't called and converted or if some hardware or software wasn't installed
during conversion: HD, CD-Rom, MSCDEX, mouse driver, and was inaccessible.
There should be no legal issues for FreeDOS, etc. Under current US
copyright law and Supreme court rulings, an individual can do anything to
licensed code for personal use (even if it contradicts the license
agreement). So, BIOS and MS-DOS shouldn't be an issue either as long as
people don't attempt to distribute converted images or .dll's.
For the most part, I don't think translating all functions is an issue with
BIOS or DOS for any computer properly setup to run DOS. And, if there was a
missing function, we could trap it like a DPMI host prior to a real-mode
callback. Save the int number args/regs to a file and have the test program
test the missing call.
Also, nearly all of the Windows overhead and Alpha recompile/reoptimize
stuff isn't necessary because Windows isn't there and we're using the same
register set.
One .pdf documents mentioned some patents pending. But, since all the
techniques we'd be using are standard x86 programming, overrides,
disassembly, reassembly, x86 cpu emulation, trap flag, debug registers, and
predated whatever they patented by a decade, I don't think that'd be a
problem. Or, the technigue was designed by Intel/AMD/IBM for x86 and since
we're on x86 it is completely fair use. The extreme recompile/optimize
stuff they created shouldn't be needed by us. And, if a problem does arise,
which I doubt, there are other binary translators out there that use
different techniques. And, without access to their code and the skimpy and
Alpha specific implementation details in the documents, I don't think we
could even come close to patent infringement on anything. In fact, I don't
think there would be a need for any technology that hasn't been used by
standard debuggers, compilers, and assemblers since the mid to late '80's.
Rod Pemberton
Alexei A. Frounze
> Here we are, a bunch of OS developers, and we can't wean our code of
> 16-bit BIOS, DOS, and Video calls without spending our lifetimes
> writing replacement routines which are obsolete by the time we
> complete them.
>
> What we need is something to leverage the code already written for us
> by others: BIOS,DOS, VESA. What we need is binary translator and
> emulator. Not a slow emulator like NTVDM, not an moderate "dynamic
> translator" like QEMU, but a fast emulator/binary translator like
> DEC's ancient FX!32. Once the code was emulated once, FX!32 produced
> native Alpha .dll's for each executed code piece.
What about self-modifying code or code that creates instructions on the
stack in the run time?
What about the code that enables and disables interrupts (in flags register
and hardware like PIC) and modifies the hardware registers you also likely
to work with? Would you safely let that happen in a multithreading system or
system with timing requirements? The BIOS doesn't guarantee any timings...
Alex
Rod Pemberton
news:yPadnZ7Y4OacezHY...@comcast.com...
I was beginning to wonder what happened to you... Long holiday? Welcome
back. :)
I haven't begun to deal with any of the issues you brought up. At the
moment, I'm just toying with ideas on basic instruction conversion. I have
read a number of the professional articles on FX!32 to give me a rough idea
of complexity.
http://citeseer.ist.psu.edu/cs
I looked into this a bit. This is what I concluded:
1) QEMU probably wouldn't be useful since it breaks each instruction into a
number of smaller instructions.
2) It would be best to not slap address and overide prefixes on every
instruction, if possible.
a) attempt to keep 8-bit instruction encodings 8-bit
b) attempt to keep 16-bit instruction encodings by letting them
"convert" to 32-bit
3) Segmented addressing is the real problem. Need a method to add a base
address to every address. Must emulate all instructions which use segments
to update the base address.
a) keep segmentation, set the base address in the descriptor of a
selector every time a segment register is loaded.
b) reject segmentation, converting to position independent code, e.g.,
changing the addressing mode to use another register: ebx which would
contain the base address.
Anyway, I put the project on the back burner for the moment. I like the
idea. I think it'd be great for future OS developers, if doable. But, I
think it has a huge upfront development cost. It might just be easier to
finish the disk routines in my OS.
Rod Pemberton
Alexei A. Frounze
> "Alexei A. Frounze" <ale...@chat.ru> wrote in message
> news:yPadnZ7Y4OacezHY...@comcast.com...
>> Rod Pemberton wrote:
>>> Here we are, a bunch of OS developers, and we can't wean our code of
>>> 16-bit BIOS, DOS, and Video calls without spending our lifetimes
>>> writing replacement routines which are obsolete by the time we
>>> complete them.
>>>
>>> What we need is something to leverage the code already written for
>>> us by others: BIOS,DOS, VESA. What we need is binary translator and
>>> emulator. Not a slow emulator like NTVDM, not an moderate "dynamic
>>> translator" like QEMU, but a fast emulator/binary translator like
>>> DEC's ancient FX!32. Once the code was emulated once, FX!32
>>> produced native Alpha .dll's for each executed code piece.
>> ...
>>
>> What about self-modifying code or code that creates instructions on
>> the stack in the run time?
>> What about the code that enables and disables interrupts (in flags
>> register and hardware like PIC) and modifies the hardware registers
>> you also likely to work with? Would you safely let that happen in a
>> multithreading system or system with timing requirements? The BIOS
>> doesn't guarantee any timings...
>>
>
> I was beginning to wonder what happened to you... Long holiday?
> Welcome back. :)
Yep. Spent 3 weeks with my parents in Moscow. I've been using comcast for
newsgroups and decided not to post for a while using a different server and
account.
> I haven't begun to deal with any of the issues you brought up. At the
> moment, I'm just toying with ideas on basic instruction conversion.
> I have read a number of the professional articles on FX!32 to give me
> a rough idea of complexity.
> http://citeseer.ist.psu.edu/cs
>
> I looked into this a bit. This is what I concluded:
>
> 1) QEMU probably wouldn't be useful since it breaks each instruction
> into a number of smaller instructions.
>
> 2) It would be best to not slap address and overide prefixes on every
> instruction, if possible.
> a) attempt to keep 8-bit instruction encodings 8-bit
> b) attempt to keep 16-bit instruction encodings by letting them
> "convert" to 32-bit
>
> 3) Segmented addressing is the real problem. Need a method to add a
> base address to every address. Must emulate all instructions which
> use segments to update the base address.
> a) keep segmentation, set the base address in the descriptor of a
> selector every time a segment register is loaded.
> b) reject segmentation, converting to position independent code,
> e.g., changing the addressing mode to use another register: ebx which
> would contain the base address.
Welcome to the x86 virtualization and emulation hell! :)
> Anyway, I put the project on the back burner for the moment. I like
> the idea. I think it'd be great for future OS developers, if doable.
> But, I think it has a huge upfront development cost. It might just
> be easier to finish the disk routines in my OS.
Sure. If the compatibility is the most important or it's a toy, just set up
a v86 server for all those things and execute the BIOS code.
Alex
Rod Pemberton
> Rod Pemberton wrote:
> > I haven't begun to deal with any of the issues you brought up. At the
> > moment, I'm just toying with ideas on basic instruction conversion.
>
>
Yeah, more coding, more difficulty...
When that happens, I tend to recycle ideas to find a better solution or
reveal one I may have missed.
In this case, I'm looking at DOSBox again. It uses the SDL library. DOSBox
claims it emulates a 286 and 386 both RM and PM. An OS/2 DOSBox porting
Wiki page says that DOSBox is a 386 DX emulator with (emulated) BIOS, DOS,
SB16, and SVGA. The only thing I don't need is PM emulation.
If the OS/2 page is correct, this functionality really would be ideal since
my "OS" is only for DJGPP and OW DPMI and/or DOS Extended binaries. It
could drastically reduce my OS coding requirements. But, it brings up
questions.
How complete are the BIOS and DOS API's? How many SDL functions are used by
DOSBox? Is it easier to port SDL to my "OS" or write a custom SDL API? Is
there a DOS DPMI port of SDL out there somewhere which would even easier to
patch into my "OS"?
SDL uses OpenGL on top of DirectX or Xlib, but, unfortunately, SDL doesn't
support DOS. On the other hand, Allegro does support DOS. But, it appears
that AllegroGL (Allegro w/OpenGL) doesn't support DOS either so it probably
wouldn't be easy to convert to SDL. Sigh... But, hey, I guess I don't need
3D support anyway so maybe porting SDL wouldn't be so bad.
Cached SDL news archives seems to indicate the problem with DJGPP was lack
of threading support (and/or VESA), but I seem to recall that a number of
the major Open Source thread packages compile and work fine for DJGPP. No
mention of OpenWatcom. More questions...
Anyway, that's the direction I'm looking at for the moment...
Rod Pemberton
Maxim S. Shatskih
From what I know on SDL, it is the replacement of the GDI's functions like
StretchDiBits which are absent in X11 - X11 is poorer in set of graphics
routines, being more low-level API.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
ma...@storagecraft.com
http://www.storagecraft.com