Adding 64-bit file support to DJGPP
0483218...@t-online.de
I would like to know how difficult it would be to add support for
64-bit file operations to DJGPP. I suppose that DJGPP currently only
handles 32-bit file operations internally; is this assumption correct?
If yes, it would be necessary to extend the internal file handling
routines of DJGPP to 64 bit before any attempt to add support for new
APIs could be made.
Is anyone currently working on this, or has been working on this
before? If yes, I would be interested in his assessment. If not, what
would be a good starting point?
Regards,
Udo
DJ Delorie
DJGPP relies on MS-DOS's file api, and MS-DOS doesn't support 64 bit
operations.
0483218...@t-online.de
> DJGPP relies on MS-DOS's file api, and MS-DOS doesn't support 64 bit
> operations.
The LFN API in MS-DOS 7.0 and higher already uses 64-bit file
operations in some functions, e.g. 714E/4Fh of Int 21h. Also, DOS is
still evolving, so newer DOS versions provide functions that old
versions lack.
For example, this is the new 64-bit LSEEK function of Enhanced DR-DOS,
which will also eventually be supported by FreeDOS:
--------O-2142-------------------------------
INT 21 - EDR-DOS 7.01.08+ - "Long LSEEK" - SET CURRENT 64-bit FILE
POSITION
AX = 7142h
CL = origin of move
00h start of file
01h current file position
02h end of file
BX = file handle
DS:DX = pointer to (signed) 64-bit offset from origin of new position
Return: CF clear if successful
DS:DX unchanged, new 64-bit position from start of file at ->
DS:DX
CF set on error
AX = error code (01h,06h) (see #01680 at AH=59h/BX=0000h)
For the moment, I would like to put the question of API support in the
OSes aside and concentrate on the implementation of 64-bit file support
in DJGPP itself.
Udo
Rod Pemberton
<0483218...@t-online.de> wrote in message
news:1137562915.9...@f14g2000cwb.googlegroups.com...
> DJ Delorie wrote:
>
> > DJGPP relies on MS-DOS's file api, and MS-DOS doesn't support 64 bit
> > operations.
> For the moment, I would like to put the question of API support in the
> OSes aside and concentrate on the implementation of 64-bit file support
> in DJGPP itself.
>
> Udo
>
For personal use, I compiled a very raw but complete list of every interrupt
called by every function for an older version of DJGPP. DJGPP version 3.4.1
calls about 170 DOS, BIOS, etc. interrupts. I'll just list the ones that
call 71xx functions below my signature. If you need to know which ones call
specific functions, just ask in this newsgroup or the other newsgroup you
post DR-DOS announcements too. Have you contacted Henrik Haftmann or Jason
Hood about 64-bit support in DOSLFN?
Rod Pemberton
71xx calls for DJGPP 3.4.1:
remove 713a 7141
_rename 7156
findfirst 714e 71a1
findnext 714f 71a1
_truename 7160
_chmode 7143
_flush_disk_cache 710d
_open 7160 7143 716c
_creat 716c
_creatnew 716c
_get_volume_info 71a0
_lfn_gen_short_fname 7100 71a8
direct_exec_tail_1 7160
_lfn_find_close 71a1
get_sft_entry 71a6
__get_current_directory 7147 7160
mkdir 7139
rmdir 713a
symlink 7160
__getcwd 7147 7160
__chdir 713b
utime 7143
Maurice.Lombardi
Rod Pemberton a écrit:
> For personal use, I compiled a very raw but complete list of every interrupt
> called by every function for an older version of DJGPP. DJGPP version 3.4.1
What is this version number ? just curious...
Maurice
--
Maurice Lombardi
Laboratoire de Spectrometrie Physique,
Universite Joseph Fourier de Grenoble, BP87
38402 Saint Martin d'Heres Cedex FRANCE
Tel: 33 (0)4 76 51 47 51
Fax: 33 (0)4 76 63 54 95
mailto:Maurice....@ujf-grenoble.fr
Charles Sandmann
> operations in some functions, e.g. 714E/4Fh of Int 21h. Also, DOS is
> still evolving, so newer DOS versions provide functions that old
> versions lack.
The unreleased, in limbo development tree v2.04 in cvs contains some
support for the larger file offsets. You could investigate and
contribute support.
> For the moment, I would like to put the question of API support in the
> OSes aside and concentrate on the implementation of 64-bit file support
> in DJGPP itself.
A review of what's been done already would be a good starting point.
DJ Delorie
> Rod Pemberton a écrit:
> > For personal use, I compiled a very raw but complete list of every interrupt
> > called by every function for an older version of DJGPP. DJGPP version 3.4.1
>
> What is this version number ? just curious...
I don't know. I'm still working on djgpp version 2.04.
Rod Pemberton
"DJ Delorie" <d...@delorie.com> wrote in message
news:200601181556....@envy.delorie.com...
Sorry, that's the version for GCC. Ah, so I guess it is a list for the
current version of DJGPP. :)
DJDEV & DJLSR v203
(w/GCC & GPP v341)
(w/GLIB v126)
(w/BNU v215)
RP
Rod Pemberton
I'm curious. Will v2.04 use the standard GNU C library or is that out of
the question?
Rod
DJ Delorie
> I'm curious. Will v2.04 use the standard GNU C library or is that out of
> the question?
That's out of the question.
Rod Pemberton
Ah, well, I figured it'd be much easier to maintain DJGPP if it used the GNU
C library. I think it'd also be a step towards 64-bit support and perhaps
even IBM's Cell cpu.
What exactly is the the issue? That the DOS and Linux API's don't match?
That there needs to be a POSIX layer?
Rod Pemberton
DJ Delorie
> What exactly is the the issue? That the DOS and Linux API's don't match?
> That there needs to be a POSIX layer?
Yes, all of that. A large chunk of the DJGPP C library deals with
letting your application pretend that MS-DOS is really a POSIX.1
system, and hiding all the gory details.
Plus, there's a lot of stuff in glibc that just isn't going to work
under MS-DOS. It really wants to be Linux-only.
In short, it's just a bad idea.
Rod Pemberton
>
> under MS-DOS.
Wow! I find it really interesting that with all the POSIX code you wrote,
and the code for POSIX-on-DOS w/DPMI of the CMU MACH DOS, that a simple
POSIX on DOS 32-bit OS couldn't be written...
I'm currently working on a personal on-top-of DOS Ring 0 OS written almost
entirely in C which (hey, there are CISC instructions...) :
1) compiles with DJGPP, and OW1.3, small amount of NASM
2) from RM, dumps the DJGPP DPMI host or the OW DPMI host/dos-extender and
returns to PM ring 0 32-bit 4Gb base 0
3) setups a new GDT and IDT
4) reprograms PICs
5) setup IRQs (rough, but there, C stack frame getting in the way for some
functions :( )
6) implements ISRs (proto routines for 0x31,0x21,0x33,0x16)
7) (in progress) initializes keyboard and mouse routines (working on the
buffers/queue)
I'm hoping to finish the keyboard and mouse routines soon so I can move on
to floppy/hard disk. I'll also need to get to the PITs sometime... With
time, it could possibly exec all DJGPP or OW programs in an exclusively
32-bit environment... Multitasking should be possible for DJGPP
applications due to the segment memory model. However, OW uses a single 4Gb
segment. I'm not sure if I can get multi-tasking working for it. I was
suprised at just how closely a large portion of my DPMI dumper code matches
the requirements for a GRUB bootloader...
Rod Pemberton
DJ Delorie
Of course it can be written. It just wouldn't be DJGPP.
Taken to extremes, you've re-invented Linux ;-)
Johann 'Myrkraverk' Oskarsson
> I'm currently working on a personal on-top-of DOS Ring 0 OS written
> almost entirely in C which (hey, there are CISC instructions...) :
Now I'm suddenly interested ;) Is it also possible to write
applications for your OS with OW or DJGPP? If so, I might want to
play a bit.
Johann
--
johann myrkraverk com (you know the drill with the @ and .)
I classify Outlook mail as spam, use something else.
Rod Pemberton
"Johann 'Myrkraverk' Oskarsson" <myrkr...@yahoo.com> wrote in message
news:m3bqy9xl...@jin.myrkraverk.com...
> "Rod Pemberton" <dont...@bitbucket.cmm> writes:
>
> > I'm currently working on a personal on-top-of DOS Ring 0 OS written
> > almost entirely in C which (hey, there are CISC instructions...) :
>
> Now I'm suddenly interested ;) Is it also possible to write
> applications for your OS with OW or DJGPP? If so, I might want to
> play a bit.
>Is it possible...
No, not yet. I'm still at the keyboard and mouse routines. No hard disk
support. No application loader.
Theoretically possible? Yes, in time it should be possible to run DJGPP/OW
apps eventually. I'm not releasing source at this time.
RP
Johann 'Myrkraverk' Oskarsson
> Of course it can be written. It just wouldn't be DJGPP.
>
> Taken to extremes, you've re-invented Linux ;-)
And with an addition of a lisp engine, he'll re-invent emacs ;)
Rod Pemberton
>
> Of course it can be written. It just wouldn't be DJGPP.
>
> Taken to extremes, you've re-invented Linux ;-)
Heh!
I think it's a matter of flavor. I don't like Linux for the most part, but
I do like GCC, Tso's fdisk, dd, and netcat. However, although I had a few
problems with the install, I was really impressed with Vector Linux. It was
blazingly fast. Otherwise, I can live without buggy cfdisk, scripts, ext,
ext2, reiserfs, multi-user, secure logins, all of which are a nightmare in
my opinion. If I have to give up 98SE, I already know I hate XP, so it'll
probably be VL. I think the X in XP stands for "Stop you crazy person!
You're not allowed to do that, because I'm the OS."
Rod Pemberton
Johann 'Myrkraverk' Oskarsson
> "Johann 'Myrkraverk' Oskarsson" <myrkr...@yahoo.com> wrote in message
> news:m3bqy9xl...@jin.myrkraverk.com...
>> Now I'm suddenly interested ;) Is it also possible to write
>> applications for your OS with OW or DJGPP? If so, I might want to
>> play a bit.
>>Is it possible...
> No, not yet. I'm still at the keyboard and mouse routines. No hard disk
> support. No application loader.
> Theoretically possible? Yes, in time it should be possible to run DJGPP/OW
> apps eventually. I'm not releasing source at this time.
Ok, but if/when you do get to that point, I don't mind a slight prod,
either at my email address at the top, or the bottom; in case I'm not
monitoring this or other dos channels then.
0483218...@t-online.de
> For personal use, I compiled a very raw but complete list of every interrupt
> called by every function for an older version of DJGPP. DJGPP version 3.4.1
> calls about 170 DOS, BIOS, etc. interrupts. I'll just list the ones that
> call 71xx functions below my signature. If you need to know which ones call
> specific functions, just ask in this newsgroup or the other newsgroup you
> post DR-DOS announcements too.
Thank you! How did you compile this list?
> Have you contacted Henrik Haftmann or Jason
> Hood about 64-bit support in DOSLFN?
Yes, I have written Jason an email today; I am awaiting his reply.
Udo
0483218...@t-online.de
But was not the Gnu C library designed to be portable to different
OSes? I remember I read that it once supported several OSes, not only
Linux and Hurd. The developers even encouraged other programmers to
develop new ports of glibc; they said it would not be too hard. What
has become of this idea?
Udo
Chuck F.
>
> Wow! I find it really interesting that with all the POSIX code
> you wrote, and the code for POSIX-on-DOS w/DPMI of the CMU MACH
> DOS, that a simple POSIX on DOS 32-bit OS couldn't be written...
DJGPP runs under DOS. That includes older DOSs, FreeDos, quasi-DOS
from windoze 9x etc. Programs for it are portable. They don't
need to worry about GUIs, because there are none.
The fundamental purpose is to make the world of Unix, Posix, Linux
programs available to DOS users. DJGPP does this very well. It
has no problems with DLL hell, etc. It is a command line environment.
--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Rod Pemberton
<0483218...@t-online.de> wrote in message
news:1137630453.9...@o13g2000cwo.googlegroups.com...
> Rod Pemberton wrote:
>
> > For personal use, I compiled a very raw but complete list of every
interrupt
> > called by every function for an older version of DJGPP. DJGPP version
3.4.1
> > calls about 170 DOS, BIOS, etc. interrupts. I'll just list the ones
that
> > call 71xx functions below my signature. If you need to know which ones
call
> > specific functions, just ask in this newsgroup or the other newsgroup
you
> > post DR-DOS announcements too.
>
> Thank you! How did you compile this list?
I compiled the list manually by searching with Windows find. That was just a
_very_ small portion of the list. DJGPP wasn't too bad since most calls use
__dpmi_int(). DJGPP calls about 170 DOS, BIOS, DPMI, etc. functions that I
could find. Today, I think I just found a few more I missed... OW1.3, on
the other hand, was very difficult because of hard code, assembly, various
methods of calling DPMI, etc. I found about 100 DOS, BIOS, DPMI calls for
OW. Of course, it doesn't have LFN support, integrated DPMI support, or
POSIX functions...
I know the LFN api has some non-71xx functions. Time perhaps? I don't
recall at the moment, but let me know if you need more...
Rod Pemberton
Jim Michaels
- there are int13h function calls for basic disk i/o. note that this is deprecated in favor of UEFI.
- there are extended int 13h function calls for LBA-based disk i.o if you don't like old severly-limited CHS. see the phoenix extended 13h disk bios calls pdf. note that this is deprecated in favor of UEFI.
- there are int 21h functions which is what DJGPP is based on for 8.3 file I/O. note that MBR+BIOS calls is deprecated in favor of UEFI.
RayeR
Georg Potthast
>
> opengem is the closest we've got to something usable in DOS for a GUI.
>
http://code.google.com/p/nanox-microwindows-nxlib-fltk-for-dos/wiki/XFDOS
Georg
RayeR
Or is there already GCC toolset for UEFI?
Charles Sandmann
>news:28c2ff16-6cef-46c3...@googlegroups.com...
>instead of DOS/BIOS calls?
support and most of the system calls.
Mostly DJGPP requires DOS calls (and memory data structures); BIOS calls are
mostly when the user needs them.
DJGPP also requires a DPMI provider; CWSDPMI also depends on DOS calls.
Any major update is equivalent to or larger than the DJGPP V2 effort, which
took a couple of years and many
man months of programming effort.
RayeR
Georg Potthast
Now the hardware has to be able to support 32 bit operating systems in 64 bit mode, like Windows XP. For this VT-X has been implemented which allows to emmulate a 32 bit PC in 64 bit mode. This is used by VirtualPC and friends.
Writing a Hypervisor that uses VT-X is really a lot of development since you have to virtualise just about anything. I evaluated an alternative Hypervisor just to see how much is left to implement to make it usable.
DJ Delorie is right to suggest MinGW as an alternative. As long as you do not access the hardware directly it is really easy to port a DJGPP program to MinGW.
Otherwise I think that a robust alternative is to put your DJGPP application together with FreeDOS on a disk image and package that together with a command line version of Qemu for Windows which is about 13 MB compressed. When the user starts your package he/she will start Qemu with FreeDOS starting your application.
I only need a 2 MB FreeDOS distro to run my DJGPP applications. Many of the Qemu files in the 13 MB package you do not need. So the entire package will probably add up to compressed 15 MB which is less than most Windows applications.
Eli Zaretskii
> From: Georg Potthast <dos...@googlemail.com>
> As long as you do not access the hardware directly it is really easy to port a DJGPP program to MinGW.
exactly the opposite: DJGPP has a lot of Posix-like features that
MinGW sorely lacks. So porting to MinGW using a DJGPP port as a
starting point will generally give you a broken port, and in many
cases will simply refuse to compile or link.
Besides, starting with DJGPP will automatically lose the advanced
features you can have with Windows: networking, threads, parallel
processes, etc.
DJ's suggestion to use MinGW is still valid, of course (although MinGW
still doesn't support generation of 64-bit executables; you need to go
to semi-official MinGW64 snapshots). But please don't underestimate
the efforts required for porting a non-trivial package to MinGW.
Heck, even running a configure script is a challenge, and requires an
installation of yet another environment (MSYS).
Georg Potthast
If you have a different experience I guess my test was too quick.
With other words one should look for a solution to run DJGPP on 64bit Windows. This means using virtualisation to avoid a major rewrite of DJGPP.
rug...@gmail.com
On Saturday, May 25, 2013 10:56:19 AM UTC-5, Georg Potthast wrote:
>
> The 64bit hardware does no longer support V86 mode. This mode was
> intended to allow 32 bit operating systems to run 16 bit
> applications, i.e. DOS applications.
can't really use 64-bit long mode and 16-bit real mode at the same
time (without later VT-X).
> Now the hardware has to be able to support 32 bit operating systems
> in 64 bit mode, like Windows XP. For this VT-X has been implemented
> which allows to emmulate a 32 bit PC in 64 bit mode. This is used by
> VirtualPC and friends.
But doesn't VPC only emulate a Pentium 2? And it's not great on
graphical support, hence why MS doesn't recommend it for home users
(games?).
> Writing a Hypervisor that uses VT-X is really a lot of development
> since you have to virtualise just about anything. I evaluated an
> alternative Hypervisor just to see how much is left to implement to
> make it usable.
an existing emulator or hypervisor than roll our own, probably.
FreeBSD is (also) working on their own, bhyve, but for now it only
is targeted at FreeBSD itself.
> DJ Delorie is right to suggest MinGW as an alternative. As long as
> you do not access the hardware directly it is really easy to port a
> DJGPP program to MinGW.
> Otherwise I think that a robust alternative is to put your DJGPP
> application together with FreeDOS on a disk image and package that
> together with a command line version of Qemu for Windows which is
> about 13 MB compressed. When the user starts your package he/she
> will start Qemu with FreeDOS starting your application.
PuppyLinux. IIRC, they stuck to old QEMU 0.8.2 (possibly for Win9x
compatibility).
> I only need a 2 MB FreeDOS distro to run my DJGPP applications. Many
> of the Qemu files in the 13 MB package you do not need. So the
> entire package will probably add up to compressed 15 MB which is
> less than most Windows applications.
Win32 host is a priority for them. IIRC they even want to merge
with KVM. Latest release is 1.5.0.
rug...@gmail.com
On Saturday, May 25, 2013 11:25:39 AM UTC-5, Eli Zaretskii wrote:
> > Date: Sat, 25 May 2013 08:56:19 -0700 (PDT)
>
> >
> > As long as you do not access the hardware directly it is really
> > easy to port a DJGPP program to MinGW.
>
> Did you actually try that? My experience in porting to MinGW is
> exactly the opposite: DJGPP has a lot of Posix-like features that
> MinGW sorely lacks. So porting to MinGW using a DJGPP port as a
> starting point will generally give you a broken port, and in many
> cases will simply refuse to compile or link.
hence the avoidance of any Cygwin .DLL (also for licensing?) and
using "known .DLL" MSVCRT.DLL instead (despite bugs, e.g. tmpfile).
Actually, it's only the old MSVC 6 runtime, not newer, that they're
using, IIRC. Though a lot of MinGW usage is also because MSVC
doesn't really support C99, which MinGW seems to prefer.
> Besides, starting with DJGPP will automatically lose the advanced
> features you can have with Windows: networking, threads, parallel
> processes, etc.
pure ISO C, are you? Almost better to "just use Java" (or Modula-3).
> DJ's suggestion to use MinGW is still valid, of course (although
> MinGW still doesn't support generation of 64-bit executables; you
> need to go to semi-official MinGW64 snapshots). But please don't
> underestimate the efforts required for porting a non-trivial
> package to MinGW. Heck, even running a configure script is a
> challenge, and requires an installation of yet another environment
> (MSYS).
AutoTools was never expected to work on systems without a native
POSIX shell. It might be (barely) wrong to say outside developers
don't care about Windows, but clearly it's not first priority target.
Eli Zaretskii
> From: rug...@gmail.com
>
> Hi,
> > exactly the opposite: DJGPP has a lot of Posix-like features that
> > MinGW sorely lacks. So porting to MinGW using a DJGPP port as a
> > starting point will generally give you a broken port, and in many
> > cases will simply refuse to compile or link.
>
> IIRC, MinGW isn't explicitly targeting POSIX, only "native" Windows,
(and they do have a small number of Posix functionality added). The
rest is up to the person who does the port.
> > Besides, starting with DJGPP will automatically lose the advanced
> > features you can have with Windows: networking, threads, parallel
> > processes, etc.
>
> If you're targeting all of that anyways, you're not really targeting
> pure ISO C, are you?
> Almost better to "just use Java" (or Modula-3).
> > DJ's suggestion to use MinGW is still valid, of course (although
> > MinGW still doesn't support generation of 64-bit executables; you
> > need to go to semi-official MinGW64 snapshots). But please don't
> > underestimate the efforts required for porting a non-trivial
> > package to MinGW. Heck, even running a configure script is a
> > challenge, and requires an installation of yet another environment
> > (MSYS).
>
> Blame those who refuse to code anything outside of POSIX.
> Obviously AutoTools was never expected to work on systems without a
> native POSIX shell. It might be (barely) wrong to say outside
> developers don't care about Windows, but clearly it's not first
> priority target.
run it without MinGW getting in the way.
RayeR
> Writing a Hypervisor that uses VT-X is really a lot of development since you have to virtualise just about anything. I evaluated an alternative Hypervisor just to see how much is left to implement to make it usable.
Georg Potthast
> I don't know much about VT-X abilities but you mean it allows to run 16b >programs with something like RM? What alternative hypervisors you found?
There is a link to a demo CD at bottom of this page:
http://os.inf.tu-dresden.de/~us15/nova/
The developers were a bit upset that they could not demotivate me to test the demo version. They do not really want any users because they consider it a pure research project.
During my tests I wrote a tutorial (diary?) since a lot of things were not documented. You may want to read the tutorial first so you will know what you can expect.
https://www.dropbox.com/sh/n5hvtdskv1te565/u4WhWs6ZTJ
My tests allowed me to run several instances of Linux and FreeDOS simultaniously when booting the demo CD. But the hardware emulation that is included with this Hypervisor is not sufficient for practical uses of these instances. The problems I found are documented in my tutorial.
The Qemu virtual machine is already widely used successfully and therefore I think it is better to use that instead.
Georg
RayeR
It would be interesting if it could be booted from HDD and then boot multiple DOS instances directly from HDD and use this hipervisor as DRDOS task manager. I'm not sure - it's just task switcher or real multitasking (give time slices also for backgroud VMs)?
Maybe such code could be reasonably managed (C is better for me than pure asm) and extended by some needed HW emulations. Maybe also other users on BTTR would be interested. I'm still busy but I will read your PDF and play more with Nova. I also found they linked http://genode.org/ project, also looks interesting.
Georg Potthast
The Hypervisor can run from a hard disk and the developers develop and run it on a hard disk. I just did not have a PC where I could install it on and there were no instructions how to install it on a hard disk. And no instructions how to install it on a partition on a PC with other operating systems installed.
Given the necessary time all your objectives could be met. I just doubt that this time will be available.
I never fully understood what Genode is for. I would name it Middleware which shall be usable on top of several alternative Mikrokernels. I asked the developer for a current demo CD and he said he is always asked for that but he does not get around doing it. So I did not test Genode. My impression is if you want to make DOS multitasking you will not need Genode.
RayeR
I'm still not clear about running the DOS (16bit) there. You wrote that VTX simiar to V86 but supports multiple 32b / 64b guests (no mention abou 16b). Then there's about VMM that is 32b and does some BIOS and 16b emulation to allow run DOS. So is this 32b VMM using hardware V86 mode or VTX instructions or does it sw emulated? And hypervison microkernel itself runs in 32b or 64b mode?
BTW Michal Sojka looks like Czech name, maybe I could find him and ask if he still involved in the project...
Georg Potthast
I had FreeDOS running as a guest in the Hypervisor. Detailed instructions how to do it in the tutorial. The virtual machine works like a 32bit processor did: starts in 16bit mode and can then be switched into protected/32bit mode by the operating system.
The PC has to run in 64bit mode or you will have no VT-X afaik. The Hypervisor was written to explore the capabilities of VT-X and uses this in hardware mode.
You can run V86 in an instance of a virtual machine based on VT-X. You need that e.g. to run Windows-XP in such a virtual machine.
Yes, Michael Sojka still works at the University of Dresden and is also in the group developing the new "userland". He is German.