RFC: PEtoBIN - a simple file format OS Development
Mike Gonta
http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4bae-9fde-d599bac8184a%2Fpecoff_v8.docx
file is a single segment flat read write executable.
With FASM
http://flatassembler.net
this is easy to create by preceding the code with:
format PE
This will create a 32 bit flat binary executable with the default
Windows ORG of 401000h. This file contains the 512 byte
PE header (which includes the MZ stub) and requires no linking.
format PE64
Will create a 64 bit executable.
The ORG can be specified as follows:
format PE at 100000h-1000h
Will set the ORG at 1 Mb for example.
By stripping off the 512 byte header a flat binary executable will
be produced.
This is where it gets interesting.
By following the code with:
section '.reloc' fixups
Will append a table of relocation information which can be used
by the loader to adjust the absolute address references so that
the executable can be located at a location other than the
default chosen.
At location 1B4h from the beginning of the header will be found
the 32 bit relative offset from the start of the header of this
relocation table. By appending this to the end of the file and
stripping off the 512 byte header a flat binary executable with
relocation information will be produced.
This header-less binary file (either 32 bit or 64 bit) can be loaded
and run from the default ORG as is. Subsequent loads can be
easily relocated by using the last 4 bytes of the file to locate
the relocation table.
Other assemblers/compilers should be able to create the simple
PE format which can then be modified to produce the desired
binary file.
A utility to do create a PEtoBIN should be straight forward to
create to produce a simple executable file format for OS
Development.
Mike Gonta
look and see - many look but few see
Mike Gonta
> The simplest PE COFFhttp://download.microsoft.com/download/9/c/5/9c5b2167-8017-4bae-9fde-...
> file is a single segment flat read write executable.
> this is easy to create by preceding the code with:
>
> format PE
>
> This will create a 32 bit flat binary executable with the default
> Windows ORG of 401000h. This file contains the 512 byte
> PE header (which includes the MZ stub) and requires no linking.
>
> format PE64
>
> Will create a 64 bit executable.
> The ORG can be specified as follows:
>
> format PE at 100000h-1000h
>
> Will set the ORG at 1 Mb for example.
> By stripping off the 512 byte header a flat binary executable will
> be produced.
>
> This is where it gets interesting.
> By following the code with:
>
> section '.reloc' fixups
>
> Will append a table of relocation information which can be used
> by the loader to adjust the absolute address references so that
> the executable can be located at a location other than the
> default chosen.
> At location 1B4h from the beginning of the header will be found
> the 32 bit relative offset from the start of the header of this
> relocation table. By appending this
value minus 512
Rod Pemberton
news:beda9572-edf5-43e2...@x21g2000yqa.googlegroups.com...
Ok.
> A utility to do create a PEtoBIN should be straight forward to
> create to produce a simple executable file format for OS
> Development.
>
Your idea is similar to how a COFF executable for DOS is created.
Unstubbing the 16-bit DPMI host from a DOS DPMI .exe created by DJGPP leaves
a 32-bit *executable* in COFF format - in the DJGPP variant of COFF. The
COFF executable is a linked version of the COFF object files.
RunReloc by Chris Giese is supposed to load Win32 PE COFF .o files into
memory and execute. I think it's for DOS.
For Linux environments, I suspect GCC is sufficient to create such a file.
For DOS environments, DJGPP uses GCC.
Topic deviation ahead...
There are a number of ways a file can be converted to a useful format for OS
development, especially for DOS:
16-bit DOS .com
DJGPP COFF executable - go32-v2 by Eli Zaretskii et. al. for unstubbed DJGPP
16-bit DOS DPMI
Zimage executables - Loadlin by Hans Lermen - e.g., Linux kernels
COFF .obj - Coff2Com by Alexei Frounze
mulitformat .o loader: Elf, DJGPP COFF, Win32 PE COFF .obj's - RunReloc by
Chris Giese
Multiboot loader - Mbload by Chris Giese
bootup COFF .obj (?) - OSLoader by Alexei Frounze
bootup DOS .com and .exe's - BootProg by Alexei Frounze
multiformat loader: COFF, Elf, Multiboot - X by Gerardo Lamastra et. al. -
FreeDOS-32
The Multiboot boot format using a Multiboot header supports Elf and a.out
formats for non-Elf. I not sure which the two above support.
Rod Pemberton
Mike Gonta
> "Mike Gonta" wrote:
> > A utility to do create a PEtoBIN should be straight forward to
> > create to produce a simple executable file format for OS
> > Development.
>
> Your idea is similar to how a COFF executable for DOS is created.
> Unstubbing the 16-bit DPMI host from a DOS DPMI .exe created by DJGPP leaves
> a 32-bit *executable* in COFF format - in the DJGPP variant of COFF. The
> COFF executable is a linked version of the COFF object files.
>
> RunReloc by Chris Giese is supposed to load Win32 PE COFF .o files into
> memory and execute. I think it's for DOS.
In a 32/64 bit system COFF needs to be relocated in order to run.
A single section (plus the relocation table fixup section) flat PE
with the
header stripped can be simply loaded and run from the preferred
location.
The 64 bit preferred address should be located next to last in the
file.
The relocation table file relative offset at the end of the file
should be 64 bit.
Both of these could be used in either system.
Currently PE and PE32+ are limited to 2 GB file size. 64 bit addresses
should be future protection if and when the format changes.
> For Linux environments, I suspect GCC is sufficient to create such a file.
> For DOS environments, DJGPP uses GCC.
>
> Topic deviation ahead...
> There are a number of ways a file can be converted to a useful format for OS
> development, especially for DOS:
The goal here is 32/64 bit OS Development.
Alexei A. Frounze
> In a 32/64 bit system COFF needs to be relocated in order to run.
It's possible to generate 64-bit position-independent (that is, RIP-
relative) code for x64. Do compilers not generate such 64-bit code?
Alex
Mike Gonta
> On Jul 25, 4:00 pm, Mike Gonta wrote:
>
> > In a 32/64 bit system COFF needs to be relocated in order to run.
At an address other than 0, which is the requirement of a simple
32 bit or 64 bit OS dev system without paging (32 bit) or
identity mapped paging (64 bit).
> It's possible to generate 64-bit position-independent (that is, RIP-
> relative) code for x64. Do compilers not generate such 64-bit code?
It's impossible to not generate 64 bit position independent code,
all x64 code is RIP relative.
Such code would suit applications but not system level code in a
simple OS dev system.
The compiler/assembler cannot know the load address
required to resolve an absolute linear address reference with
a COFF file which virtualizes the base address to 0.
When the COFF file is linked
(in the crippled assembler/compiler - linker toolset) the
result is not COFF or when ORG'ed with FASM (which
requires no linking to produce a simple executable) the
result is without relocation information.
Rod Pemberton
> On Jul 26, 5:23 am, "Alexei A. Frounze" wrote:
>
> > It's possible to generate 64-bit position-independent (that is, RIP-
> > relative) code for x64. Do compilers not generate such 64-bit code?
>
> It's impossible to not generate 64 bit position independent code,
> all x64 code is RIP relative.
>
... "all x64 code is RIP relative" ...
What?
AIUI, in 64-bit mode, you can use either RIP relative or absolute
addressing. RIP "steals" one of the ModRM encoding forms. Absolute
addressing is zero-based and encoded with the SIB format.
Rod Pemberton
Mike Gonta
> "Mike Gonta" wrote:
>
> > > It's possible to generate 64-bit position-independent (that is, RIP-
> > > relative) code for x64. Do compilers not generate such 64-bit code?
>
> > It's impossible to not generate 64 bit position independent code,
> > all x64 code is RIP relative.
>
> ... "all x64 code is RIP relative" ...
>
> What?
>
> AIUI, in 64-bit mode, you can use either RIP relative or absolute
> addressing.
Yes indeed, my mistake.
I wasn't paying attention to this paragraph from the FASM manual:
" The long mode uses also the instruction pointer based addresses,
you can specify it manually with the special RIP register symbol,
but such addressing is also automatically generated by flat
assembler,
since there is no 64-bit absolute addressing in long mode.
"
Of course this doesn't mean that there is no absolute addressing in
long mode (the source of my mistake) but rather (with one exception
which is explained in the line after next) there is no 64 bit form
(with the exception of a MOV to or from RAX from or to a 64 bit
address) of absolute addressing or displacement.
> RIP "steals" one of the ModRM encoding forms. Absolute
> addressing is zero-based and encoded with the SIB format.
Actually the ModRM encoding of the RIP relative 32 bit displacement
is that of the legacy 32 bit (absolute) displacement encoding. That's
the reason why it is the default in FASM, in order to get the old
form an assembler notation is required for the now different SIB
encoding. Other assemblers/compilers may require notation to get
the same (improved) ModRM encoded instruction.
Anyway, my original response was that even position independent
code will require some fixups when using a base 0 addressed
COFF file as an executable loaded at an address other than 0.
Maxim S. Shatskih
>code will require some fixups
It will surely require fixups on DLL import resolution, and not once in the import table, but on any occurence of the reference to the imported name.
Thus, for modern OSes, all this "position-independent" stuff is null and void. You need a relocation table anyway (unless you mandate the load address like for Windows KnownDLLs).
--
Maxim S. Shatskih
Windows DDK MVP
ma...@storagecraft.com
http://www.storagecraft.com
Mike Gonta
> Mike Gonta wrote:
> >Anyway, my original response was that even position independent
> >code will require some fixups
>
> It will surely require fixups on DLL import resolution, and not once in the import table, but on any occurence of the reference to the imported name.
All references to the imported name are indirect
(RIP relative in 64 bit mode) calls to the import table slot
which require no fixups.
> Thus, for modern OSes, all this "position-independent" stuff is null and void.
Definitely results in less fixups if required.
> You need a relocation table anyway (unless you mandate the load address like for Windows KnownDLLs).
The relocation table is required to relocate the
executable if it cannot be loaded at the preferred
location.
A simple header-less flat binary executable with a relocation
table, preferred load address (2nd to last) and a pointer
to the table as the last 64 bit value of the file is the
proposed format. Easy to produce from a PE and
easy to load. Designed for simple OS Development.